Rule based design of conceptual models for formative evaluation

NASA Technical Reports Server (NTRS)

Moore, Loretta A.; Chang, Kai; Hale, Joseph P.; Bester, Terri; Rix, Thomas; Wang, Yaowen

1994-01-01

A Human-Computer Interface (HCI) Prototyping Environment with embedded evaluation capability has been investigated. This environment will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. This environment, which allows for rapid prototyping and evaluation of graphical interfaces, includes the following four components: (1) a HCI development tool; (2) a low fidelity simulator development tool; (3) a dynamic, interactive interface between the HCI and the simulator; and (4) an embedded evaluator that evaluates the adequacy of a HCI based on a user's performance. The embedded evaluation tool collects data while the user is interacting with the system and evaluates the adequacy of an interface based on a user's performance. This paper describes the design of conceptual models for the embedded evaluation system using a rule-based approach.

Rule based design of conceptual models for formative evaluation

NASA Technical Reports Server (NTRS)

Moore, Loretta A.; Chang, Kai; Hale, Joseph P.; Bester, Terri; Rix, Thomas; Wang, Yaowen

1994-01-01

A Human-Computer Interface (HCI) Prototyping Environment with embedded evaluation capability has been investigated. This environment will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. This environment, which allows for rapid prototyping and evaluation of graphical interfaces, includes the following four components: (1) a HCI development tool, (2) a low fidelity simulator development tool, (3) a dynamic, interactive interface between the HCI and the simulator, and (4) an embedded evaluator that evaluates the adequacy of a HCI based on a user's performance. The embedded evaluation tool collects data while the user is interacting with the system and evaluates the adequacy of an interface based on a user's performance. This paper describes the design of conceptual models for the embedded evaluation system using a rule-based approach.

Recommendations and Improvements for the Evaluation of Integrated Community-Wide Interventions Approaches.

PubMed

van Koperen, Tessa M; Renders, Carry M; Spierings, Eline J M; Hendriks, Anna-Marie; Westerman, Marjan J; Seidell, Jacob C; Schuit, Albertine J

2016-01-01

Background . Integrated community-wide intervention approaches (ICIAs) are implemented to prevent childhood obesity. Programme evaluation improves these ICIAs, but professionals involved often struggle with performance. Evaluation tools have been developed to support Dutch professionals involved in ICIAs. It is unclear how useful these tools are to intended users. We therefore researched the facilitators of and barriers to ICIA programme evaluation as perceived by professionals and their experiences of the evaluation tools. Methods . Focus groups and interviews with 33 public health professionals. Data were analysed using a thematic content approach. Findings . Evaluation is hampered by insufficient time, budget, and experience with ICIAs, lack of leadership, and limited advocacy for evaluation. Epidemiologists are regarded as responsible for evaluation but feel incompetent to perform evaluation or advocate its need in a political environment. Managers did not prioritise process evaluations, involvement of stakeholders, and capacity building. The evaluation tools are perceived as valuable but too comprehensive considering limited resources. Conclusion . Evaluating ICIAs is important but most professionals are unfamiliar with it and management does not prioritise process evaluation nor incentivize professionals to evaluate. To optimise programme evaluation, more resources and coaching are required to improve professionals' evaluation capabilities and specifically the use of evaluation.

Recommendations and Improvements for the Evaluation of Integrated Community-Wide Interventions Approaches

PubMed Central

Spierings, Eline J. M.; Westerman, Marjan J.; Seidell, Jacob C.; Schuit, Albertine J.

2016-01-01

Background. Integrated community-wide intervention approaches (ICIAs) are implemented to prevent childhood obesity. Programme evaluation improves these ICIAs, but professionals involved often struggle with performance. Evaluation tools have been developed to support Dutch professionals involved in ICIAs. It is unclear how useful these tools are to intended users. We therefore researched the facilitators of and barriers to ICIA programme evaluation as perceived by professionals and their experiences of the evaluation tools. Methods. Focus groups and interviews with 33 public health professionals. Data were analysed using a thematic content approach. Findings. Evaluation is hampered by insufficient time, budget, and experience with ICIAs, lack of leadership, and limited advocacy for evaluation. Epidemiologists are regarded as responsible for evaluation but feel incompetent to perform evaluation or advocate its need in a political environment. Managers did not prioritise process evaluations, involvement of stakeholders, and capacity building. The evaluation tools are perceived as valuable but too comprehensive considering limited resources. Conclusion. Evaluating ICIAs is important but most professionals are unfamiliar with it and management does not prioritise process evaluation nor incentivize professionals to evaluate. To optimise programme evaluation, more resources and coaching are required to improve professionals' evaluation capabilities and specifically the use of evaluation. PMID:28116149

Syndromic surveillance of influenza activity in Sweden: an evaluation of three tools.

PubMed

Ma, T; Englund, H; Bjelkmar, P; Wallensten, A; Hulth, A

2015-08-01

An evaluation was conducted to determine which syndromic surveillance tools complement traditional surveillance by serving as earlier indicators of influenza activity in Sweden. Web queries, medical hotline statistics, and school absenteeism data were evaluated against two traditional surveillance tools. Cross-correlation calculations utilized aggregated weekly data for all-age, nationwide activity for four influenza seasons, from 2009/2010 to 2012/2013. The surveillance tool indicative of earlier influenza activity, by way of statistical and visual evidence, was identified. The web query algorithm and medical hotline statistics performed equally well as each other and to the traditional surveillance tools. School absenteeism data were not reliable resources for influenza surveillance. Overall, the syndromic surveillance tools did not perform with enough consistency in season lead nor in earlier timing of the peak week to be considered as early indicators. They do, however, capture incident cases before they have formally entered the primary healthcare system.

Cost analysis of objective resident cataract surgery assessments.

PubMed

Nandigam, Kiran; Soh, Jonathan; Gensheimer, William G; Ghazi, Ahmed; Khalifa, Yousuf M

2015-05-01

To compare 8 ophthalmology resident surgical training tools to determine which is most cost effective. University of Rochester Medical Center, Rochester, New York, USA. Retrospective evaluation of technology. A cost-analysis model was created to compile all relevant costs in running each tool in a medium-sized ophthalmology program. Quantitative cost estimates were obtained based on cost of tools, cost of time in evaluations, and supply and maintenance costs. For wet laboratory simulation, Eyesi was the least expensive cataract surgery simulation method; however, it is only capable of evaluating simulated cataract surgery rehearsal and requires supplementation with other evaluative methods for operating room performance and for noncataract wet lab training and evaluation. The most expensive training tool was the Eye Surgical Skills Assessment Test (ESSAT). The 2 most affordable methods for resident evaluation in operating room performance were the Objective Assessment of Skills in Intraocular Surgery (OASIS) and Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Cost-based analysis of ophthalmology resident surgical training tools are needed so residency programs can implement tools that are valid, reliable, objective, and cost effective. There is no perfect training system at this time. Copyright © 2015 ASCRS and ESCRS. Published by Elsevier Inc. All rights reserved.

Programming Tools: Status, Evaluation, and Comparison

NASA Technical Reports Server (NTRS)

Cheng, Doreen Y.; Cooper, D. M. (Technical Monitor)

1994-01-01

In this tutorial I will first describe the characteristics of scientific applications and their developers, and describe the computing environment in a typical high-performance computing center. I will define the user requirements for tools that support application portability and present the difficulties to satisfy them. These form the basis of the evaluation and comparison of the tools. I will then describe the tools available in the market and the tools available in the public domain. Specifically, I will describe the tools for converting sequential programs, tools for developing portable new programs, tools for debugging and performance tuning, tools for partitioning and mapping, and tools for managing network of resources. I will introduce the main goals and approaches of the tools, and show main features of a few tools in each category. Meanwhile, I will compare tool usability for real-world application development and compare their different technological approaches. Finally, I will indicate the future directions of the tools in each category.

Green and sustainable remediation (GSR) evaluation: framework, standards, and tool. A case study in Taiwan.

PubMed

Huang, Wen-Yen; Hung, Weiteng; Vu, Chi Thanh; Chen, Wei-Ting; Lai, Jhih-Wei; Lin, Chitsan

2016-11-01

Taiwan has a large number of poorly managed contaminated sites in need of remediation. This study proposes a framework, a set of standards, and a spreadsheet-based evaluation tool for implementing green and sustainable principles into remediation projects and evaluating the projects from this perspective. We performed a case study to understand how the framework would be applied. For the case study, we used a spreadsheet-based evaluation tool (SEFA) and performed field scale cultivation tests on a site contaminated with total petroleum hydrocarbons (TPHs). The site was divided into two lots: one treated by chemical oxidation and the other by bioremediation. We evaluated five core elements of green and sustainable remediation (GSR): energy, air, water resources, materials and wastes, and land and ecosystem. The proposed evaluation tool and field scale cultivation test were found to efficiently assess the effectiveness of the two remediation alternatives. The framework and related tools proposed herein can potentially be used to support decisions about the remediation of contaminated sites taking into account engineering management, cost effectiveness, and social reconciliation.

The SAFE-T assessment tool: derivation and validation of a web-based application for point-of-care evaluation of gastroenterology fellow performance in colonoscopy.

PubMed

Kumar, Navin L; Kugener, Guillaume; Perencevich, Molly L; Saltzman, John R

2018-01-01

Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise assessment tool to evaluate real-time fellow performance in colonoscopy administered via a web-based application. The Skill Assessment in Fellow Endoscopy Training (SAFE-T) tool was derived as a novel 5-question evaluation tool that captures both summative and formative feedback adapted into a web-based application. A prospective study of 15 gastroenterology fellows (5 fellows each from years 1 to 3 of training) was performed using the SAFE-T tool. An independent reviewer evaluated a subset of these procedures and completed the SAFE-T tool and Mayo Colonoscopy Skills Assessment Tool (MCSAT) for reliability testing. Twenty-six faculty completed 350 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score (year 1, 2.00; year 2, 3.84; year 3, 4.28) differentiated each sequential fellow year of training (P < .0001). The mean SAFE-T overall score decreased with increasing case complexity score, with straightforward cases compared with average cases (4.07 vs 3.50, P < .0001), and average cases compared with challenging cases (3.50 vs 3.08, P = .0134). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a kappa agreement statistic of 0.898 (P < .0001). Correlation of the SAFE-T overall score with the MCSAT overall hands-on and individual motor scores was excellent (each r > 0.90, P < .0001). We developed and validated the SAFE-T assessment tool, a concise and web-based means of assessing real-time gastroenterology fellow performance in colonoscopy. Copyright © 2018 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

Mutation extraction tools can be combined for robust recognition of genetic variants in the literature

PubMed Central

Jimeno Yepes, Antonio; Verspoor, Karin

2014-01-01

As the cost of genomic sequencing continues to fall, the amount of data being collected and studied for the purpose of understanding the genetic basis of disease is increasing dramatically. Much of the source information relevant to such efforts is available only from unstructured sources such as the scientific literature, and significant resources are expended in manually curating and structuring the information in the literature. As such, there have been a number of systems developed to target automatic extraction of mutations and other genetic variation from the literature using text mining tools. We have performed a broad survey of the existing publicly available tools for extraction of genetic variants from the scientific literature. We consider not just one tool but a number of different tools, individually and in combination, and apply the tools in two scenarios. First, they are compared in an intrinsic evaluation context, where the tools are tested for their ability to identify specific mentions of genetic variants in a corpus of manually annotated papers, the Variome corpus. Second, they are compared in an extrinsic evaluation context based on our previous study of text mining support for curation of the COSMIC and InSiGHT databases. Our results demonstrate that no single tool covers the full range of genetic variants mentioned in the literature. Rather, several tools have complementary coverage and can be used together effectively. In the intrinsic evaluation on the Variome corpus, the combined performance is above 0.95 in F-measure, while in the extrinsic evaluation the combined recall performance is above 0.71 for COSMIC and above 0.62 for InSiGHT, a substantial improvement over the performance of any individual tool. Based on the analysis of these results, we suggest several directions for the improvement of text mining tools for genetic variant extraction from the literature. PMID:25285203

Development and validation of a Clinical Assessment Tool for Nursing Education (CAT-NE).

PubMed

Skúladóttir, Hafdís; Svavarsdóttir, Margrét Hrönn

2016-09-01

The aim of this study was to develop a valid assessment tool to guide clinical education and evaluate students' performance in clinical nursing education. The development of the Clinical Assessment Tool for Nursing Education (CAT-NE) was based on the theory of nursing as professional caring and the Bologna learning outcomes. Benson and Clark's four steps of instrument development and validation guided the development and assessment of the tool. A mixed-methods approach with individual structured cognitive interviewing and quantitative assessments was used to validate the tool. Supervisory teachers, a pedagogical consultant, clinical expert teachers, clinical teachers, and nursing students at the University of Akureyri in Iceland participated in the process. This assessment tool is valid to assess the clinical performance of nursing students; it consists of rubrics that list the criteria for the students' expected performance. According to the students and their clinical teachers, the assessment tool clarified learning objectives, enhanced the focus of the assessment process, and made evaluation more objective. Training clinical teachers on how to assess students' performances in clinical studies and use the tool enhanced the quality of clinical assessment in nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

Teaching leadership in trauma resuscitation: Immediate feedback from a real-time, competency-based evaluation tool shows long-term improvement in resident performance.

PubMed

Gregg, Shea C; Heffernan, Daithi S; Connolly, Michael D; Stephen, Andrew H; Leuckel, Stephanie N; Harrington, David T; Machan, Jason T; Adams, Charles A; Cioffi, William G

2016-10-01

Limited data exist on how to develop resident leadership and communication skills during actual trauma resuscitations. An evaluation tool was developed to grade senior resident performance as the team leader during full-trauma-team activations. Thirty actions that demonstrated the Accreditation Council for Graduate Medical Education core competencies were graded on a Likert scale of 1 (poor) to 5 (exceptional). These actions were grouped by their respective core competencies on 5 × 7-inch index cards. In Phase 1, baseline performance scores were obtained. In Phase 2, trauma-focused communication in-services were conducted early in the academic year, and immediate, personalized feedback sessions were performed after resuscitations based on the evaluation tool. In Phase 3, residents received only evaluation-based feedback following resuscitations. In Phase 1 (October 2009 to April 2010), 27 evaluations were performed on 10 residents. In Phase 2 (April 2010 to October 2010), 28 evaluations were performed on nine residents. In Phase 3 (October 2010 to January 2012), 44 evaluations were performed on 13 residents. Total scores improved significantly between Phases 1 and 2 (p = 0.003) and remained elevated throughout Phase 3. When analyzing performance by competency, significant improvement between Phases 1 and 2 (p < 0.05) was seen in all competencies (patient care, knowledge, system-based practice, practice-based learning) with the exception of "communication and professionalism" (p = 0.56). Statistically similar scores were observed between Phases 2 and 3 in all competencies with the exception of "medical knowledge," which showed ongoing significant improvement (p = 0.003). Directed resident feedback sessions utilizing data from a real-time, competency-based evaluation tool have allowed us to improve our residents' abilities to lead trauma resuscitations over a 30-month period. Given pressures to maximize clinical educational opportunities among work-hour constraints, such a model may help decrease the need for costly simulation-based training. Therapeutic study, level III.

Performance characteristics of five triage tools for major incidents involving traumatic injuries to children.

PubMed

Price, C L; Brace-McDonnell, S J; Stallard, N; Bleetman, A; Maconochie, I; Perkins, G D

2016-05-01

Context Triage tools are an essential component of the emergency response to a major incident. Although fortunately rare, mass casualty incidents involving children are possible which mandate reliable triage tools to determine the priority of treatment. To determine the performance characteristics of five major incident triage tools amongst paediatric casualties who have sustained traumatic injuries. Retrospective observational cohort study using data from 31,292 patients aged less than 16 years who sustained a traumatic injury. Data were obtained from the UK Trauma Audit and Research Network (TARN) database. Interventions Statistical evaluation of five triage tools (JumpSTART, START, CareFlight, Paediatric Triage Tape/Sieve and Triage Sort) to predict death or severe traumatic injury (injury severity score >15). Main outcome measures Performance characteristics of triage tools (sensitivity, specificity and level of agreement between triage tools) to identify patients at high risk of death or severe injury. Of the 31,292 cases, 1029 died (3.3%), 6842 (21.9%) had major trauma (defined by an injury severity score >15) and 14,711 (47%) were aged 8 years or younger. There was variation in the performance accuracy of the tools to predict major trauma or death (sensitivities ranging between 36.4 and 96.2%; specificities 66.0-89.8%). Performance characteristics varied with the age of the child. CareFlight had the best overall performance at predicting death, with the following sensitivity and specificity (95% CI) respectively: 95.3% (93.8-96.8) and 80.4% (80.0-80.9). JumpSTART was superior for the triaging of children under 8 years; sensitivity and specificity (95% CI) respectively: 86.3% (83.1-89.5) and 84.8% (84.2-85.5). The triage tools were generally better at identifying patients who would die than those with non-fatal severe injury. This statistical evaluation has demonstrated variability in the accuracy of triage tools at predicting outcomes for children who sustain traumatic injuries. No single tool performed consistently well across all evaluated scenarios. Copyright © 2015 Elsevier Ltd. All rights reserved.

Model Performance Evaluation and Scenario Analysis (MPESA)

EPA Pesticide Factsheets

Model Performance Evaluation and Scenario Analysis (MPESA) assesses the performance with which models predict time series data. The tool was developed Hydrological Simulation Program-Fortran (HSPF) and the Stormwater Management Model (SWMM)

77 FR 4638 - Defense Federal Acquisition Regulation Supplement; Performance-Based Payments (DFARS Case 2011-D045)

Federal Register 2010, 2011, 2012, 2013, 2014

2012-01-30

... tool. The PBP analysis tool is a cash-flow model for evaluating alternative financing arrangements, and... PBP analysis tool is a cash-flow model for evaluating alternative financing arrangements, and is... that reflects adequate consideration to the Government for the improved contractor cash flow...

Estimating learning outcomes from pre- and posttest student self-assessments: a longitudinal study.

PubMed

Schiekirka, Sarah; Reinhardt, Deborah; Beißbarth, Tim; Anders, Sven; Pukrop, Tobias; Raupach, Tobias

2013-03-01

Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course. In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives. Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains. The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.

Professional Development through Organizational Assessment: Using APPA's Facilities Management Evaluation Program

ERIC Educational Resources Information Center

Medlin, E. Lander; Judd, R. Holly

2013-01-01

APPA's Facilities Management Evaluation Program (FMEP) provides an integrated system to optimize organizational performance. The criteria for evaluation not only provide a tool for organizational continuous improvement, they serve as a compelling leadership development tool essential for today's facilities management professional. The senior…

DECIDE: a software for computer-assisted evaluation of diagnostic test performance.

PubMed

Chiecchio, A; Bo, A; Manzone, P; Giglioli, F

1993-05-01

The evaluation of the performance of clinical tests is a complex problem involving different steps and many statistical tools, not always structured in an organic and rational system. This paper presents a software which provides an organic system of statistical tools helping evaluation of clinical test performance. The program allows (a) the building and the organization of a working database, (b) the selection of the minimal set of tests with the maximum information content, (c) the search of the model best fitting the distribution of the test values, (d) the selection of optimal diagnostic cut-off value of the test for every positive/negative situation, (e) the evaluation of performance of the combinations of correlated and uncorrelated tests. The uncertainty associated with all the variables involved is evaluated. The program works in a MS-DOS environment with EGA or higher performing graphic card.

Investigation of Formative Assessment of Learning (INFORMAL): The Performance Indicator Tool (PIT)

ERIC Educational Resources Information Center

Dutton, Philip J.; Bickerstaff, Helen E.; Rymer, Janice M.; Webb, Mary E.; Ballinger-Mills, Deborah; Greenough, Anne; Reynolds, Patricia A.

2017-01-01

This pilot study evaluated the student user experience of a performance indicator tool (PIT) for formative assessment of a cohort of fourth year medical undergraduates undertaking a 12 week rotation. Scoping and elaboration for the standalone web-based tool was based on the existing paper-based formative assessment system for which training and…

Evaluation of Process Performance for Sustainable Hard Machining

NASA Astrophysics Data System (ADS)

Rotella, Giovanna; Umbrello, Domenico; , Oscar W. Dillon, Jr.; Jawahir, I. S.

This paper aims to evaluate the sustainability performance of machining operation of through-hardening steel, AISI 52100, taking into account the impact of the material removal process in its various aspects. Experiments were performed for dry and cryogenic cutting conditions using chamfered cubic boron nitride (CBN) tool inserts at varying cutting conditions (cutting speed and feed rate). Cutting forces, mechanical power, tool wear, white layer thickness, surface roughness and residual stresses were investigated in order to evaluate the effects of extreme in-process cooling on the machined surface. The results indicate that cryogenic cooling has the potential to be used for surface integrity enhancement for improved product life and more sustainable functional performance.

Financial Statement Analysis for Colleges and Universities.

ERIC Educational Resources Information Center

Woelfel, Charles J.

1987-01-01

Presents ratio analysis of financial statements as a tool applicable for use by nonprofit institutions for evaluation of financial and operational performance of an institution. It can be used as a screening, forecasting, diagnostic, and evaluative tool for administration and governance. (MD)

Community-led Air Sensor Evaluation: New Tools for Citizen Scientists Fact Sheet

EPA Pesticide Factsheets

EPA has developed a guide and analysis tool for citizen scientists to evaluate the performance of low-cost sensors and interpret the data they collect to help citizen scientists interested in learning about local air quality.

Human Factors Evaluations of Two-Dimensional Spacecraft Conceptual Layouts

NASA Technical Reports Server (NTRS)

Kennedy, Kriss J.; Toups, Larry D.; Rudisill, Marianne

2010-01-01

Much of the human factors work done in support of the NASA Constellation lunar program has been with low fidelity mockups. These volumetric replicas of the future lunar spacecraft allow researchers to insert test subjects from the engineering and astronaut population and evaluate the vehicle design as the test subjects perform simulations of various operational tasks. However, lunar outpost designs must be evaluated without the use of mockups, creating a need for evaluation tools that can be performed on two-dimension conceptual spacecraft layouts, such as floor plans. A tool based on the Cooper- Harper scale was developed and applied to one lunar scenario, enabling engineers to select between two competing floor plan layouts. Keywords: Constellation, human factors, tools, processes, habitat, outpost, Net Habitable Volume, Cooper-Harper.

SU-E-J-102: Performance Variations Among Clinically Available Deformable Image Registration Tools in Adaptive Radiotherapy: How Should We Evaluate and Interpret the Result?

DOE Office of Scientific and Technical Information (OSTI.GOV)

Nie, K; Pouliot, J; Smith, E

Purpose: To evaluate the performance variations in commercial deformable image registration (DIR) tools for adaptive radiation therapy. Methods: Representative plans from three different anatomical sites, prostate, head-and-neck (HN) and cranial spinal irradiation (CSI) with L-spine boost, were included. Computerized deformed CT images were first generated using virtual DIR QA software (ImSimQA) for each case. The corresponding transformations served as the “reference”. Three commercial software packages MIMVista v5.5 and MIMMaestro v6.0, VelocityAI v2.6.2, and OnQ rts v2.1.15 were tested. The warped contours and doses were compared with the “reference” and among each other. Results: The performance in transferring contours was comparablemore » among all three tools with an average DICE coefficient of 0.81 for all the organs. However, the performance of dose warping accuracy appeared to rely on the evaluation end points. Volume based DVH comparisons were not sensitive enough to illustrate all the detailed variations while isodose assessment on a slice-by-slice basis could be tedious. Point-based evaluation was over-sensitive by having up to 30% hot/cold-spot differences. If adapting the 3mm/3% gamma analysis into the evaluation of dose warping, all three algorithms presented a reasonable level of equivalency. One algorithm had over 10% of the voxels not meeting this criterion for the HN case while another showed disagreement for the CSI case. Conclusion: Overall, our results demonstrated that evaluation based only on the performance of contour transformation could not guarantee the accuracy in dose warping. However, the performance of dose warping accuracy relied on the evaluation methodologies. Nevertheless, as more DIR tools are available for clinical use, the performance could vary at certain degrees. A standard quality assurance criterion with clinical meaning should be established for DIR QA, similar to the gamma index concept, in the near future.« less

Knowledge-Acquisition Tool For Expert System

NASA Technical Reports Server (NTRS)

Disbrow, James D.; Duke, Eugene L.; Regenie, Victoria A.

1988-01-01

Digital flight-control systems monitored by computer program that evaluates and recommends. Flight-systems engineers for advanced, high-performance aircraft use knowlege-acquisition tool for expert-system flight-status monitor suppling interpretative data. Interpretative function especially important in time-critical, high-stress situations because it facilitates problem identification and corrective strategy. Conditions evaluated and recommendations made by ground-based engineers having essential knowledge for analysis and monitoring of performances of advanced aircraft systems.

Diagnostic Tools for Performance Evaluation of Innovative In-Situ Remediation Technologies at Chlorinated Solvent-Contaminated Sites

DTIC Science & Technology

2011-07-01

to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT...these innovative methods with conventional diagnostic tools that are currently used for assessing bioremediation performance. 132 Rula Deeb (510) 596...conventional diagnostic tools that are currently used for assessing bioremediation performance. DEMONSTRATION RESULTS 3-D multi-level systems

Multi-modal virtual environment research at Armstrong Laboratory

NASA Technical Reports Server (NTRS)

Eggleston, Robert G.

1995-01-01

One mission of the Paul M. Fitts Human Engineering Division of Armstrong Laboratory is to improve the user interface for complex systems through user-centered exploratory development and research activities. In support of this goal, many current projects attempt to advance and exploit user-interface concepts made possible by virtual reality (VR) technologies. Virtual environments may be used as a general purpose interface medium, an alternative display/control method, a data visualization and analysis tool, or a graphically based performance assessment tool. An overview is given of research projects within the division on prototype interface hardware/software development, integrated interface concept development, interface design and evaluation tool development, and user and mission performance evaluation tool development.

Optimizing the ASC WAN: evaluating network performance tools for comparing transport protocols.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lydick, Christopher L.

2007-07-01

The Advanced Simulation & Computing Wide Area Network (ASC WAN), which is a high delay-bandwidth network connection between US Department of Energy National Laboratories, is constantly being examined and evaluated for efficiency. One of the current transport-layer protocols which is used, TCP, was developed for traffic demands which are different from that on the ASC WAN. The Stream Control Transport Protocol (SCTP), on the other hand, has shown characteristics which make it more appealing to networks such as these. Most important, before considering a replacement for TCP on any network, a testing tool that performs well against certain criteria needsmore » to be found. In order to try to find such a tool, two popular networking tools (Netperf v.2.4.3 & v.2.4.6 (OpenSS7 STREAMS), and Iperf v.2.0.6) were tested. These tools implement both TCP and SCTP and were evaluated using four metrics: (1) How effectively can the tool reach a throughput near the bandwidth? (2) How much of the CPU does the tool utilize during operation? (3) Is the tool freely and widely available? And, (4) Is the tool actively developed? Following the analysis of those tools, this paper goes further into explaining some recommendations and ideas for future work.« less

An Object-Based Approach to Evaluation of Climate Variability Projections and Predictions

NASA Astrophysics Data System (ADS)

Ammann, C. M.; Brown, B.; Kalb, C. P.; Bullock, R.

2017-12-01

Evaluations of the performance of earth system model predictions and projections are of critical importance to enhance usefulness of these products. Such evaluations need to address specific concerns depending on the system and decisions of interest; hence, evaluation tools must be tailored to inform about specific issues. Traditional approaches that summarize grid-based comparisons of analyses and models, or between current and future climate, often do not reveal important information about the models' performance (e.g., spatial or temporal displacements; the reason behind a poor score) and are unable to accommodate these specific information needs. For example, summary statistics such as the correlation coefficient or the mean-squared error provide minimal information to developers, users, and decision makers regarding what is "right" and "wrong" with a model. New spatial and temporal-spatial object-based tools from the field of weather forecast verification (where comparisons typically focus on much finer temporal and spatial scales) have been adapted to more completely answer some of the important earth system model evaluation questions. In particular, the Method for Object-based Diagnostic Evaluation (MODE) tool and its temporal (three-dimensional) extension (MODE-TD) have been adapted for these evaluations. More specifically, these tools can be used to address spatial and temporal displacements in projections of El Nino-related precipitation and/or temperature anomalies, ITCZ-associated precipitation areas, atmospheric rivers, seasonal sea-ice extent, and other features of interest. Examples of several applications of these tools in a climate context will be presented, using output of the CESM large ensemble. In general, these tools provide diagnostic information about model performance - accounting for spatial, temporal, and intensity differences - that cannot be achieved using traditional (scalar) model comparison approaches. Thus, they can provide more meaningful information that can be used in decision-making and planning. Future extensions and applications of these tools in a climate context will be considered.

Assessment Tools for the Evaluation of Risk

EPA Science Inventory

ASTER (Assessment Tools for the Evaluation of Risk) was developed by the U.S. EPA Mid-Continent Ecology Division, Duluth, MN to assist regulators in performing ecological risk assessments. ASTER is an integration of the ECOTOXicology Database (ECOTOX; A Tool for the Automated Design and Evaluation of Habitat Interior Layouts

NASA Technical Reports Server (NTRS)

Simon, Matthew A.; Wilhite, Alan W.

2013-01-01

The objective of space habitat design is to minimize mass and system size while providing adequate space for all necessary equipment and a functional layout that supports crew health and productivity. Unfortunately, development and evaluation of interior layouts is often ignored during conceptual design because of the subjectivity and long times required using current evaluation methods (e.g., human-in-the-loop mockup tests and in-depth CAD evaluations). Early, more objective assessment could prevent expensive design changes that may increase vehicle mass and compromise functionality. This paper describes a new interior design evaluation method to enable early, structured consideration of habitat interior layouts. This interior layout evaluation method features a comprehensive list of quantifiable habitat layout evaluation criteria, automatic methods to measure these criteria from a geometry model, and application of systems engineering tools and numerical methods to construct a multi-objective value function measuring the overall habitat layout performance. In addition to a detailed description of this method, a C++/OpenGL software tool which has been developed to implement this method is also discussed. This tool leverages geometry modeling coupled with collision detection techniques to identify favorable layouts subject to multiple constraints and objectives (e.g., minimize mass, maximize contiguous habitable volume, maximize task performance, and minimize crew safety risks). Finally, a few habitat layout evaluation examples are described to demonstrate the effectiveness of this method and tool to influence habitat design.

Validation of Lower Tier Exposure Tools Used for REACH: Comparison of Tools Estimates With Available Exposure Measurements.

PubMed

van Tongeren, Martie; Lamb, Judith; Cherrie, John W; MacCalman, Laura; Basinas, Ioannis; Hesse, Susanne

2017-10-01

Tier 1 exposure tools recommended for use under REACH are designed to easily identify situations that may pose a risk to health through conservative exposure predictions. However, no comprehensive evaluation of the performance of the lower tier tools has previously been carried out. The ETEAM project aimed to evaluate several lower tier exposure tools (ECETOC TRA, MEASE, and EMKG-EXPO-TOOL) as well as one higher tier tool (STOFFENMANAGER®). This paper describes the results of the external validation of tool estimates using measurement data. Measurement data were collected from a range of providers, both in Europe and United States, together with contextual information. Individual measurement and aggregated measurement data were obtained. The contextual information was coded into the tools to obtain exposure estimates. Results were expressed as percentage of measurements exceeding the tool estimates and presented by exposure category (non-volatile liquid, volatile liquid, metal abrasion, metal processing, and powder handling). We also explored tool performance for different process activities as well as different scenario conditions and exposure levels. In total, results from nearly 4000 measurements were obtained, with the majority for the use of volatile liquids and powder handling. The comparisons of measurement results with tool estimates suggest that the tools are generally conservative. However, the tools were more conservative when estimating exposure from powder handling compared to volatile liquids and other exposure categories. In addition, results suggested that tool performance varies between process activities and scenario conditions. For example, tools were less conservative when estimating exposure during activities involving tabletting, compression, extrusion, pelletisation, granulation (common process activity PROC14) and transfer of substance or mixture (charging and discharging) at non-dedicated facilities (PROC8a; powder handling only). With the exception of STOFFENMANAGER® (for estimating exposure during powder handling), the tools were less conservative for scenarios with lower estimated exposure levels. This is the most comprehensive evaluation of the performance of REACH exposure tools carried out to date. The results show that, although generally conservative, the tools may not always achieve the performance specified in the REACH guidance, i.e. using the 75th or 90th percentile of the exposure distribution for the risk characterisation. Ongoing development, adjustment, and recalibration of the tools with new measurement data are essential to ensure adequate characterisation and control of worker exposure to hazardous substances. © The Author 2017. Published by Oxford University Press on behalf of the British Occupational Hygiene Society.

An automated benchmarking platform for MHC class II binding prediction methods.

PubMed

Andreatta, Massimo; Trolle, Thomas; Yan, Zhen; Greenbaum, Jason A; Peters, Bjoern; Nielsen, Morten

2018-05-01

Computational methods for the prediction of peptide-MHC binding have become an integral and essential component for candidate selection in experimental T cell epitope discovery studies. The sheer amount of published prediction methods-and often discordant reports on their performance-poses a considerable quandary to the experimentalist who needs to choose the best tool for their research. With the goal to provide an unbiased, transparent evaluation of the state-of-the-art in the field, we created an automated platform to benchmark peptide-MHC class II binding prediction tools. The platform evaluates the absolute and relative predictive performance of all participating tools on data newly entered into the Immune Epitope Database (IEDB) before they are made public, thereby providing a frequent, unbiased assessment of available prediction tools. The benchmark runs on a weekly basis, is fully automated, and displays up-to-date results on a publicly accessible website. The initial benchmark described here included six commonly used prediction servers, but other tools are encouraged to join with a simple sign-up procedure. Performance evaluation on 59 data sets composed of over 10 000 binding affinity measurements suggested that NetMHCIIpan is currently the most accurate tool, followed by NN-align and the IEDB consensus method. Weekly reports on the participating methods can be found online at: http://tools.iedb.org/auto_bench/mhcii/weekly/. mniel@bioinformatics.dtu.dk. Supplementary data are available at Bioinformatics online.

System Life Cycle Evaluation(SM) (SLiCE): harmonizing water treatment systems with implementers' needs.

PubMed

Goodman, Joseph; Caravati, Kevin; Foote, Andrew; Nelson, Molly; Woods, Emily

2013-06-01

One of the methods proposed to improve access to clean drinking water is the mobile packaged water treatment system (MPWTS). The lack of published system performance comparisons combined with the diversity of technology available and intended operating conditions make it difficult for stakeholders to choose the system best suited for their application. MPWTS are often deployed in emergency situations, making selection of the appropriate system crucial to avoiding wasted resources and loss of life. Measurable critical-to-quality characteristics (CTQs) and a system selection tool for MPWTS were developed by utilizing relevant literature, including field studies, and implementing and comparing seven different MPWTS. The proposed System Life Cycle Evaluation (SLiCE) method uses these CTQs to evaluate the diversity in system performance and harmonize relevant performance with stakeholder preference via a selection tool. Agencies and field workers can use SLiCE results to inform and drive decision-making. The evaluation and selection tool also serves as a catalyst for communicating system performance, common design flaws, and stakeholder needs to system manufacturers. The SLiCE framework can be adopted into other emerging system technologies to communicate system performance over the life cycle of use.

TEMPORALLY-RESOLVED AMMONIA EMISSION INVENTORIES: CURRENT ESTIMATES, EVALUATION TOOLS, AND MEASUREMENT NEEDS

EPA Science Inventory

In this study, we evaluate the suitability of a three-dimensional chemical transport model (CTM) as a tool for assessing ammonia emission inventories, calculate the improvement in CTM performance owing to recent advances in temporally-varying ammonia emission estimates, and ident...

Technology Evaluation Tools and Teacher Performance in Public Schools

ERIC Educational Resources Information Center

Stonehouse, Pauline; Keengwe, Jared

2013-01-01

The purpose of this study was, (a) to describe the introduction of mVAL software and Charlotte Danielson Rubrics (CDR) as teacher evaluation tools; (b) to compare the process and outcomes of the new initiative with traditional systems; and (c) to evaluate the software from the perspective of participants in the system. This study highlights the…

A new framework for sustainable hydropower development project

NASA Astrophysics Data System (ADS)

Johan, Kartina; Turan, Faiz Mohd; Gani, Nur Syazwani Abdul

2018-03-01

This project studies on the establishment of a new framework for sustainable hydropower development. A hydropower development is listed as one of the prescribed activities under the Environmental Quality Order 1987. Thus, Environmental Impact Assessment (EIA) guidelines must be referred to comply with the Department of Environment (DoE) requirements. In order to execute EIA, an assessment tool that will be utilized in the final evaluation phase must be determined. The selected assessment tool that will be used is Systematic Sustainability Assessment(SSA) which is a new integrated tool to evaluate the sustainability performance. A pilot run is conducted in five different departments within the Energy Company to validate the efficiency of the SSA tool. The parameters to be evaluated are constructed aligned with the Sustainable Development Goals (SDG) to maintain the sustainability features. Consequently, the performance level of the sustainability with respect to People, Planet and Profit (3P’s) is able to be discovered during evaluation phase in the hydropower development for continuous improvement.

Performance evaluation of the Engineering Analysis and Data Systems (EADS) 2

NASA Technical Reports Server (NTRS)

Debrunner, Linda S.

1994-01-01

The Engineering Analysis and Data System (EADS)II (1) was installed in March 1993 to provide high performance computing for science and engineering at Marshall Space Flight Center (MSFC). EADS II increased the computing capabilities over the existing EADS facility in the areas of throughput and mass storage. EADS II includes a Vector Processor Compute System (VPCS), a Virtual Memory Compute System (CFS), a Common Output System (COS), as well as Image Processing Station, Mini Super Computers, and Intelligent Workstations. These facilities are interconnected by a sophisticated network system. This work considers only the performance of the VPCS and the CFS. The VPCS is a Cray YMP. The CFS is implemented on an RS 6000 using the UniTree Mass Storage System. To better meet the science and engineering computing requirements, EADS II must be monitored, its performance analyzed, and appropriate modifications for performance improvement made. Implementing this approach requires tool(s) to assist in performance monitoring and analysis. In Spring 1994, PerfStat 2.0 was purchased to meet these needs for the VPCS and the CFS. PerfStat(2) is a set of tools that can be used to analyze both historical and real-time performance data. Its flexible design allows significant user customization. The user identifies what data is collected, how it is classified, and how it is displayed for evaluation. Both graphical and tabular displays are supported. The capability of the PerfStat tool was evaluated, appropriate modifications to EADS II to optimize throughput and enhance productivity were suggested and implemented, and the effects of these modifications on the systems performance were observed. In this paper, the PerfStat tool is described, then its use with EADS II is outlined briefly. Next, the evaluation of the VPCS, as well as the modifications made to the system are described. Finally, conclusions are drawn and recommendations for future worked are outlined.

New Automotive Air Conditioning System Simulation Tool Developed in MATLAB/Simulink

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kiss, T.; Chaney, L.; Meyer, J.

Further improvements in vehicle fuel efficiency require accurate evaluation of the vehicle's transient total power requirement. When operated, the air conditioning (A/C) system is the largest auxiliary load on a vehicle; therefore, accurate evaluation of the load it places on the vehicle's engine and/or energy storage system is especially important. Vehicle simulation software, such as 'Autonomie,' has been used by OEMs to evaluate vehicles' energy performance. A transient A/C simulation tool incorporated into vehicle simulation models would also provide a tool for developing more efficient A/C systems through a thorough consideration of the transient A/C system performance. The dynamic systemmore » simulation software Matlab/Simulink was used to develop new and more efficient vehicle energy system controls. The various modeling methods used for the new simulation tool are described in detail. Comparison with measured data is provided to demonstrate the validity of the model.« less

ATAMM enhancement and multiprocessor performance evaluation

NASA Technical Reports Server (NTRS)

Stoughton, John W.; Mielke, Roland R.; Som, Sukhamoy; Obando, Rodrigo; Malekpour, Mahyar R.; Jones, Robert L., III; Mandala, Brij Mohan V.

1991-01-01

ATAMM (Algorithm To Architecture Mapping Model) enhancement and multiprocessor performance evaluation is discussed. The following topics are included: the ATAMM model; ATAMM enhancement; ADM (Advanced Development Model) implementation of ATAMM; and ATAMM support tools.

National trends in safety performance of electronic health record systems in children's hospitals.

PubMed

Chaparro, Juan D; Classen, David C; Danforth, Melissa; Stockwell, David C; Longhurst, Christopher A

2017-03-01

To evaluate the safety of computerized physician order entry (CPOE) and associated clinical decision support (CDS) systems in electronic health record (EHR) systems at pediatric inpatient facilities in the US using the Leapfrog Group's pediatric CPOE evaluation tool. The Leapfrog pediatric CPOE evaluation tool, a previously validated tool to assess the ability of a CPOE system to identify orders that could potentially lead to patient harm, was used to evaluate 41 pediatric hospitals over a 2-year period. Evaluation of the last available test for each institution was performed, assessing performance overall as well as by decision support category (eg, drug-drug, dosing limits). Longitudinal analysis of test performance was also carried out to assess the impact of testing and the overall trend of CPOE performance in pediatric hospitals. Pediatric CPOE systems were able to identify 62% of potential medication errors in the test scenarios, but ranged widely from 23-91% in the institutions tested. The highest scoring categories included drug-allergy interactions, dosing limits (both daily and cumulative), and inappropriate routes of administration. We found that hospitals with longer periods since their CPOE implementation did not have better scores upon initial testing, but after initial testing there was a consistent improvement in testing scores of 4 percentage points per year. Pediatric computerized physician order entry (CPOE) systems on average are able to intercept a majority of potential medication errors, but vary widely among implementations. Prospective and repeated testing using the Leapfrog Group's evaluation tool is associated with improved ability to intercept potential medication errors. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Performance evaluation of inpatient service in Beijing: a horizontal comparison with risk adjustment based on Diagnosis Related Groups.

PubMed

Jian, Weiyan; Huang, Yinmin; Hu, Mu; Zhang, Xiumei

2009-04-30

The medical performance evaluation, which provides a basis for rational decision-making, is an important part of medical service research. Current progress with health services reform in China is far from satisfactory, without sufficient regulation. To achieve better progress, an effective tool for evaluating medical performance needs to be established. In view of this, this study attempted to develop such a tool appropriate for the Chinese context. Data was collected from the front pages of medical records (FPMR) of all large general public hospitals (21 hospitals) in the third and fourth quarter of 2007. Locally developed Diagnosis Related Groups (DRGs) were introduced as a tool for risk adjustment and performance evaluation indicators were established: Charge Efficiency Index (CEI), Time Efficiency Index (TEI) and inpatient mortality of low-risk group cases (IMLRG), to reflect respectively work efficiency and medical service quality. Using these indicators, the inpatient services' performance was horizontally compared among hospitals. Case-mix Index (CMI) was used to adjust efficiency indices and then produce adjusted CEI (aCEI) and adjusted TEI (aTEI). Poisson distribution analysis was used to test the statistical significance of the IMLRG differences between different hospitals. Using the aCEI, aTEI and IMLRG scores for the 21 hospitals, Hospital A and C had relatively good overall performance because their medical charges were lower, LOS shorter and IMLRG smaller. The performance of Hospital P and Q was the worst due to their relatively high charge level, long LOS and high IMLRG. Various performance problems also existed in the other hospitals. It is possible to develop an accurate and easy to run performance evaluation system using Case-Mix as the tool for risk adjustment, choosing indicators close to consumers and managers, and utilizing routine report forms as the basic information source. To keep such a system running effectively, it is necessary to improve the reliability of clinical information and the risk-adjustment ability of Case-Mix.

Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool.

PubMed

Leung, Alexander A; Keohane, Carol; Lipsitz, Stuart; Zimlichman, Eyal; Amato, Mary; Simon, Steven R; Coffey, Michael; Kaufman, Nathan; Cadet, Bismarck; Schiff, Gordon; Seger, Diane L; Bates, David W

2013-06-01

The Leapfrog CPOE evaluation tool has been promoted as a means of monitoring computerized physician order entry (CPOE). We sought to determine the relationship between Leapfrog scores and the rates of preventable adverse drug events (ADE) and potential ADE. A cross-sectional study of 1000 adult admissions in five community hospitals from October 1, 2008 to September 30, 2010 was performed. Observed rates of preventable ADE and potential ADE were compared with scores reported by the Leapfrog CPOE evaluation tool. The primary outcome was the rate of preventable ADE and the secondary outcome was the composite rate of preventable ADE and potential ADE. Leapfrog performance scores were highly related to the primary outcome. A 43% relative reduction in the rate of preventable ADE was predicted for every 5% increase in Leapfrog scores (rate ratio 0.57; 95% CI 0.37 to 0.88). In absolute terms, four fewer preventable ADE per 100 admissions were predicted for every 5% increase in overall Leapfrog scores (rate difference -4.2; 95% CI -7.4 to -1.1). A statistically significant relationship between Leapfrog scores and the secondary outcome, however, was not detected. Our findings support the use of the Leapfrog tool as a means of evaluating and monitoring CPOE performance after implementation, as addressed by current certification standards. Scores from the Leapfrog CPOE evaluation tool closely relate to actual rates of preventable ADE. Leapfrog testing may alert providers to potential vulnerabilities and highlight areas for further improvement.

Moving Large Data Sets Over High-Performance Long Distance Networks

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hodson, Stephen W; Poole, Stephen W; Ruwart, Thomas

2011-04-01

In this project we look at the performance characteristics of three tools used to move large data sets over dedicated long distance networking infrastructure. Although performance studies of wide area networks have been a frequent topic of interest, performance analyses have tended to focus on network latency characteristics and peak throughput using network traffic generators. In this study we instead perform an end-to-end long distance networking analysis that includes reading large data sets from a source file system and committing large data sets to a destination file system. An evaluation of end-to-end data movement is also an evaluation of themore » system configurations employed and the tools used to move the data. For this paper, we have built several storage platforms and connected them with a high performance long distance network configuration. We use these systems to analyze the capabilities of three data movement tools: BBcp, GridFTP, and XDD. Our studies demonstrate that existing data movement tools do not provide efficient performance levels or exercise the storage devices in their highest performance modes. We describe the device information required to achieve high levels of I/O performance and discuss how this data is applicable in use cases beyond data movement performance.« less

Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises.

PubMed

Agboola, Foluso; Bernard, Dorothy; Savoia, Elena; Biddinger, Paul D

2015-10-01

Exercises that simulate emergency scenarios are accepted widely as an essential component of a robust Emergency Preparedness program. Unfortunately, the variability in the quality of the exercises conducted, and the lack of standardized processes to measure performance, has limited the value of exercises in measuring preparedness. In order to help health organizations improve the quality and standardization of the performance data they collect during simulated emergencies, a model online exercise evaluation toolkit was developed using performance measures tested in over 60 Emergency Preparedness exercises. The exercise evaluation toolkit contains three major components: (1) a database of measures that can be used to assess performance during an emergency response exercise; (2) a standardized data collection tool (form); and (3) a program that populates the data collection tool with the measures that have been selected by the user from the database. The evaluation toolkit was pilot tested from January through September 2014 in collaboration with 14 partnering organizations representing 10 public health agencies and four health care agencies from eight states across the US. Exercise planners from the partnering organizations were asked to use the toolkit for their exercise evaluation process and were interviewed to provide feedback on the use of the toolkit, the generated evaluation tool, and the usefulness of the data being gathered for the development of the exercise after-action report. Ninety-three percent (93%) of exercise planners reported that they found the online database of performance measures appropriate for the creation of exercise evaluation forms, and they stated that they would use it again for future exercises. Seventy-two percent (72%) liked the exercise evaluation form that was generated from the toolkit, and 93% reported that the data collected by the use of the evaluation form were useful in gauging their organization's performance during the exercise. Seventy-nine percent (79%) of exercise planners preferred the evaluation form generated by the toolkit to other forms of evaluations. Results of this project show that users found the newly developed toolkit to be user friendly and more relevant to measurement of specific public health and health care capabilities than other tools currently available. The developed toolkit may contribute to the further advancement of developing a valid approach to exercise performance measurement.

Health Indicators: A Tool for Program Review

ERIC Educational Resources Information Center

Abou-Sayf, Frank K.

2006-01-01

A visual tool used to evaluate instructional program performance has been designed by the University of Hawaii Community College system. The tool combines features from traffic lights, blood-chemistry test reports, and industry production control charts, and is labeled the Program Health-Indicator Chart. The tool was designed to minimize the labor…

Intraocular robotic interventional surgical system (IRISS): Mechanical design, evaluation, and master-slave manipulation.

PubMed

Wilson, Jason T; Gerber, Matthew J; Prince, Stephen W; Chen, Cheng-Wei; Schwartz, Steven D; Hubschman, Jean-Pierre; Tsao, Tsu-Chin

2018-02-01

Since the advent of robotic-assisted surgery, the value of using robotic systems to assist in surgical procedures has been repeatedly demonstrated. However, existing technologies are unable to perform complete, multi-step procedures from start to finish. Many intraocular surgical steps continue to be manually performed. An intraocular robotic interventional surgical system (IRISS) capable of performing various intraocular surgical procedures was designed, fabricated, and evaluated. Methods were developed to evaluate the performance of the remote centers of motion (RCMs) using a stereo-camera setup and to assess the accuracy and precision of positioning the tool tip using an optical coherence tomography (OCT) system. The IRISS can simultaneously manipulate multiple surgical instruments, change between mounted tools using an onboard tool-change mechanism, and visualize the otherwise invisible RCMs to facilitate alignment of the RCM to the surgical incision. The accuracy of positioning the tool tip was measured to be 0.205±0.003 mm. The IRISS was evaluated by trained surgeons in a remote surgical theatre using post-mortem pig eyes and shown to be effective in completing many key steps in a variety of intraocular surgical procedures as well as being capable of performing an entire cataract extraction from start to finish. The IRISS represents a necessary step towards fully automated intraocular surgery and demonstrated accurate and precise master-slave manipulation for cataract removal and-through visual feedback-retinal vein cannulation. Copyright © 2017 John Wiley & Sons, Ltd.

A Hybrid Evaluation System Framework (Shell & Web) with Standardized Access to Climate Model Data and Verification Tools for a Clear Climate Science Infrastructure on Big Data High Performance Computers

NASA Astrophysics Data System (ADS)

Kadow, C.; Illing, S.; Kunst, O.; Cubasch, U.

2014-12-01

The project 'Integrated Data and Evaluation System for Decadal Scale Prediction' (INTEGRATION) as part of the German decadal prediction project MiKlip develops a central evaluation system. The fully operational hybrid features a HPC shell access and an user friendly web-interface. It employs one common system with a variety of verification tools and validation data from different projects in- and outside of MiKlip. The evaluation system is located at the German Climate Computing Centre (DKRZ) and has direct access to the bulk of its ESGF node including millions of climate model data sets, e.g. from CMIP5 and CORDEX. The database is organized by the international CMOR standard using the meta information of the self-describing model, reanalysis and observational data sets. Apache Solr is used for indexing the different data projects into one common search environment. This implemented meta data system with its advanced but easy to handle search tool supports users, developers and their tools to retrieve the required information. A generic application programming interface (API) allows scientific developers to connect their analysis tools with the evaluation system independently of the programming language used. Users of the evaluation techniques benefit from the common interface of the evaluation system without any need to understand the different scripting languages. Facilitating the provision and usage of tools and climate data increases automatically the number of scientists working with the data sets and identify discrepancies. Additionally, the history and configuration sub-system stores every analysis performed with the evaluation system in a MySQL database. Configurations and results of the tools can be shared among scientists via shell or web-system. Therefore, plugged-in tools gain automatically from transparency and reproducibility. Furthermore, when configurations match while starting a evaluation tool, the system suggests to use results already produced by other users-saving CPU time, I/O and disk space. This study presents the different techniques and advantages of such a hybrid evaluation system making use of a Big Data HPC in climate science. website: www-miklip.dkrz.de visitor-login: guest password: miklip

A Hybrid Evaluation System Framework (Shell & Web) with Standardized Access to Climate Model Data and Verification Tools for a Clear Climate Science Infrastructure on Big Data High Performance Computers

NASA Astrophysics Data System (ADS)

Kadow, Christopher; Illing, Sebastian; Kunst, Oliver; Ulbrich, Uwe; Cubasch, Ulrich

2015-04-01

The project 'Integrated Data and Evaluation System for Decadal Scale Prediction' (INTEGRATION) as part of the German decadal prediction project MiKlip develops a central evaluation system. The fully operational hybrid features a HPC shell access and an user friendly web-interface. It employs one common system with a variety of verification tools and validation data from different projects in- and outside of MiKlip. The evaluation system is located at the German Climate Computing Centre (DKRZ) and has direct access to the bulk of its ESGF node including millions of climate model data sets, e.g. from CMIP5 and CORDEX. The database is organized by the international CMOR standard using the meta information of the self-describing model, reanalysis and observational data sets. Apache Solr is used for indexing the different data projects into one common search environment. This implemented meta data system with its advanced but easy to handle search tool supports users, developers and their tools to retrieve the required information. A generic application programming interface (API) allows scientific developers to connect their analysis tools with the evaluation system independently of the programming language used. Users of the evaluation techniques benefit from the common interface of the evaluation system without any need to understand the different scripting languages. Facilitating the provision and usage of tools and climate data increases automatically the number of scientists working with the data sets and identify discrepancies. Additionally, the history and configuration sub-system stores every analysis performed with the evaluation system in a MySQL database. Configurations and results of the tools can be shared among scientists via shell or web-system. Therefore, plugged-in tools gain automatically from transparency and reproducibility. Furthermore, when configurations match while starting a evaluation tool, the system suggests to use results already produced by other users-saving CPU time, I/O and disk space. This study presents the different techniques and advantages of such a hybrid evaluation system making use of a Big Data HPC in climate science. website: www-miklip.dkrz.de visitor-login: click on "Guest"

Development of RAD-Score: A Tool to Assess the Procedural Competence of Diagnostic Radiology Residents.

PubMed

Isupov, Inga; McInnes, Matthew D F; Hamstra, Stan J; Doherty, Geoffrey; Gupta, Ashish; Peddle, Susan; Jibri, Zaid; Rakhra, Kawan; Hibbert, Rebecca M

2017-04-01

The purpose of this study is to develop a tool to assess the procedural competence of radiology trainees, with sources of evidence gathered from five categories to support the construct validity of tool: content, response process, internal structure, relations to other variables, and consequences. A pilot form for assessing procedural competence among radiology residents, known as the RAD-Score tool, was developed by evaluating published literature and using a modified Delphi procedure involving a group of local content experts. The pilot version of the tool was tested by seven radiology department faculty members who evaluated procedures performed by 25 residents at one institution between October 2014 and June 2015. Residents were evaluated while performing multiple procedures in both clinical and simulation settings. The main outcome measure was the percentage of residents who were considered ready to perform procedures independently, with testing conducted to determine differences between levels of training. A total of 105 forms (for 52 procedures performed in a clinical setting and 53 procedures performed in a simulation setting) were collected for a variety of procedures (eight vascular or interventional, 42 body, 12 musculoskeletal, 23 chest, and 20 breast procedures). A statistically significant difference was noted in the percentage of trainees who were rated as being ready to perform a procedure independently (in postgraduate year [PGY] 2, 12% of residents; in PGY3, 61%; in PGY4, 85%; and in PGY5, 88%; p < 0.05); this difference persisted in the clinical and simulation settings. User feedback and psychometric analysis were used to create a final version of the form. This prospective study describes the successful development of a tool for assessing the procedural competence of radiology trainees with high levels of construct validity in multiple domains. Implementation of the tool in the radiology residency curriculum is planned and can play an instrumental role in the transition to competency-based radiology training.

Performance evaluation of automated segmentation software on optical coherence tomography volume data

PubMed Central

Tian, Jing; Varga, Boglarka; Tatrai, Erika; Fanni, Palya; Somfai, Gabor Mark; Smiddy, William E.

2016-01-01

Over the past two decades a significant number of OCT segmentation approaches have been proposed in the literature. Each methodology has been conceived for and/or evaluated using specific datasets that do not reflect the complexities of the majority of widely available retinal features observed in clinical settings. In addition, there does not exist an appropriate OCT dataset with ground truth that reflects the realities of everyday retinal features observed in clinical settings. While the need for unbiased performance evaluation of automated segmentation algorithms is obvious, the validation process of segmentation algorithms have been usually performed by comparing with manual labelings from each study and there has been a lack of common ground truth. Therefore, a performance comparison of different algorithms using the same ground truth has never been performed. This paper reviews research-oriented tools for automated segmentation of the retinal tissue on OCT images. It also evaluates and compares the performance of these software tools with a common ground truth. PMID:27159849

Practical Guide to Designing Comprehensive Principal Evaluation Systems: A Tool to Assist in the Development of Principal Evaluation Systems. Revised Edition

ERIC Educational Resources Information Center

Clifford, Matthew; Hansen, Ulcca Joshni; Wraight, Sara

2014-01-01

Across the country, states and districts are designing principal evaluation systems as a means of improving leadership, learning, and school performance. Principal evaluation systems hold potential for supporting leaders' learning and sense of accountability for instructional excellence and student performance. Principal evaluation also is an…

A Practical Guide to Designing Comprehensive Principal Evaluation Systems: A Tool to Assist in the Development of Principal Evaluation Systems

ERIC Educational Resources Information Center

Clifford, Matthew; Hansen, Ulcca Joshni; Wraight, Sara

2012-01-01

Across the country, states and districts are designing principal evaluation systems as a means of improving leadership, learning, and school performance. Principal evaluation systems hold potential for supporting leaders' learning and sense of accountability for instructional excellence and student performance. Principal evaluation is also an…

Agent-based modeling as a tool for program design and evaluation.

PubMed

Lawlor, Jennifer A; McGirr, Sara

2017-12-01

Recently, systems thinking and systems science approaches have gained popularity in the field of evaluation; however, there has been relatively little exploration of how evaluators could use quantitative tools to assist in the implementation of systems approaches therein. The purpose of this paper is to explore potential uses of one such quantitative tool, agent-based modeling, in evaluation practice. To this end, we define agent-based modeling and offer potential uses for it in typical evaluation activities, including: engaging stakeholders, selecting an intervention, modeling program theory, setting performance targets, and interpreting evaluation results. We provide demonstrative examples from published agent-based modeling efforts both inside and outside the field of evaluation for each of the evaluative activities discussed. We further describe potential pitfalls of this tool and offer cautions for evaluators who may chose to implement it in their practice. Finally, the article concludes with a discussion of the future of agent-based modeling in evaluation practice and a call for more formal exploration of this tool as well as other approaches to simulation modeling in the field. Copyright © 2017 Elsevier Ltd. All rights reserved.

MVIAeval: a web tool for comprehensively evaluating the performance of a new missing value imputation algorithm.

PubMed

Wu, Wei-Sheng; Jhou, Meng-Jhun

2017-01-13

Missing value imputation is important for microarray data analyses because microarray data with missing values would significantly degrade the performance of the downstream analyses. Although many microarray missing value imputation algorithms have been developed, an objective and comprehensive performance comparison framework is still lacking. To solve this problem, we previously proposed a framework which can perform a comprehensive performance comparison of different existing algorithms. Also the performance of a new algorithm can be evaluated by our performance comparison framework. However, constructing our framework is not an easy task for the interested researchers. To save researchers' time and efforts, here we present an easy-to-use web tool named MVIAeval (Missing Value Imputation Algorithm evaluator) which implements our performance comparison framework. MVIAeval provides a user-friendly interface allowing users to upload the R code of their new algorithm and select (i) the test datasets among 20 benchmark microarray (time series and non-time series) datasets, (ii) the compared algorithms among 12 existing algorithms, (iii) the performance indices from three existing ones, (iv) the comprehensive performance scores from two possible choices, and (v) the number of simulation runs. The comprehensive performance comparison results are then generated and shown as both figures and tables. MVIAeval is a useful tool for researchers to easily conduct a comprehensive and objective performance evaluation of their newly developed missing value imputation algorithm for microarray data or any data which can be represented as a matrix form (e.g. NGS data or proteomics data). Thus, MVIAeval will greatly expedite the progress in the research of missing value imputation algorithms.

Blinded evaluation of interrater reliability of an operative competency assessment tool for direct laryngoscopy and rigid bronchoscopy.

PubMed

Ishman, Stacey L; Benke, James R; Johnson, Kaalan Erik; Zur, Karen B; Jacobs, Ian N; Thorne, Marc C; Brown, David J; Lin, Sandra Y; Bhatti, Nasir; Deutsch, Ellen S

2012-10-01

OBJECTIVES To confirm interrater reliability using blinded evaluation of a skills-assessment instrument to assess the surgical performance of resident and fellow trainees performing pediatric direct laryngoscopy and rigid bronchoscopy in simulated models. DESIGN Prospective, paired, blinded observational validation study. SUBJECTS Paired observers from multiple institutions simultaneously evaluated residents and fellows who were performing surgery in an animal laboratory or using high-fidelity manikins. The evaluators had no previous affiliation with the residents and fellows and did not know their year of training. INTERVENTIONS One- and 2-page versions of an objective structured assessment of technical skills (OSATS) assessment instrument composed of global and a task-specific surgical items were used to evaluate surgical performance. RESULTS Fifty-two evaluations were completed by 17 attending evaluators. The instrument agreement for the 2-page assessment was 71.4% when measured as a binary variable (ie, competent vs not competent) (κ = 0.38; P = .08). Evaluation as a continuous variable revealed a 42.9% percentage agreement (κ = 0.18; P = .14). The intraclass correlation was 0.53, considered substantial/good interrater reliability (69% reliable). For the 1-page instrument, agreement was 77.4% when measured as a binary variable (κ = 0.53, P = .0015). Agreement when evaluated as a continuous measure was 71.0% (κ = 0.54, P < .001). The intraclass correlation was 0.73, considered high interrater reliability (85% reliable). CONCLUSIONS The OSATS assessment instrument is an effective tool for evaluating surgical performance among trainees with acceptable interrater reliability in a simulator setting. Reliability was good for both the 1- and 2-page OSATS checklists, and both serve as excellent tools to provide immediate formative feedback on operational competency.

Image Navigation and Registration Performance Assessment Evaluation Tools for GOES-R ABI and GLM

NASA Technical Reports Server (NTRS)

Houchin, Scott; Porter, Brian; Graybill, Justin; Slingerland, Philip

2017-01-01

The GOES-R Flight Project has developed an Image Navigation and Registration (INR) Performance Assessment Tool Set (IPATS) for measuring Advanced Baseline Imager (ABI) and Geostationary Lightning Mapper (GLM) INR performance metrics in the post-launch period for performance evaluation and long term monitoring. IPATS utilizes a modular algorithmic design to allow user selection of data processing sequences optimized for generation of each INR metric. This novel modular approach minimizes duplication of common processing elements, thereby maximizing code efficiency and speed. Fast processing is essential given the large number of sub-image registrations required to generate INR metrics for the many images produced over a 24 hour evaluation period. This paper describes the software design and implementation of IPATS and provides preliminary test results.

New methodology to baseline and match AME polysilicon etcher using advanced diagnostic tools

NASA Astrophysics Data System (ADS)

Poppe, James; Shipman, John; Reinhardt, Barbara E.; Roussel, Myriam; Hedgecock, Raymond; Fonda, Arturo

1999-09-01

As process controls tighten in the semiconductor industry, the need to understand the variables that determine system performance become more important. For plasma etch systems, process success depends on the control of key parameters such as: vacuum integrity, pressure, gas flows, and RF power. It is imperative to baseline, monitor, and control these variables. This paper presents an overview of the methods and tools used by Motorola BMC fabrication facility to characterize an Applied Materials polysilicon etcher. Tool performance data obtained from our traditional measurement techniques are limited in their scope and do not provide a complete picture of the ultimate tool performance. Presently the BMC traditional characterization tools provide a snapshot of the static operation of the equipment under test (EUT); however, complete evaluation of the dynamic performance cannot be monitored without the aid of specialized diagnostic equipment. To provide us with a complete system baseline evaluation of the polysilicon etcher, three diagnostic tools were utilized: Lucas Labs Vacuum Diagnostic System, Residual Gas Analyzer, and the ENI Voltage/Impedance Probe. The diagnostic methodology used to baseline and match key parameters of qualified production equipment has had an immense impact on other equipment characterization in the facility. It has resulted in reduced cycle time for new equipment introduction as well.

Methods for Evaluating the Performance and Human Stress-Factors of Percussive Riveting

NASA Astrophysics Data System (ADS)

Ahn, Jonathan Y.

The aerospace industry automates portions of their manufacturing and assembly processes. However, mechanics still remain vital to production, especially in areas where automated machines cannot fit, or have yet to match the quality of human craftsmanship. One such task is percussive riveting. Because percussive riveting is associated with a high risk of injury, these tool must be certified prior to release. The major contribution of this thesis is to develop a test bench capable of percussive riveting for ergonomic evaluation purposes. The major issues investigated are: (i) automate the tool evaluation method to be repeatable; (ii) demonstrate use of displacement and force sensors; and (iii) correlate performance and risk exposure of percussive tools. A test bench equipped with servomotors and pneumatic cylinders to control xyz-position of a rivet gun and bucking bar simultaneously, is used to explore this evaluation approach.

Clinical peer review program self-evaluation for US hospitals.

PubMed

Edwards, Marc T

2010-01-01

Prior research has shown wide variation in clinical peer review program structure, process, governance, and perceived effectiveness. This study sought to validate the utility of a Peer Review Program Self-Evaluation Tool as a potential guide to physician and hospital leaders seeking greater program value. Data from 330 hospitals show that the total score from the self-evaluation tool is strongly associated with perceived quality impact. Organizational culture also plays a significant role. When controlling for these factors, there was no evidence of benefit from a multispecialty review process. Physicians do not generally use reliable methods to measure clinical performance. A high rate of change since 2007 has not produced much improvement. The Peer Review Program Self-Evaluation Tool reliably differentiates hospitals along a continuum of perceived program performance. The full potential of peer review as a process to improve the quality and safety of care has yet to be realized.

An evaluation of copy number variation detection tools for cancer using whole exome sequencing data.

PubMed

Zare, Fatima; Dow, Michelle; Monteleone, Nicholas; Hosny, Abdelrahman; Nabavi, Sheida

2017-05-31

Recently copy number variation (CNV) has gained considerable interest as a type of genomic/genetic variation that plays an important role in disease susceptibility. Advances in sequencing technology have created an opportunity for detecting CNVs more accurately. Recently whole exome sequencing (WES) has become primary strategy for sequencing patient samples and study their genomics aberrations. However, compared to whole genome sequencing, WES introduces more biases and noise that make CNV detection very challenging. Additionally, tumors' complexity makes the detection of cancer specific CNVs even more difficult. Although many CNV detection tools have been developed since introducing NGS data, there are few tools for somatic CNV detection for WES data in cancer. In this study, we evaluated the performance of the most recent and commonly used CNV detection tools for WES data in cancer to address their limitations and provide guidelines for developing new ones. We focused on the tools that have been designed or have the ability to detect cancer somatic aberrations. We compared the performance of the tools in terms of sensitivity and false discovery rate (FDR) using real data and simulated data. Comparative analysis of the results of the tools showed that there is a low consensus among the tools in calling CNVs. Using real data, tools show moderate sensitivity (~50% - ~80%), fair specificity (~70% - ~94%) and poor FDRs (~27% - ~60%). Also, using simulated data we observed that increasing the coverage more than 10× in exonic regions does not improve the detection power of the tools significantly. The limited performance of the current CNV detection tools for WES data in cancer indicates the need for developing more efficient and precise CNV detection methods. Due to the complexity of tumors and high level of noise and biases in WES data, employing advanced novel segmentation, normalization and de-noising techniques that are designed specifically for cancer data is necessary. Also, CNV detection development suffers from the lack of a gold standard for performance evaluation. Finally, developing tools with user-friendly user interfaces and visualization features can enhance CNV studies for a broader range of users.

Performance Analysis of GYRO: A Tool Evaluation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Worley, P.; Roth, P.; Candy, J.

2005-06-26

The performance of the Eulerian gyrokinetic-Maxwell solver code GYRO is analyzed on five high performance computing systems. First, a manual approach is taken, using custom scripts to analyze the output of embedded wall clock timers, floating point operation counts collected using hardware performance counters, and traces of user and communication events collected using the profiling interface to Message Passing Interface (MPI) libraries. Parts of the analysis are then repeated or extended using a number of sophisticated performance analysis tools: IPM, KOJAK, SvPablo, TAU, and the PMaC modeling tool suite. The paper briefly discusses what has been discovered via this manualmore » analysis process, what performance analyses are inconvenient or infeasible to attempt manually, and to what extent the tools show promise in accelerating or significantly extending the manual performance analyses.« less

Improving smoothing efficiency of rigid conformal polishing tool using time-dependent smoothing evaluation model

NASA Astrophysics Data System (ADS)

Song, Chi; Zhang, Xuejun; Zhang, Xin; Hu, Haifei; Zeng, Xuefeng

2017-06-01

A rigid conformal (RC) lap can smooth mid-spatial-frequency (MSF) errors, which are naturally smaller than the tool size, while still removing large-scale errors in a short time. However, the RC-lap smoothing efficiency performance is poorer than expected, and existing smoothing models cannot explicitly specify the methods to improve this efficiency. We presented an explicit time-dependent smoothing evaluation model that contained specific smoothing parameters directly derived from the parametric smoothing model and the Preston equation. Based on the time-dependent model, we proposed a strategy to improve the RC-lap smoothing efficiency, which incorporated the theoretical model, tool optimization, and efficiency limit determination. Two sets of smoothing experiments were performed to demonstrate the smoothing efficiency achieved using the time-dependent smoothing model. A high, theory-like tool influence function and a limiting tool speed of 300 RPM were o

User manual of the CATSS system (version 1.0) communication analysis tool for space station

NASA Technical Reports Server (NTRS)

Tsang, C. S.; Su, Y. T.; Lindsey, W. C.

1983-01-01

The Communication Analysis Tool for the Space Station (CATSS) is a FORTRAN language software package capable of predicting the communications links performance for the Space Station (SS) communication and tracking (C & T) system. An interactive software package was currently developed to run on the DEC/VAX computers. The CATSS models and evaluates the various C & T links of the SS, which includes the modulation schemes such as Binary-Phase-Shift-Keying (BPSK), BPSK with Direct Sequence Spread Spectrum (PN/BPSK), and M-ary Frequency-Shift-Keying with Frequency Hopping (FH/MFSK). Optical Space Communication link is also included. CATSS is a C & T system engineering tool used to predict and analyze the system performance for different link environment. Identification of system weaknesses is achieved through evaluation of performance with varying system parameters. System tradeoff for different values of system parameters are made based on the performance prediction.

PCTFPeval: a web tool for benchmarking newly developed algorithms for predicting cooperative transcription factor pairs in yeast.

PubMed

Lai, Fu-Jou; Chang, Hong-Tsun; Wu, Wei-Sheng

2015-01-01

Computational identification of cooperative transcription factor (TF) pairs helps understand the combinatorial regulation of gene expression in eukaryotic cells. Many advanced algorithms have been proposed to predict cooperative TF pairs in yeast. However, it is still difficult to conduct a comprehensive and objective performance comparison of different algorithms because of lacking sufficient performance indices and adequate overall performance scores. To solve this problem, in our previous study (published in BMC Systems Biology 2014), we adopted/proposed eight performance indices and designed two overall performance scores to compare the performance of 14 existing algorithms for predicting cooperative TF pairs in yeast. Most importantly, our performance comparison framework can be applied to comprehensively and objectively evaluate the performance of a newly developed algorithm. However, to use our framework, researchers have to put a lot of effort to construct it first. To save researchers time and effort, here we develop a web tool to implement our performance comparison framework, featuring fast data processing, a comprehensive performance comparison and an easy-to-use web interface. The developed tool is called PCTFPeval (Predicted Cooperative TF Pair evaluator), written in PHP and Python programming languages. The friendly web interface allows users to input a list of predicted cooperative TF pairs from their algorithm and select (i) the compared algorithms among the 15 existing algorithms, (ii) the performance indices among the eight existing indices, and (iii) the overall performance scores from two possible choices. The comprehensive performance comparison results are then generated in tens of seconds and shown as both bar charts and tables. The original comparison results of each compared algorithm and each selected performance index can be downloaded as text files for further analyses. Allowing users to select eight existing performance indices and 15 existing algorithms for comparison, our web tool benefits researchers who are eager to comprehensively and objectively evaluate the performance of their newly developed algorithm. Thus, our tool greatly expedites the progress in the research of computational identification of cooperative TF pairs.

PCTFPeval: a web tool for benchmarking newly developed algorithms for predicting cooperative transcription factor pairs in yeast

PubMed Central

2015-01-01

Background Computational identification of cooperative transcription factor (TF) pairs helps understand the combinatorial regulation of gene expression in eukaryotic cells. Many advanced algorithms have been proposed to predict cooperative TF pairs in yeast. However, it is still difficult to conduct a comprehensive and objective performance comparison of different algorithms because of lacking sufficient performance indices and adequate overall performance scores. To solve this problem, in our previous study (published in BMC Systems Biology 2014), we adopted/proposed eight performance indices and designed two overall performance scores to compare the performance of 14 existing algorithms for predicting cooperative TF pairs in yeast. Most importantly, our performance comparison framework can be applied to comprehensively and objectively evaluate the performance of a newly developed algorithm. However, to use our framework, researchers have to put a lot of effort to construct it first. To save researchers time and effort, here we develop a web tool to implement our performance comparison framework, featuring fast data processing, a comprehensive performance comparison and an easy-to-use web interface. Results The developed tool is called PCTFPeval (Predicted Cooperative TF Pair evaluator), written in PHP and Python programming languages. The friendly web interface allows users to input a list of predicted cooperative TF pairs from their algorithm and select (i) the compared algorithms among the 15 existing algorithms, (ii) the performance indices among the eight existing indices, and (iii) the overall performance scores from two possible choices. The comprehensive performance comparison results are then generated in tens of seconds and shown as both bar charts and tables. The original comparison results of each compared algorithm and each selected performance index can be downloaded as text files for further analyses. Conclusions Allowing users to select eight existing performance indices and 15 existing algorithms for comparison, our web tool benefits researchers who are eager to comprehensively and objectively evaluate the performance of their newly developed algorithm. Thus, our tool greatly expedites the progress in the research of computational identification of cooperative TF pairs. PMID:26677932

Assessment of Energy Storage Alternatives in the Puget Sound Energy System Volume 2: Energy Storage Evaluation Tool

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wu, Di; Jin, Chunlian; Balducci, Patrick J.

2013-12-01

This volume presents the battery storage evaluation tool developed at Pacific Northwest National Laboratory (PNNL), which is used to evaluate benefits of battery storage for multiple grid applications, including energy arbitrage, balancing service, capacity value, distribution system equipment deferral, and outage mitigation. This tool is based on the optimal control strategies to capture multiple services from a single energy storage device. In this control strategy, at each hour, a look-ahead optimization is first formulated and solved to determine battery base operating point. The minute by minute simulation is then performed to simulate the actual battery operation. This volume provide backgroundmore » and manual for this evaluation tool.« less

Access to Teacher Evaluations Divides Advocates

ERIC Educational Resources Information Center

Sawchuk, Stephen

2012-01-01

As the movement to overhaul teacher evaluation marches onward, an emerging question is splitting the swath of advocates who support the new tools used to gauge teacher performance: Who should get access to the resulting information? Supporters of typing teacher evaluations to student performance differ over whether individuals' results should be…

Automated benchmarking of peptide-MHC class I binding predictions.

PubMed

Trolle, Thomas; Metushi, Imir G; Greenbaum, Jason A; Kim, Yohan; Sidney, John; Lund, Ole; Sette, Alessandro; Peters, Bjoern; Nielsen, Morten

2015-07-01

Numerous in silico methods predicting peptide binding to major histocompatibility complex (MHC) class I molecules have been developed over the last decades. However, the multitude of available prediction tools makes it non-trivial for the end-user to select which tool to use for a given task. To provide a solid basis on which to compare different prediction tools, we here describe a framework for the automated benchmarking of peptide-MHC class I binding prediction tools. The framework runs weekly benchmarks on data that are newly entered into the Immune Epitope Database (IEDB), giving the public access to frequent, up-to-date performance evaluations of all participating tools. To overcome potential selection bias in the data included in the IEDB, a strategy was implemented that suggests a set of peptides for which different prediction methods give divergent predictions as to their binding capability. Upon experimental binding validation, these peptides entered the benchmark study. The benchmark has run for 15 weeks and includes evaluation of 44 datasets covering 17 MHC alleles and more than 4000 peptide-MHC binding measurements. Inspection of the results allows the end-user to make educated selections between participating tools. Of the four participating servers, NetMHCpan performed the best, followed by ANN, SMM and finally ARB. Up-to-date performance evaluations of each server can be found online at http://tools.iedb.org/auto_bench/mhci/weekly. All prediction tool developers are invited to participate in the benchmark. Sign-up instructions are available at http://tools.iedb.org/auto_bench/mhci/join. mniel@cbs.dtu.dk or bpeters@liai.org Supplementary data are available at Bioinformatics online. © The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

Automated benchmarking of peptide-MHC class I binding predictions

PubMed Central

Trolle, Thomas; Metushi, Imir G.; Greenbaum, Jason A.; Kim, Yohan; Sidney, John; Lund, Ole; Sette, Alessandro; Peters, Bjoern; Nielsen, Morten

2015-01-01

Motivation: Numerous in silico methods predicting peptide binding to major histocompatibility complex (MHC) class I molecules have been developed over the last decades. However, the multitude of available prediction tools makes it non-trivial for the end-user to select which tool to use for a given task. To provide a solid basis on which to compare different prediction tools, we here describe a framework for the automated benchmarking of peptide-MHC class I binding prediction tools. The framework runs weekly benchmarks on data that are newly entered into the Immune Epitope Database (IEDB), giving the public access to frequent, up-to-date performance evaluations of all participating tools. To overcome potential selection bias in the data included in the IEDB, a strategy was implemented that suggests a set of peptides for which different prediction methods give divergent predictions as to their binding capability. Upon experimental binding validation, these peptides entered the benchmark study. Results: The benchmark has run for 15 weeks and includes evaluation of 44 datasets covering 17 MHC alleles and more than 4000 peptide-MHC binding measurements. Inspection of the results allows the end-user to make educated selections between participating tools. Of the four participating servers, NetMHCpan performed the best, followed by ANN, SMM and finally ARB. Availability and implementation: Up-to-date performance evaluations of each server can be found online at http://tools.iedb.org/auto_bench/mhci/weekly. All prediction tool developers are invited to participate in the benchmark. Sign-up instructions are available at http://tools.iedb.org/auto_bench/mhci/join. Contact: mniel@cbs.dtu.dk or bpeters@liai.org Supplementary information: Supplementary data are available at Bioinformatics online. PMID:25717196

Assurance Policy Evaluation - Spacecraft and Strategic Systems

DTIC Science & Technology

2014-09-17

electromechanical (EEE) parts, software, design and workmanship, work instructions, manufacturing and tooling, cleanrooms, electrostatic discharge ...T9001B.  An external group, called the Evaluation and Assessment Team, made up of product assurance subject matter experts from NSWC Corona performs...NSWC, Corona and SSP Technical Branch(es). The FTPE, performed every 3 years, is an objective evaluation of facility performance to assure proper

U.S. Team Green Building Challenge 2002

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

2002-09-01

Flier about the U.S. Team and its projects participating in the International Green Building Challenge. Along with many other countries, the United States accepted the Green Building Challenge (GBC), an international effort to evaluate and improve the performance of buildings worldwide. GBC started out in 1996 as a competition to determine which country had the greenest buildings; it evolved into a cooperative process among the countries to measure the performance of green buildings. Although the auto industry can easily measure efficiency in terms of miles per gallon, the buildings industry has no standard way to quantify energy and environmental performance.more » The Green Building Challenge participants hope that better tools for measuring the energy and environmental performance of buildings will be an outcome of their efforts and that these tools will lead to higher and better performance levels in buildings around the world. The ultimate goal is to design, construct, and operate buildings that contribute to global sustainability by conserving and/or regenerating natural resources and minimizing nonrenewable energy use. The United States' Green Building Challenge Team '02 selected five buildings from around the country to serve as case studies; each of the five U.S. building designs (as well as all international case studies) were assessed using an in-depth evaluation tool, called the Green Building Assessment Tool (GBTool). The GBTool was specifically created and refined by international teams, for the GBC efforts. The goal of this collaborative effort is to improve this evaluation software tool so that it can be used globally, while taking into account regional and national conditions. The GBTool was used by the U.S. Team to assess and evaluate the energy and environmental performance of these five buildings: (1) Retail (in operation): BigHorn Home Improvement Center, Silverthorne, Colorado; (2) Office (in operation), Philip Merrill Environmental; (3) School (in construction), Clearview Elementary School, Hanover, Pennsylvania; (4) Multi-family residential (in construction), Twenty River Terrace, Battery Park City, New York; and (5) Office/lab (in design), National Oceanic Atmospheric Administration, Honolulu, Hawaii. These projects were selected, not only because they were good examples of high-performance buildings and had interested owners/design team members, but also because building data was available as inputs to test the software tool. Both the tool and the process have been repeatedly refined and enhanced since the first Green Building Challenge event in 1998; participating countries are continuously providing feedback to further improve the tool and global process for the greatest positive effect.« less

Evaluation of the Langmuir model in the Soil and Water Assessment Tool for high soil phosphorus condition

USDA-ARS?s Scientific Manuscript database

Phosphorus adsorption by a water treatment residual was tested through Langmuir and linear sorption isotherms and applied in the Soil and Water Assessment Tool (SWAT). The objective of this study was to use laboratory and greenhouse experimental phosphorus data to evaluate the performance of a modi...

Drive Cycle Analysis Tool - DriveCAT | NREL

Science.gov Websites

tool was created by NREL's fleet test and evaluation team, which conducts in-service performance their needs. Learn more about NREL's fleet test and evaluation research. Contact Us Let us know if you have any questions about the data, need assistance, or would like to contribute test cycles. We also

Guidelines for evaluating the educational performance of medical school faculty: priming a national conversation.

PubMed

Baldwin, Constance; Chandran, Latha; Gusic, Maryellen

2011-01-01

The academic community needs a sound framework for the promotion and advancement of educators. The Group on Educational Affairs of the Association of American Medical Colleges organized a consensus conference that affirmed the use of five domains for documenting the quantity and quality of scholarly engagement in educational activities: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment. In this article, we offer detailed guidelines to evaluate these five domains of educator performance and the essential elements of scholarly activity. The guidelines are adapted from our developmental educator portfolio template and educator portfolio analysis tool, previously published in MedEdPORTAL. A short tool for educator performance evaluation that summarizes items in the guidelines is proposed for discussion. Our goal in this article is to itemize criteria for systematic faculty evaluation that can be applied in any institutional setting to assist promotion decision makers in their task of evaluating medical school faculty.

Logic Modeling as a Tool to Prepare to Evaluate Disaster and Emergency Preparedness, Response, and Recovery in Schools

ERIC Educational Resources Information Center

Zantal-Wiener, Kathy; Horwood, Thomas J.

2010-01-01

The authors propose a comprehensive evaluation framework to prepare for evaluating school emergency management programs. This framework involves a logic model that incorporates Government Performance and Results Act (GPRA) measures as a foundation for comprehensive evaluation that complements performance monitoring used by the U.S. Department of…

hydropower biological evaluation tools

DOE Office of Scientific and Technical Information (OSTI.GOV)

This software is a set of analytical tools to evaluate the physical and biological performance of existing, refurbished, or newly installed conventional hydro-turbines nationwide where fish passage is a regulatory concern. The current version is based on information collected by the Sensor Fish. Future version will include other technologies. The tool set includes data acquisition, data processing, and biological response tools with applications to various turbine designs and other passage alternatives. The associated database is centralized, and can be accessed remotely. We have demonstrated its use for various applications including both turbines and spillways

Evaluation tools to support ITS planning process : development of a sketch planning tool in FSUTMS/cube environment.

DOT National Transportation Integrated Search

2008-12-15

Intelligent Transportation Systems (ITS) planning requires the use of tools to assess the performance of ITS deployment alternatives relative to each other and to other types of transportation system improvement alternatives. This research project in...

Validity evidence for the Simulated Colonoscopy Objective Performance Evaluation scoring system.

PubMed

Trinca, Kristen D; Cox, Tiffany C; Pearl, Jonathan P; Ritter, E Matthew

2014-02-01

Low-cost, objective systems to assess and train endoscopy skills are needed. The aim of this study was to evaluate the ability of Simulated Colonoscopy Objective Performance Evaluation to assess the skills required to perform endoscopy. Thirty-eight subjects were included in this study, all of whom performed 4 tasks. The scoring system measured performance by calculating precision and efficiency. Data analysis assessed the relationship between colonoscopy experience and performance on each task and the overall score. Endoscopic trainees' Simulated Colonoscopy Objective Performance Evaluation scores correlated significantly with total colonoscopy experience (r = .61, P = .003) and experience in the past 12 months (r = .63, P = .002). Significant differences were seen among practicing endoscopists, nonendoscopic surgeons, and trainees (P < .0001). When the 4 tasks were analyzed, each showed significant correlation with colonoscopy experience (scope manipulation, r = .44, P = .044; tool targeting, r = .45, P = .04; loop management, r = .47, P = .032; mucosal inspection, r = .65, P = .001) and significant differences in performance between the endoscopist groups, except for mucosal inspection (scope manipulation, P < .0001; tool targeting, P = .002; loop management, P = .0008; mucosal inspection, P = .27). Simulated Colonoscopy Objective Performance Evaluation objectively assesses the technical skills required to perform endoscopy and shows promise as a platform for proficiency-based skills training. Published by Elsevier Inc.

Performance evaluation of structure based and ligand based virtual screening methods on ten selected anti-cancer targets.

PubMed

Ramasamy, Thilagavathi; Selvam, Chelliah

2015-10-15

Virtual screening has become an important tool in drug discovery process. Structure based and ligand based approaches are generally used in virtual screening process. To date, several benchmark sets for evaluating the performance of the virtual screening tool are available. In this study, our aim is to compare the performance of both structure based and ligand based virtual screening methods. Ten anti-cancer targets and their corresponding benchmark sets from 'Demanding Evaluation Kits for Objective In silico Screening' (DEKOIS) library were selected. X-ray crystal structures of protein-ligand complexes were selected based on their resolution. Openeye tools such as FRED, vROCS were used and the results were carefully analyzed. At EF1%, vROCS produced better results but at EF5% and EF10%, both FRED and ROCS produced almost similar results. It was noticed that the enrichment factor values were decreased while going from EF1% to EF5% and EF10% in many cases. Published by Elsevier Ltd.

Experimental evaluation of the impact of packet capturing tools for web services.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Choe, Yung Ryn; Mohapatra, Prasant; Chuah, Chen-Nee

Network measurement is a discipline that provides the techniques to collect data that are fundamental to many branches of computer science. While many capturing tools and comparisons have made available in the literature and elsewhere, the impact of these packet capturing tools on existing processes have not been thoroughly studied. While not a concern for collection methods in which dedicated servers are used, many usage scenarios of packet capturing now requires the packet capturing tool to run concurrently with operational processes. In this work we perform experimental evaluations of the performance impact that packet capturing process have on web-based services;more » in particular, we observe the impact on web servers. We find that packet capturing processes indeed impact the performance of web servers, but on a multi-core system the impact varies depending on whether the packet capturing and web hosting processes are co-located or not. In addition, the architecture and behavior of the web server and process scheduling is coupled with the behavior of the packet capturing process, which in turn also affect the web server's performance.« less

Evaluating the Fraser Health Balanced Scorecard--a formative evaluation.

PubMed

Barnardo, Catherine; Jivanni, Amin

2009-01-01

Fraser Health (FH), a large, Canadian, integrated health care network, adopted the Balanced Scorecard (BSC) approach to monitor organizational performance in 2006. This paper reports on the results of a formative evaluation, conducted in April, 2008, to assess the usefulness of the BSC as a performance-reporting system and a performance management tool. Results indicated that the BSC has proven to be useful for reporting performance but is not currently used for performance management in a substantial way.

The effect of ergonomic laparoscopic tool handle design on performance and efficiency.

PubMed

Tung, Kryztopher D; Shorti, Rami M; Downey, Earl C; Bloswick, Donald S; Merryweather, Andrew S

2015-09-01

Many factors can affect a surgeon's performance in the operating room; these may include surgeon comfort, ergonomics of tool handle design, and fatigue. A laparoscopic tool handle designed with ergonomic considerations (pistol grip) was tested against a current market tool with a traditional pinch grip handle. The goal of this study is to quantify the impact ergonomic design considerations which have on surgeon performance. We hypothesized that there will be measurable differences between the efficiency while performing FLS surgical trainer tasks when using both tool handle designs in three categories: time to completion, technical skill, and subjective user ratings. The pistol grip incorporates an ergonomic interface intended to reduce contact stress points on the hand and fingers, promote a more neutral operating wrist posture, and reduce hand tremor and fatigue. The traditional pinch grip is a laparoscopic tool developed by Stryker Inc. widely used during minimal invasive surgery. Twenty-three (13 M, 10 F) participants with no existing upper extremity musculoskeletal disorders or experience performing laparoscopic procedures were selected to perform in this study. During a training session prior to testing, participants performed practice trials in a SAGES FLS trainer with both tools. During data collection, participants performed three evaluation tasks using both handle designs (order was randomized, and each trial completed three times). The tasks consisted of FLS peg transfer, cutting, and suturing tasks. Feedback from test participants indicated that they significantly preferred the ergonomic pistol grip in every category (p < 0.05); most notably, participants experienced greater degrees of discomfort in their hands after using the pinch grip tool. Furthermore, participants completed cutting and peg transfer tasks in a shorter time duration (p < 0.05) with the pistol grip than with the pinch grip design; there was no significant difference between completion times for the suturing task. Finally, there was no significant interaction between tool type and errors made during trials. There was a significant preference for as well as lower pain experienced during use of the pistol grip tool as seen from the survey feedback. Both evaluation tasks (cutting and peg transfer) were also completed significantly faster with the pistol grip tool. Finally, due to the high degree of variability in the error data, it was not possible to draw any meaningful conclusions about the effect of tool design on the number or degree of errors made.

Student Evaluations of Teaching Are an Inadequate Assessment Tool for Evaluating Faculty Performance

ERIC Educational Resources Information Center

Hornstein, Henry A.

2017-01-01

Literature is examined to support the contention that student evaluations of teaching (SET) should not be used for summative evaluation of university faculty. Recommendations for alternatives to SET are provided.

A multi-center study benchmarks software tools for label-free proteome quantification

PubMed Central

Gillet, Ludovic C; Bernhardt, Oliver M.; MacLean, Brendan; Röst, Hannes L.; Tate, Stephen A.; Tsou, Chih-Chiang; Reiter, Lukas; Distler, Ute; Rosenberger, George; Perez-Riverol, Yasset; Nesvizhskii, Alexey I.; Aebersold, Ruedi; Tenzer, Stefan

2016-01-01

The consistent and accurate quantification of proteins by mass spectrometry (MS)-based proteomics depends on the performance of instruments, acquisition methods and data analysis software. In collaboration with the software developers, we evaluated OpenSWATH, SWATH2.0, Skyline, Spectronaut and DIA-Umpire, five of the most widely used software methods for processing data from SWATH-MS (sequential window acquisition of all theoretical fragment ion spectra), a method that uses data-independent acquisition (DIA) for label-free protein quantification. We analyzed high-complexity test datasets from hybrid proteome samples of defined quantitative composition acquired on two different MS instruments using different SWATH isolation windows setups. For consistent evaluation we developed LFQbench, an R-package to calculate metrics of precision and accuracy in label-free quantitative MS, and report the identification performance, robustness and specificity of each software tool. Our reference datasets enabled developers to improve their software tools. After optimization, all tools provided highly convergent identification and reliable quantification performance, underscoring their robustness for label-free quantitative proteomics. PMID:27701404

A multicenter study benchmarks software tools for label-free proteome quantification.

PubMed

Navarro, Pedro; Kuharev, Jörg; Gillet, Ludovic C; Bernhardt, Oliver M; MacLean, Brendan; Röst, Hannes L; Tate, Stephen A; Tsou, Chih-Chiang; Reiter, Lukas; Distler, Ute; Rosenberger, George; Perez-Riverol, Yasset; Nesvizhskii, Alexey I; Aebersold, Ruedi; Tenzer, Stefan

2016-11-01

Consistent and accurate quantification of proteins by mass spectrometry (MS)-based proteomics depends on the performance of instruments, acquisition methods and data analysis software. In collaboration with the software developers, we evaluated OpenSWATH, SWATH 2.0, Skyline, Spectronaut and DIA-Umpire, five of the most widely used software methods for processing data from sequential window acquisition of all theoretical fragment-ion spectra (SWATH)-MS, which uses data-independent acquisition (DIA) for label-free protein quantification. We analyzed high-complexity test data sets from hybrid proteome samples of defined quantitative composition acquired on two different MS instruments using different SWATH isolation-window setups. For consistent evaluation, we developed LFQbench, an R package, to calculate metrics of precision and accuracy in label-free quantitative MS and report the identification performance, robustness and specificity of each software tool. Our reference data sets enabled developers to improve their software tools. After optimization, all tools provided highly convergent identification and reliable quantification performance, underscoring their robustness for label-free quantitative proteomics.

Use MACES IVA Suit for EVA Mobility Evaluations

NASA Technical Reports Server (NTRS)

Watson, Richard D.

2014-01-01

The use of an Intra-Vehicular Activity (IVA) suit for a spacewalk or Extra-Vehicular Activity (EVA) was evaluated for mobility and usability in the Neutral Buoyancy Lab (NBL) environment. The Space Shuttle Advanced Crew Escape Suit (ACES) has been modified (MACES) to integrate with the Orion spacecraft. The first several missions of the Orion MPCV spacecraft will not have mass available to carry an EVA specific suit so any EVA required will have to be performed by the MACES. Since the MACES was not designed with EVA in mind, it was unknown what mobility the suit would be able to provide for an EVA or if a person could perform useful tasks for an extended time inside the pressurized suit. The suit was evaluated in multiple NBL runs by a variety of subjects including crewmembers with significant EVA experience. Various functional mobility tasks performed included: translation, body positioning, carrying tools, body stabilization, equipment handling, and use of tools. Hardware configurations included with and without TMG, suit with IVA gloves and suit with EVA gloves. Most tasks were completed on ISS mockups with existing EVA tools. Some limited tasks were completed with prototype tools on a simulated rocky surface. Major findings include: demonstration of the ability to weigh-out the suit, understanding the need to have subjects perform multiple runs prior to getting feedback, determination of critical sizing factors, and need for adjustment of suit work envelop. The early testing has demonstrated the feasibility of EVA's limited duration and limited scope. Further testing is required with more flight like tasking and constraints to validate these early results. If the suit is used for EVA, it will require mission specific modifications for umbilical management or PLSS integration, safety tether attachment, and tool interfaces. These evaluations are continuing through calendar year 2014.

Beyond information access: Support for complex cognitive activities in public health informatics tools.

PubMed

Sedig, Kamran; Parsons, Paul; Dittmer, Mark; Ola, Oluwakemi

2012-01-01

Public health professionals work with a variety of information sources to carry out their everyday activities. In recent years, interactive computational tools have become deeply embedded in such activities. Unlike the early days of computational tool use, the potential of tools nowadays is not limited to simply providing access to information; rather, they can act as powerful mediators of human-information discourse, enabling rich interaction with public health information. If public health informatics tools are designed and used properly, they can facilitate, enhance, and support the performance of complex cognitive activities that are essential to public health informatics, such as problem solving, forecasting, sense-making, and planning. However, the effective design and evaluation of public health informatics tools requires an understanding of the cognitive and perceptual issues pertaining to how humans work and think with information to perform such activities. This paper draws on research that has examined some of the relevant issues, including interaction design, complex cognition, and visual representations, to offer some human-centered design and evaluation considerations for public health informatics tools.

A New Tool for Identifying Research Standards and Evaluating Research Performance

ERIC Educational Resources Information Center

Bacon, Donald R.; Paul, Pallab; Stewart, Kim A.; Mukhopadhyay, Kausiki

2012-01-01

Much has been written about the evaluation of faculty research productivity in promotion and tenure decisions, including many articles that seek to determine the rank of various marketing journals. Yet how faculty evaluators combine journal quality, quantity, and author contribution to form judgments of a scholar's performance is unclear. A…

Performance evaluation of inpatient service in Beijing: a horizontal comparison with risk adjustment based on Diagnosis Related Groups

PubMed Central

Jian, Weiyan; Huang, Yinmin; Hu, Mu; Zhang, Xiumei

2009-01-01

Background The medical performance evaluation, which provides a basis for rational decision-making, is an important part of medical service research. Current progress with health services reform in China is far from satisfactory, without sufficient regulation. To achieve better progress, an effective tool for evaluating medical performance needs to be established. In view of this, this study attempted to develop such a tool appropriate for the Chinese context. Methods Data was collected from the front pages of medical records (FPMR) of all large general public hospitals (21 hospitals) in the third and fourth quarter of 2007. Locally developed Diagnosis Related Groups (DRGs) were introduced as a tool for risk adjustment and performance evaluation indicators were established: Charge Efficiency Index (CEI), Time Efficiency Index (TEI) and inpatient mortality of low-risk group cases (IMLRG), to reflect respectively work efficiency and medical service quality. Using these indicators, the inpatient services' performance was horizontally compared among hospitals. Case-mix Index (CMI) was used to adjust efficiency indices and then produce adjusted CEI (aCEI) and adjusted TEI (aTEI). Poisson distribution analysis was used to test the statistical significance of the IMLRG differences between different hospitals. Results Using the aCEI, aTEI and IMLRG scores for the 21 hospitals, Hospital A and C had relatively good overall performance because their medical charges were lower, LOS shorter and IMLRG smaller. The performance of Hospital P and Q was the worst due to their relatively high charge level, long LOS and high IMLRG. Various performance problems also existed in the other hospitals. Conclusion It is possible to develop an accurate and easy to run performance evaluation system using Case-Mix as the tool for risk adjustment, choosing indicators close to consumers and managers, and utilizing routine report forms as the basic information source. To keep such a system running effectively, it is necessary to improve the reliability of clinical information and the risk-adjustment ability of Case-Mix. PMID:19402913

Evaluation of the Effectiveness of Stormwater Decision Support Tools for Infrastructure Selection and the Barriers to Implementation

NASA Astrophysics Data System (ADS)

Spahr, K.; Hogue, T. S.

2016-12-01

Selecting the most appropriate green, gray, and / or hybrid system for stormwater treatment and conveyance can prove challenging to decision markers across all scales, from site managers to large municipalities. To help streamline the selection process, a multi-disciplinary team of academics and professionals is developing an industry standard for selecting and evaluating the most appropriate stormwater management technology for different regions. To make the tool more robust and comprehensive, life-cycle cost assessment and optimization modules will be included to evaluate non-monetized and ecosystem benefits of selected technologies. Initial work includes surveying advisory board members based in cities that use existing decision support tools in their infrastructure planning process. These surveys will qualify the decisions currently being made and identify challenges within the current planning process across a range of hydroclimatic regions and city size. Analysis of social and other non-technical barriers to adoption of the existing tools is also being performed, with identification of regional differences and institutional challenges. Surveys will also gage the regional appropriateness of certain stormwater technologies based off experiences in implementing stormwater treatment and conveyance plans. In additional to compiling qualitative data on existing decision support tools, a technical review of components of the decision support tool used will be performed. Gaps in each tool's analysis, like the lack of certain critical functionalities, will be identified and ease of use will be evaluated. Conclusions drawn from both the qualitative and quantitative analyses will be used to inform the development of the new decision support tool and its eventual dissemination.

Overcoming redundancies in bedside nursing assessments by validating a parsimonious meta-tool: findings from a methodological exercise study.

PubMed

Palese, Alvisa; Marini, Eva; Guarnier, Annamaria; Barelli, Paolo; Zambiasi, Paola; Allegrini, Elisabetta; Bazoli, Letizia; Casson, Paola; Marin, Meri; Padovan, Marisa; Picogna, Michele; Taddia, Patrizia; Chiari, Paolo; Salmaso, Daniele; Marognolli, Oliva; Canzan, Federica; Ambrosi, Elisa; Saiani, Luisa; Grassetti, Luca

2016-10-01

There is growing interest in validating tools aimed at supporting the clinical decision-making process and research. However, an increased bureaucratization of clinical practice and redundancies in the measures collected have been reported by clinicians. Redundancies in clinical assessments affect negatively both patients and nurses. To validate a meta-tool measuring the risks/problems currently estimated by multiple tools used in daily practice. A secondary analysis of a database was performed, using a cross-validation and a longitudinal study designs. In total, 1464 patients admitted to 12 medical units in 2012 were assessed at admission with the Brass, Barthel, Conley and Braden tools. Pertinent outcomes such as the occurrence of post-discharge need for resources and functional decline at discharge, as well as falls and pressure sores, were measured. Explorative factor analysis of each tool, inter-tool correlations and a conceptual evaluation of the redundant/similar items across tools were performed. Therefore, the validation of the meta-tool was performed through explorative factor analysis, confirmatory factor analysis and the structural equation model to establish the ability of the meta-tool to predict the outcomes estimated by the original tools. High correlations between the tools have emerged (from r 0.428 to 0.867) with a common variance from 18.3% to 75.1%. Through a conceptual evaluation and explorative factor analysis, the items were reduced from 42 to 20, and the three factors that emerged were confirmed by confirmatory factor analysis. According to the structural equation model results, two out of three emerged factors predicted the outcomes. From the initial 42 items, the meta-tool is composed of 20 items capable of predicting the outcomes as with the original tools. © 2016 John Wiley & Sons, Ltd.

FTAPE: A fault injection tool to measure fault tolerance

NASA Technical Reports Server (NTRS)

Tsai, Timothy K.; Iyer, Ravishankar K.

1995-01-01

The paper introduces FTAPE (Fault Tolerance And Performance Evaluator), a tool that can be used to compare fault-tolerant computers. The tool combines system-wide fault injection with a controllable workload. A workload generator is used to create high stress conditions for the machine. Faults are injected based on this workload activity in order to ensure a high level of fault propagation. The errors/fault ratio and performance degradation are presented as measures of fault tolerance.

Battery Storage Evaluation Tool, version 1.x

DOE Office of Scientific and Technical Information (OSTI.GOV)

2015-10-02

The battery storage evaluation tool developed at Pacific Northwest National Laboratory is used to run a one-year simulation to evaluate the benefits of battery storage for multiple grid applications, including energy arbitrage, balancing service, capacity value, distribution system equipment deferral, and outage mitigation. This tool is based on the optimal control strategies to capture multiple services from a single energy storage device. In this control strategy, at each hour, a lookahead optimization is first formulated and solved to determine the battery base operating point. The minute-by-minute simulation is then performed to simulate the actual battery operation.

Assisting allied health in performance evaluation: a systematic review.

PubMed

Lizarondo, Lucylynn; Grimmer, Karen; Kumar, Saravana

2014-11-14

Performance evaluation raises several challenges to allied health practitioners and there is no agreed approach to measuring or monitoring allied health service performance. The aim of this review was to examine the literature on performance evaluation in healthcare to assist in the establishment of a framework that can guide the measurement and evaluation of allied health clinical service performance. This review determined the core elements of a performance evaluation system, tools for evaluating performance, and barriers to the implementation of performance evaluation. A systematic review of the literature was undertaken. Five electronic databases were used to search for relevant articles: MEDLINE, Embase, CINAHL, PsychInfo, and Academic Search Premier. Articles which focussed on any allied health performance evaluation or those which examined performance in health care in general were considered in the review. Content analysis was used to synthesise the findings from individual articles. A total of 37 articles were included in the review. The literature suggests there are core elements involved in performance evaluation which include prioritising clinical areas for measurement, setting goals, selecting performance measures, identifying sources of feedback, undertaking performance measurement, and reporting the results to relevant stakeholders. The literature describes performance evaluation as multi-dimensional, requiring information or data from more than one perspective to provide a rich assessment of performance. A range of tools or instruments are available to capture various perspectives and gather a comprehensive picture of health care quality. Every allied health care delivery system has different performance needs and will therefore require different approaches. However, there are core processes that can be used as a framework to evaluate allied health performance. A careful examination of barriers to performance evaluation and subsequent tailoring of strategies to overcome these barriers should be undertaken to achieve the aims of performance evaluation. The findings of this review should inform the development of a standardised framework that can be used to measure and evaluate allied health performance. Future research should explore the utility and overall impact of such framework in allied health service delivery.

Solid waste management in primary healthcare centers: application of a facilitation tool 1

PubMed Central

Moreira, Ana Maria Maniero; Günther, Wanda Maria Risso

2016-01-01

Abstract Objectives: to propose a tool to facilitate diagnosis, formulation and evaluation of the Waste Management Plan in Primary Healthcare Centers and to present the results of the application in four selected units. Method: descriptive research, covering the stages of formulation /application of the proposed instrument and the evaluation of waste management performance at the units. Results: the tool consists in five forms; specific indicators of waste generation for outpatients healthcare units were proposed, and performance indicators that give scores for compliance with current legislation. In the studied units it is generated common waste (52-60%), infectious-sharps (31-42%) and recyclable (5-17%). The average rates of generation are: 0,09kg of total waste/outpatient assistance and 0,09kg of infectious-sharps waste/outpatient procedure. The compliance with regulations, initially 26-30%, then reached 30-38% a year later. Conclusion: the tool showed to be easy to use, bypassing the existence of a complex range of existing regulatory requirements, allowed to identify non-conformities, pointed out corrective measures and evaluated the performance of waste management. In this sense, it contributes to decision making and management practices relating to waste, tasks usually assigned to nurses. It is recommended that the tool be applied in similar healthcare units for comparative studies, and implementation of necessary adaptations for other medical services. PMID:27556874

A knowledge based search tool for performance measures in health care systems.

PubMed

Beyan, Oya D; Baykal, Nazife

2012-02-01

Performance measurement is vital for improving the health care systems. However, we are still far from having accepted performance measurement models. Researchers and developers are seeking comparable performance indicators. We developed an intelligent search tool to identify appropriate measures for specific requirements by matching diverse care settings. We reviewed the literature and analyzed 229 performance measurement studies published after 2000. These studies are evaluated with an original theoretical framework and stored in the database. A semantic network is designed for representing domain knowledge and supporting reasoning. We have applied knowledge based decision support techniques to cope with uncertainty problems. As a result we designed a tool which simplifies the performance indicator search process and provides most relevant indicators by employing knowledge based systems.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Carpenter, Alberta; Mann, Margaret; Gelman, Rachel

In evaluating next-generation materials and processes, the supply chain can have a large impact on the life cycle energy impacts. The Materials Flow through Industry (MFI) tool was developed for the Department of Energy's Advanced Manufacturing Office to be able to evaluate the energy impacts of the U.S. supply chain. The tool allows users to perform process comparisons, material substitutions, and grid modifications, and to see the effects of implementing sector efficiency potentials (Masanet, et al. 2009). This paper reviews the methodology of the tool and provides results around specific scenarios.

Assessment of a human computer interface prototyping environment

NASA Technical Reports Server (NTRS)

Moore, Loretta A.

1993-01-01

A Human Computer Interface (HCI) prototyping environment with embedded evaluation capability has been successfully assessed which will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. The HCI prototyping environment is designed to include four components: (1) a HCI format development tool, (2) a test and evaluation simulator development tool, (3) a dynamic, interactive interface between the HCI prototype and simulator, and (4) an embedded evaluation capability to evaluate the adequacy of an HCI based on a user's performance.

DSC: software tool for simulation-based design of control strategies applied to wastewater treatment plants.

PubMed

Ruano, M V; Ribes, J; Seco, A; Ferrer, J

2011-01-01

This paper presents a computer tool called DSC (Simulation based Controllers Design) that enables an easy design of control systems and strategies applied to wastewater treatment plants. Although the control systems are developed and evaluated by simulation, this tool aims to facilitate the direct implementation of the designed control system to the PC of the full-scale WWTP (wastewater treatment plants). The designed control system can be programmed in a dedicated control application and can be connected to either the simulation software or the SCADA of the plant. To this end, the developed DSC incorporates an OPC server (OLE for process control) which facilitates an open-standard communication protocol for different industrial process applications. The potential capabilities of the DSC tool are illustrated through the example of a full-scale application. An aeration control system applied to a nutrient removing WWTP was designed, tuned and evaluated with the DSC tool before its implementation in the full scale plant. The control parameters obtained by simulation were suitable for the full scale plant with only few modifications to improve the control performance. With the DSC tool, the control systems performance can be easily evaluated by simulation. Once developed and tuned by simulation, the control systems can be directly applied to the full-scale WWTP.

A benchmarking tool to evaluate computer tomography perfusion infarct core predictions against a DWI standard.

PubMed

Cereda, Carlo W; Christensen, Søren; Campbell, Bruce Cv; Mishra, Nishant K; Mlynash, Michael; Levi, Christopher; Straka, Matus; Wintermark, Max; Bammer, Roland; Albers, Gregory W; Parsons, Mark W; Lansberg, Maarten G

2016-10-01

Differences in research methodology have hampered the optimization of Computer Tomography Perfusion (CTP) for identification of the ischemic core. We aim to optimize CTP core identification using a novel benchmarking tool. The benchmarking tool consists of an imaging library and a statistical analysis algorithm to evaluate the performance of CTP. The tool was used to optimize and evaluate an in-house developed CTP-software algorithm. Imaging data of 103 acute stroke patients were included in the benchmarking tool. Median time from stroke onset to CT was 185 min (IQR 180-238), and the median time between completion of CT and start of MRI was 36 min (IQR 25-79). Volumetric accuracy of the CTP-ROIs was optimal at an rCBF threshold of <38%; at this threshold, the mean difference was 0.3 ml (SD 19.8 ml), the mean absolute difference was 14.3 (SD 13.7) ml, and CTP was 67% sensitive and 87% specific for identification of DWI positive tissue voxels. The benchmarking tool can play an important role in optimizing CTP software as it provides investigators with a novel method to directly compare the performance of alternative CTP software packages. © The Author(s) 2015.

Classification and assessment tools for structural motif discovery algorithms.

PubMed

Badr, Ghada; Al-Turaiki, Isra; Mathkour, Hassan

2013-01-01

Motif discovery is the problem of finding recurring patterns in biological data. Patterns can be sequential, mainly when discovered in DNA sequences. They can also be structural (e.g. when discovering RNA motifs). Finding common structural patterns helps to gain a better understanding of the mechanism of action (e.g. post-transcriptional regulation). Unlike DNA motifs, which are sequentially conserved, RNA motifs exhibit conservation in structure, which may be common even if the sequences are different. Over the past few years, hundreds of algorithms have been developed to solve the sequential motif discovery problem, while less work has been done for the structural case. In this paper, we survey, classify, and compare different algorithms that solve the structural motif discovery problem, where the underlying sequences may be different. We highlight their strengths and weaknesses. We start by proposing a benchmark dataset and a measurement tool that can be used to evaluate different motif discovery approaches. Then, we proceed by proposing our experimental setup. Finally, results are obtained using the proposed benchmark to compare available tools. To the best of our knowledge, this is the first attempt to compare tools solely designed for structural motif discovery. Results show that the accuracy of discovered motifs is relatively low. The results also suggest a complementary behavior among tools where some tools perform well on simple structures, while other tools are better for complex structures. We have classified and evaluated the performance of available structural motif discovery tools. In addition, we have proposed a benchmark dataset with tools that can be used to evaluate newly developed tools.

mdtmFTP and its evaluation on ESNET SDN testbed

DOE PAGES

Zhang, Liang; Wu, Wenji; DeMar, Phil; ...

2017-04-21

In this paper, to address the high-performance challenges of data transfer in the big data era, we are developing and implementing mdtmFTP: a high-performance data transfer tool for big data. mdtmFTP has four salient features. First, it adopts an I/O centric architecture to execute data transfer tasks. Second, it more efficiently utilizes the underlying multicore platform through optimized thread scheduling. Third, it implements a large virtual file mechanism to address the lots-of-small-files (LOSF) problem. In conclusion, mdtmFTP integrates multiple optimization mechanisms, including–zero copy, asynchronous I/O, pipelining, batch processing, and pre-allocated buffer pools–to enhance performance. mdtmFTP has been extensively tested andmore » evaluated within the ESNET 100G testbed. Evaluations show that mdtmFTP can achieve higher performance than existing data transfer tools, such as GridFTP, FDT, and BBCP.« less

Heterogeneous Roles and Heterogeneous Practices: Understanding the Adoption and Uses of Nonprofit Performance Evaluations

ERIC Educational Resources Information Center

Eckerd, Adam; Moulton, Stephanie

2011-01-01

Evaluating the performance of nonprofit organizations has been of growing importance for the last several decades. Although there is much good that can come out of self-improvement, there is substantial heterogeneity within the sector that calls into question the usefulness of "across the board" evaluation tools. In this article, the authors…

Evaluating Web 2.0 Technologies in Higher Education Using Students' Perceptions and Performance

ERIC Educational Resources Information Center

Karvounidis, T.; Chimos, K.; Bersimis, S.; Douligeris, C.

2014-01-01

In this work, Web 2.0 technologies in higher education are evaluated using students' perceptions, satisfaction, performance and behaviour. The study evaluates the Web 2.0 tools as stand-alone entities as well in terms of their cross-operability and integration (confluence) to synergistic contributions towards the enhancement of student…

The need for psycho-oncological support for melanoma patients: Central role of patients' self-evaluation.

PubMed

Mayer, Simone; Teufel, Martin; Schaeffeler, Norbert; Keim, Ulrike; Garbe, Claus; Eigentler, Thomas Kurt; Zipfel, Stephan; Forschner, Andrea

2017-09-01

Despite an increasing number of promising treatment options, only a limited number of studies concerning melanoma patients' psycho-oncological distress have been carried out. However, multiple screening tools are in use to assess the need for psycho-oncological support. This study aimed first to identify parameters in melanoma patients that are associated with a higher risk for being psycho-oncologically distressed and second to compare patients' self-evaluation concerning the need for psycho-oncological support with the results of established screening tools.We performed a cross-sectional study including 254 melanoma patients from the Center for Dermatooncology at the University of Tuebingen. The study was performed between June 2010 and February 2013. Several screening instruments were included: the Distress Thermometer (DT), Hospital Anxiety and Depression Scale and the patients' subjective evaluation concerning psycho-oncological support. Binary logistic regression was performed to identify factors that indicate the need for psycho-oncological support.Patients' subjective evaluation concerning the need for psycho-oncological support, female gender, and psychotherapeutic or psychiatric treatment at present or in the past had the highest impact on values above threshold in the DT. The odds ratio of patients' self-evaluation (9.89) was even higher than somatic factors like female gender (1.85), duration of illness (0.99), or increasing age (0.97). Patients' self-evaluation concerning the need for psycho-oncological support indicated a moderate correlation with the results of the screening tools included.In addition to the results obtained by screening tools like the DT, we could demonstrate that patients' self-evaluation is an important instrument to identify patients who need psycho-oncological support.

NetBenchmark: a bioconductor package for reproducible benchmarks of gene regulatory network inference.

PubMed

Bellot, Pau; Olsen, Catharina; Salembier, Philippe; Oliveras-Vergés, Albert; Meyer, Patrick E

2015-09-29

In the last decade, a great number of methods for reconstructing gene regulatory networks from expression data have been proposed. However, very few tools and datasets allow to evaluate accurately and reproducibly those methods. Hence, we propose here a new tool, able to perform a systematic, yet fully reproducible, evaluation of transcriptional network inference methods. Our open-source and freely available Bioconductor package aggregates a large set of tools to assess the robustness of network inference algorithms against different simulators, topologies, sample sizes and noise intensities. The benchmarking framework that uses various datasets highlights the specialization of some methods toward network types and data. As a result, it is possible to identify the techniques that have broad overall performances.

Performance evaluation of Bragg coherent diffraction imaging

NASA Astrophysics Data System (ADS)

Öztürk, H.; Huang, X.; Yan, H.; Robinson, I. K.; Noyan, I. C.; Chu, Y. S.

2017-10-01

In this study, we present a numerical framework for modeling three-dimensional (3D) diffraction data in Bragg coherent diffraction imaging (Bragg CDI) experiments and evaluating the quality of obtained 3D complex-valued real-space images recovered by reconstruction algorithms under controlled conditions. The approach is used to systematically explore the performance and the detection limit of this phase-retrieval-based microscopy tool. The numerical investigation suggests that the superb performance of Bragg CDI is achieved with an oversampling ratio above 30 and a detection dynamic range above 6 orders. The observed performance degradation subject to the data binning processes is also studied. This numerical tool can be used to optimize experimental parameters and has the potential to significantly improve the throughput of Bragg CDI method.

Winter maintenance performance measure.

DOT National Transportation Integrated Search

2016-01-01

The Winter Performance Index is a method of quantifying winter storm events and the DOTs response to them. : It is a valuable tool for evaluating the States maintenance practices, performing post-storm analysis, training : maintenance personnel...

Automated subtyping of HIV-1 genetic sequences for clinical and surveillance purposes: performance evaluation of the new REGA version 3 and seven other tools.

PubMed

Pineda-Peña, Andrea-Clemencia; Faria, Nuno Rodrigues; Imbrechts, Stijn; Libin, Pieter; Abecasis, Ana Barroso; Deforche, Koen; Gómez-López, Arley; Camacho, Ricardo J; de Oliveira, Tulio; Vandamme, Anne-Mieke

2013-10-01

To investigate differences in pathogenesis, diagnosis and resistance pathways between HIV-1 subtypes, an accurate subtyping tool for large datasets is needed. We aimed to evaluate the performance of automated subtyping tools to classify the different subtypes and circulating recombinant forms using pol, the most sequenced region in clinical practice. We also present the upgraded version 3 of the Rega HIV subtyping tool (REGAv3). HIV-1 pol sequences (PR+RT) for 4674 patients retrieved from the Portuguese HIV Drug Resistance Database, and 1872 pol sequences trimmed from full-length genomes retrieved from the Los Alamos database were classified with statistical-based tools such as COMET, jpHMM and STAR; similarity-based tools such as NCBI and Stanford; and phylogenetic-based tools such as REGA version 2 (REGAv2), REGAv3, and SCUEAL. The performance of these tools, for pol, and for PR and RT separately, was compared in terms of reproducibility, sensitivity and specificity with respect to the gold standard which was manual phylogenetic analysis of the pol region. The sensitivity and specificity for subtypes B and C was more than 96% for seven tools, but was variable for other subtypes such as A, D, F and G. With regard to the most common circulating recombinant forms (CRFs), the sensitivity and specificity for CRF01_AE was ~99% with statistical-based tools, with phylogenetic-based tools and with Stanford, one of the similarity based tools. CRF02_AG was correctly identified for more than 96% by COMET, REGAv3, Stanford and STAR. All the tools reached a specificity of more than 97% for most of the subtypes and the two main CRFs (CRF01_AE and CRF02_AG). Other CRFs were identified only by COMET, REGAv2, REGAv3, and SCUEAL and with variable sensitivity. When analyzing sequences for PR and RT separately, the performance for PR was generally lower and variable between the tools. Similarity and statistical-based tools were 100% reproducible, but this was lower for phylogenetic-based tools such as REGA (~99%) and SCUEAL (~96%). REGAv3 had an improved performance for subtype B and CRF02_AG compared to REGAv2 and is now able to also identify all epidemiologically relevant CRFs. In general the best performing tools, in alphabetical order, were COMET, jpHMM, REGAv3, and SCUEAL when analyzing pure subtypes in the pol region, and COMET and REGAv3 when analyzing most of the CRFs. Based on this study, we recommend to confirm subtyping with 2 well performing tools, and be cautious with the interpretation of short sequences. Copyright © 2013 The Authors. Published by Elsevier B.V. All rights reserved.

Development of task network models of human performance in microgravity

NASA Technical Reports Server (NTRS)

Diaz, Manuel F.; Adam, Susan

1992-01-01

This paper discusses the utility of task-network modeling for quantifying human performance variability in microgravity. The data are gathered for: (1) improving current methodologies for assessing human performance and workload in the operational space environment; (2) developing tools for assessing alternative system designs; and (3) developing an integrated set of methodologies for the evaluation of performance degradation during extended duration spaceflight. The evaluation entailed an analysis of the Remote Manipulator System payload-grapple task performed on many shuttle missions. Task-network modeling can be used as a tool for assessing and enhancing human performance in man-machine systems, particularly for modeling long-duration manned spaceflight. Task-network modeling can be directed toward improving system efficiency by increasing the understanding of basic capabilities of the human component in the system and the factors that influence these capabilities.

Expressions of personhood in Alzheimer's disease: an evaluation of research-based theatre as a pedagogical tool.

PubMed

Kontos, Pia C; Naglie, Gary

2007-07-01

A growing number of scholars are turning to dramatic performance as an innovative approach to disseminating the results of qualitative research. With an interest in this aesthetic practice, the authors transformed ethnographic research on personhood in Alzheimer's disease into a dramatic production, Expressions of Personhood in Alzheimer's , which was performed at the outset of focus group discussions with health practitioners. In this article, the authors report the results of an evaluation of health practitioners' perceptions of this theatrical adaptation as a pedagogical tool. Data from the focus group discussions and a self-administered post-performance survey suggest that there was a perceived relevance of the production to practitioners' practices, an increased understanding gained from attending the production, and a strong endorsement of the use of drama as an educational tool for disseminating information about dementia care.

Benchmarking government action for obesity prevention--an innovative advocacy strategy.

PubMed

Martin, J; Peeters, A; Honisett, S; Mavoa, H; Swinburn, B; de Silva-Sanigorski, A

2014-01-01

Successful obesity prevention will require a leading role for governments, but internationally they have been slow to act. League tables of benchmark indicators of action can be a valuable advocacy and evaluation tool. To develop a benchmarking tool for government action on obesity prevention, implement it across Australian jurisdictions and to publicly award the best and worst performers. A framework was developed which encompassed nine domains, reflecting best practice government action on obesity prevention: whole-of-government approaches; marketing restrictions; access to affordable, healthy food; school food and physical activity; food in public facilities; urban design and transport; leisure and local environments; health services, and; social marketing. A scoring system was used by non-government key informants to rate the performance of their government. National rankings were generated and the results were communicated to all Premiers/Chief Ministers, the media and the national obesity research and practice community. Evaluation of the initial tool in 2010 showed it to be feasible to implement and able to discriminate the better and worse performing governments. Evaluation of the rubric in 2011 confirmed this to be a robust and useful method. In relation to government action, the best performing governments were those with whole-of-government approaches, had extended common initiatives and demonstrated innovation and strong political will. This new benchmarking tool, the Obesity Action Award, has enabled identification of leading government action on obesity prevention and the key characteristics associated with their success. We recommend this tool for other multi-state/country comparisons. Copyright © 2013 Asian Oceanian Association for the Study of Obesity. Published by Elsevier Ltd. All rights reserved.

NEXT GENERATION ANALYSIS SOFTWARE FOR COMPONENT EVALUATION - Results of Rotational Seismometer Evaluation

NASA Astrophysics Data System (ADS)

Hart, D. M.; Merchant, B. J.; Abbott, R. E.

2012-12-01

The Component Evaluation project at Sandia National Laboratories supports the Ground-based Nuclear Explosion Monitoring program by performing testing and evaluation of the components that are used in seismic and infrasound monitoring systems. In order to perform this work, Component Evaluation maintains a testing facility called the FACT (Facility for Acceptance, Calibration, and Testing) site, a variety of test bed equipment, and a suite of software tools for analyzing test data. Recently, Component Evaluation has successfully integrated several improvements to its software analysis tools and test bed equipment that have substantially improved our ability to test and evaluate components. The software tool that is used to analyze test data is called TALENT: Test and AnaLysis EvaluatioN Tool. TALENT is designed to be a single, standard interface to all test configuration, metadata, parameters, waveforms, and results that are generated in the course of testing monitoring systems. It provides traceability by capturing everything about a test in a relational database that is required to reproduce the results of that test. TALENT provides a simple, yet powerful, user interface to quickly acquire, process, and analyze waveform test data. The software tool has also been expanded recently to handle sensors whose output is proportional to rotation angle, or rotation rate. As an example of this new processing capability, we show results from testing the new ATA ARS-16 rotational seismometer. The test data was collected at the USGS ASL. Four datasets were processed: 1) 1 Hz with increasing amplitude, 2) 4 Hz with increasing amplitude, 3) 16 Hz with increasing amplitude and 4) twenty-six discrete frequencies between 0.353 Hz to 64 Hz. The results are compared to manufacture-supplied data sheets.

Computer-Based Tools for Evaluating Graphical User Interfaces

NASA Technical Reports Server (NTRS)

Moore, Loretta A.

1997-01-01

The user interface is the component of a software system that connects two very complex system: humans and computers. Each of these two systems impose certain requirements on the final product. The user is the judge of the usability and utility of the system; the computer software and hardware are the tools with which the interface is constructed. Mistakes are sometimes made in designing and developing user interfaces because the designers and developers have limited knowledge about human performance (e.g., problem solving, decision making, planning, and reasoning). Even those trained in user interface design make mistakes because they are unable to address all of the known requirements and constraints on design. Evaluation of the user inter-face is therefore a critical phase of the user interface development process. Evaluation should not be considered the final phase of design; but it should be part of an iterative design cycle with the output of evaluation being feed back into design. The goal of this research was to develop a set of computer-based tools for objectively evaluating graphical user interfaces. The research was organized into three phases. The first phase resulted in the development of an embedded evaluation tool which evaluates the usability of a graphical user interface based on a user's performance. An expert system to assist in the design and evaluation of user interfaces based upon rules and guidelines was developed during the second phase. During the final phase of the research an automatic layout tool to be used in the initial design of graphical inter- faces was developed. The research was coordinated with NASA Marshall Space Flight Center's Mission Operations Laboratory's efforts in developing onboard payload display specifications for the Space Station.

Instruction Guide and Macro Analysis Tool for Community-led Air Monitoring

EPA Pesticide Factsheets

EPA has developed two tools for evaluating the performance of low-cost sensors and interpreting the data they collect to help citizen scientists, communities, and professionals interested in learning about local air quality.

Documenting Teacher Candidates' Professional Growth through Performance Evaluation

ERIC Educational Resources Information Center

Brown, Elizabeth Levine; Suh, Jennifer; Parsons, Seth A.; Parker, Audra K.; Ramirez, Erin M.

2015-01-01

In the United States, colleges of education are responding to demands for increased accountability. The purpose of this article is to describe one teacher education program's implementation of a performance evaluation tool during final internship that measures teacher candidates' development across four domains: Planning and Preparation,…

A Data-Driven Design Evaluation Tool for Handheld Device Soft Keyboards

PubMed Central

Trudeau, Matthieu B.; Sunderland, Elsie M.; Jindrich, Devin L.; Dennerlein, Jack T.

2014-01-01

Thumb interaction is a primary technique used to operate small handheld devices such as smartphones. Despite the different techniques involved in operating a handheld device compared to a personal computer, the keyboard layouts for both devices are similar. A handheld device keyboard that considers the physical capabilities of the thumb may improve user experience. We developed and applied a design evaluation tool for different geometries of the QWERTY keyboard using a performance evaluation model. The model utilizes previously collected data on thumb motor performance and posture for different tap locations and thumb movement directions. We calculated a performance index (PITOT, 0 is worst and 2 is best) for 663 designs consisting in different combinations of three variables: the keyboard's radius of curvature (R) (mm), orientation (O) (°), and vertical location on the screen (L). The current standard keyboard performed poorly (PITOT = 0.28) compared to other designs considered. Keyboard location (L) contributed to the greatest variability in performance out of the three design variables, suggesting that designers should modify this variable first. Performance was greatest for designs in the middle keyboard location. In addition, having a slightly upward curve (R = −20 mm) and orientated perpendicular to the thumb's long axis (O = −20°) improved performance to PITOT = 1.97. Poorest performances were associated with placement of the keyboard's spacebar in the bottom right corner of the screen (e.g., the worst was for R = 20 mm, O = 40°, L =  Bottom (PITOT = 0.09)). While this evaluation tool can be used in the design process as an ergonomic reference to promote user motor performance, other design variables such as visual access and usability still remain unexplored. PMID:25211465

Interagency Transition Team Development and Facilitation. Essential Tools.

ERIC Educational Resources Information Center

Stodden, Robert A.; Brown, Steven E.; Galloway, L. M.; Mrazek, Susan; Noy, Liora

2005-01-01

The purpose of this Essential Tool is to assist state-level transition coordinators and others responsible for forming, conducting, and evaluating the performance of interagency transition teams that are focused upon the school and post-school needs of youth with disabilities. This Essential Tool is designed to guide the coordination efforts of…

Report Central: quality reporting tool in an electronic health record.

PubMed

Jung, Eunice; Li, Qi; Mangalampalli, Anil; Greim, Julie; Eskin, Michael S; Housman, Dan; Isikoff, Jeremy; Abend, Aaron H; Middleton, Blackford; Einbinder, Jonathan S

2006-01-01

Quality reporting tools, integrated with ambulatory electronic health records, can help clinicians and administrators understand performance, manage populations, and improve quality. Report Central is a secure web report delivery tool built on Crystal Reports XItrade mark and ASP.NET technologies. Pilot evaluation of Report Central indicates that clinicians prefer a quality reporting tool that is integrated with our home-grown EHR to support clinical workflow.

An Integrated Knowledge Translation Experience

PubMed Central

Bagatto, Marlene P.; Miller, Linda T.; Kothari, Anita; Seewald, Richard; Scollie, Susan D.

2011-01-01

Pediatric audiologists lack evidence-based, age-appropriate outcome evaluation tools with well-developed normative data that could be used to evaluate the auditory development and performance of children aged birth to 6 years with permanent childhood hearing impairment. Bagatto and colleagues recommend a battery of outcome tools that may be used with this population. This article provides results of an evaluation of the individual components of the University of Western Ontario Pediatric Audiological Monitoring Protocol (UWO PedAMP) version 1.0 by the audiologists associated with the Network of Pediatric Audiologists of Canada. It also provides information regarding barriers and facilitators to implementing outcome measures in clinical practice. Results indicate that when compared to the Parents’ Evaluation of Aural/Oral Performance of Children (PEACH) Diary, audiologists found the PEACH Rating Scale to be a more clinically feasible evaluation tool to implement in practice from a time, task, and consistency of use perspective. Results also indicate that the LittlEARS® Auditory Questionnaire could be used to evaluate the auditory development and performance of children aged birth to 6 years with permanent childhood hearing impairment (PCHI). The most cited barrier to implementation is time. The result of this social collaboration was the creation of a knowledge product, the UWO PedAMP v1.0, which has the potential to be useful to audiologists and the children and families they serve. PMID:22194315

Perceptions of nursing students after performing an individual activity designed to develop their critical thinking: The "critical card" tool.

PubMed

Urcola-Pardo, Fernando; Blázquez-Ornat, Isabel; Anguas-Gracia, Ana; Gasch-Gallen, Ángel; Germán-Bes, Concepción

2018-03-01

Critical thinking in Health Sciences is among the transversal competences in the Nursing Degree. The critical card is a tool of individual learning, designed to develop critical thinking, and set in the process of environmental health learning. Every student must perform the activity to obtain the highest qualification in Community Health Nursing subject. The aim of this project was to evaluate this learning tool using the students' perceptions after its performance. The evaluation was based on the answers to a questionnaire obtained from the third course students of Nursing Degree at the University of Zaragoza. The questionnaire was made up of 14 Likert-type questions, grouped in four dimensions. The student participation rate was higher than 50%. The analysis of the questionnaire obtained 67,8% positive answers. The variability between dimensions ranged between 49% of positive answers for application in other subjects and 87% of positive answers for the improvements applicable to the instrument. The students coincided in indicating that the critical card is a useful learning tool and could be applicable in other subjects. However, the weight it is given in the global evaluation of the subject is considered to be too low, considering the time used to complete the activity. Copyright © 2017 Elsevier Ltd. All rights reserved.

A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools

PubMed Central

2012-01-01

Background We introduce the linguistic annotation of a corpus of 97 full-text biomedical publications, known as the Colorado Richly Annotated Full Text (CRAFT) corpus. We further assess the performance of existing tools for performing sentence splitting, tokenization, syntactic parsing, and named entity recognition on this corpus. Results Many biomedical natural language processing systems demonstrated large differences between their previously published results and their performance on the CRAFT corpus when tested with the publicly available models or rule sets. Trainable systems differed widely with respect to their ability to build high-performing models based on this data. Conclusions The finding that some systems were able to train high-performing models based on this corpus is additional evidence, beyond high inter-annotator agreement, that the quality of the CRAFT corpus is high. The overall poor performance of various systems indicates that considerable work needs to be done to enable natural language processing systems to work well when the input is full-text journal articles. The CRAFT corpus provides a valuable resource to the biomedical natural language processing community for evaluation and training of new models for biomedical full text publications. PMID:22901054

Instrumentation, performance visualization, and debugging tools for multiprocessors

NASA Technical Reports Server (NTRS)

Yan, Jerry C.; Fineman, Charles E.; Hontalas, Philip J.

1991-01-01

The need for computing power has forced a migration from serial computation on a single processor to parallel processing on multiprocessor architectures. However, without effective means to monitor (and visualize) program execution, debugging, and tuning parallel programs becomes intractably difficult as program complexity increases with the number of processors. Research on performance evaluation tools for multiprocessors is being carried out at ARC. Besides investigating new techniques for instrumenting, monitoring, and presenting the state of parallel program execution in a coherent and user-friendly manner, prototypes of software tools are being incorporated into the run-time environments of various hardware testbeds to evaluate their impact on user productivity. Our current tool set, the Ames Instrumentation Systems (AIMS), incorporates features from various software systems developed in academia and industry. The execution of FORTRAN programs on the Intel iPSC/860 can be automatically instrumented and monitored. Performance data collected in this manner can be displayed graphically on workstations supporting X-Windows. We have successfully compared various parallel algorithms for computational fluid dynamics (CFD) applications in collaboration with scientists from the Numerical Aerodynamic Simulation Systems Division. By performing these comparisons, we show that performance monitors and debuggers such as AIMS are practical and can illuminate the complex dynamics that occur within parallel programs.

Self-Reacting Friction Stir Welding for Aluminum Alloy Circumferential Weld Applications

NASA Technical Reports Server (NTRS)

Bjorkman, Gerry; Cantrell, Mark; Carter, Robert

2003-01-01

Friction stir welding is an innovative weld process that continues to grow in use, in the commercial, defense, and space sectors. It produces high quality and high strength welds in aluminum alloys. The process consists of a rotating weld pin tool that plasticizes material through friction. The plasticized material is welded by applying a high weld forge force through the weld pin tool against the material during pin tool rotation. The high weld forge force is reacted against an anvil and a stout tool structure. A variation of friction stir welding currently being evaluated is self-reacting friction stir welding. Self-reacting friction stir welding incorporates two opposing shoulders on the crown and root sides of the weld joint. In self-reacting friction stir welding, the weld forge force is reacted against the crown shoulder portion of the weld pin tool by the root shoulder. This eliminates the need for a stout tooling structure to react the high weld forge force required in the typical friction stir weld process. Therefore, the self-reacting feature reduces tooling requirements and, therefore, process implementation costs. This makes the process attractive for aluminum alloy circumferential weld applications. To evaluate the application of self-reacting friction stir welding for aluminum alloy circumferential welding, a feasibility study was performed. The study consisted of performing a fourteen-foot diameter aluminum alloy circumferential demonstration weld using typical fusion weld tooling. To accomplish the demonstration weld, weld and tack weld development were performed and fourteen-foot diameter rings were fabricated. Weld development consisted of weld pin tool selection and the generation of a process map and envelope. Tack weld development evaluated gas tungsten arc welding and friction stir welding for tack welding rings together for circumferential welding. As a result of the study, a successful circumferential demonstration weld was produced leading the way for future circumferential weld implementation.

Comparing the performance of biomedical clustering methods.

PubMed

Wiwie, Christian; Baumbach, Jan; Röttger, Richard

2015-11-01

Identifying groups of similar objects is a popular first step in biomedical data analysis, but it is error-prone and impossible to perform manually. Many computational methods have been developed to tackle this problem. Here we assessed 13 well-known methods using 24 data sets ranging from gene expression to protein domains. Performance was judged on the basis of 13 common cluster validity indices. We developed a clustering analysis platform, ClustEval (http://clusteval.mpi-inf.mpg.de), to promote streamlined evaluation, comparison and reproducibility of clustering results in the future. This allowed us to objectively evaluate the performance of all tools on all data sets with up to 1,000 different parameter sets each, resulting in a total of more than 4 million calculated cluster validity indices. We observed that there was no universal best performer, but on the basis of this wide-ranging comparison we were able to develop a short guideline for biomedical clustering tasks. ClustEval allows biomedical researchers to pick the appropriate tool for their data type and allows method developers to compare their tool to the state of the art.

Using Coupled Energy, Airflow and IAQ Software (TRNSYS/CONTAM) to Evaluate Building Ventilation Strategies.

PubMed

Dols, W Stuart; Emmerich, Steven J; Polidoro, Brian J

2016-03-01

Building energy analysis tools are available in many forms that provide the ability to address a broad spectrum of energy-related issues in various combinations. Often these tools operate in isolation from one another, making it difficult to evaluate the interactions between related phenomena and interacting systems, forcing oversimplified assumptions to be made about various phenomena that could otherwise be addressed directly with another tool. One example of such interdependence is the interaction between heat transfer, inter-zone airflow and indoor contaminant transport. In order to better address these interdependencies, the National Institute of Standards and Technology (NIST) has developed an updated version of the multi-zone airflow and contaminant transport modelling tool, CONTAM, along with a set of utilities to enable coupling of the full CONTAM model with the TRNSYS simulation tool in a more seamless manner and with additional capabilities that were previously not available. This paper provides an overview of these new capabilities and applies them to simulating a medium-size office building. These simulations address the interaction between whole-building energy, airflow and contaminant transport in evaluating various ventilation strategies including natural and demand-controlled ventilation. CONTAM has been in practical use for many years allowing building designers, as well as IAQ and ventilation system analysts, to simulate the complex interactions between building physical layout and HVAC system configuration in determining building airflow and contaminant transport. It has been widely used to design and analyse smoke management systems and evaluate building performance in response to chemical, biological and radiological events. While CONTAM has been used to address design and performance of buildings implementing energy conserving ventilation systems, e.g., natural and hybrid, this new coupled simulation capability will enable users to apply the tool to couple CONTAM with existing energy analysis software to address the interaction between indoor air quality considerations and energy conservation measures in building design and analysis. This paper presents two practical case studies using the coupled modelling tool to evaluate IAQ performance of a CO 2 -based demand-controlled ventilation system under different levels of building envelope airtightness and the design and analysis of a natural ventilation system.

Using Coupled Energy, Airflow and IAQ Software (TRNSYS/CONTAM) to Evaluate Building Ventilation Strategies

PubMed Central

Dols, W. Stuart.; Emmerich, Steven J.; Polidoro, Brian J.

2016-01-01

Building energy analysis tools are available in many forms that provide the ability to address a broad spectrum of energy-related issues in various combinations. Often these tools operate in isolation from one another, making it difficult to evaluate the interactions between related phenomena and interacting systems, forcing oversimplified assumptions to be made about various phenomena that could otherwise be addressed directly with another tool. One example of such interdependence is the interaction between heat transfer, inter-zone airflow and indoor contaminant transport. In order to better address these interdependencies, the National Institute of Standards and Technology (NIST) has developed an updated version of the multi-zone airflow and contaminant transport modelling tool, CONTAM, along with a set of utilities to enable coupling of the full CONTAM model with the TRNSYS simulation tool in a more seamless manner and with additional capabilities that were previously not available. This paper provides an overview of these new capabilities and applies them to simulating a medium-size office building. These simulations address the interaction between whole-building energy, airflow and contaminant transport in evaluating various ventilation strategies including natural and demand-controlled ventilation. Practical Application CONTAM has been in practical use for many years allowing building designers, as well as IAQ and ventilation system analysts, to simulate the complex interactions between building physical layout and HVAC system configuration in determining building airflow and contaminant transport. It has been widely used to design and analyse smoke management systems and evaluate building performance in response to chemical, biological and radiological events. While CONTAM has been used to address design and performance of buildings implementing energy conserving ventilation systems, e.g., natural and hybrid, this new coupled simulation capability will enable users to apply the tool to couple CONTAM with existing energy analysis software to address the interaction between indoor air quality considerations and energy conservation measures in building design and analysis. This paper presents two practical case studies using the coupled modelling tool to evaluate IAQ performance of a CO2-based demand-controlled ventilation system under different levels of building envelope airtightness and the design and analysis of a natural ventilation system. PMID:27099405

Evaluating evaluation forms form.

PubMed

Smith, Roger P

2004-02-01

To provide a tool for evaluating evaluation forms. A new form has been developed and tested on itself and a sample of evaluation forms obtained from the graduate medical education offices of several local universities. Additional forms from hospital administration were also subjected to analysis. The new form performed well when applied to itself. The form performed equally well when applied to the other (subject) forms, although their scores were embarrassingly poor. A new form for evaluating evaluation forms is needed, useful, and now available.

Open source tools for ATR development and performance evaluation

NASA Astrophysics Data System (ADS)

Baumann, James M.; Dilsavor, Ronald L.; Stubbles, James; Mossing, John C.

2002-07-01

Early in almost every engineering project, a decision must be made about tools; should I buy off-the-shelf tools or should I develop my own. Either choice can involve significant cost and risk. Off-the-shelf tools may be readily available, but they can be expensive to purchase and to maintain licenses, and may not be flexible enough to satisfy all project requirements. On the other hand, developing new tools permits great flexibility, but it can be time- (and budget-) consuming, and the end product still may not work as intended. Open source software has the advantages of both approaches without many of the pitfalls. This paper examines the concept of open source software, including its history, unique culture, and informal yet closely followed conventions. These characteristics influence the quality and quantity of software available, and ultimately its suitability for serious ATR development work. We give an example where Python, an open source scripting language, and OpenEV, a viewing and analysis tool for geospatial data, have been incorporated into ATR performance evaluation projects. While this case highlights the successful use of open source tools, we also offer important insight into risks associated with this approach.

Ensembles of NLP Tools for Data Element Extraction from Clinical Notes

PubMed Central

Kuo, Tsung-Ting; Rao, Pallavi; Maehara, Cleo; Doan, Son; Chaparro, Juan D.; Day, Michele E.; Farcas, Claudiu; Ohno-Machado, Lucila; Hsu, Chun-Nan

2016-01-01

Natural Language Processing (NLP) is essential for concept extraction from narrative text in electronic health records (EHR). To extract numerous and diverse concepts, such as data elements (i.e., important concepts related to a certain medical condition), a plausible solution is to combine various NLP tools into an ensemble to improve extraction performance. However, it is unclear to what extent ensembles of popular NLP tools improve the extraction of numerous and diverse concepts. Therefore, we built an NLP ensemble pipeline to synergize the strength of popular NLP tools using seven ensemble methods, and to quantify the improvement in performance achieved by ensembles in the extraction of data elements for three very different cohorts. Evaluation results show that the pipeline can improve the performance of NLP tools, but there is high variability depending on the cohort. PMID:28269947

Ensembles of NLP Tools for Data Element Extraction from Clinical Notes.

PubMed

Kuo, Tsung-Ting; Rao, Pallavi; Maehara, Cleo; Doan, Son; Chaparro, Juan D; Day, Michele E; Farcas, Claudiu; Ohno-Machado, Lucila; Hsu, Chun-Nan

2016-01-01

Natural Language Processing (NLP) is essential for concept extraction from narrative text in electronic health records (EHR). To extract numerous and diverse concepts, such as data elements (i.e., important concepts related to a certain medical condition), a plausible solution is to combine various NLP tools into an ensemble to improve extraction performance. However, it is unclear to what extent ensembles of popular NLP tools improve the extraction of numerous and diverse concepts. Therefore, we built an NLP ensemble pipeline to synergize the strength of popular NLP tools using seven ensemble methods, and to quantify the improvement in performance achieved by ensembles in the extraction of data elements for three very different cohorts. Evaluation results show that the pipeline can improve the performance of NLP tools, but there is high variability depending on the cohort.

Design of Friction Stir Spot Welding Tools by Using a Novel Thermal-Mechanical Approach

PubMed Central

Su, Zheng-Ming; Qiu, Qi-Hong; Lin, Pai-Chen

2016-01-01

A simple thermal-mechanical model for friction stir spot welding (FSSW) was developed to obtain similar weld performance for different weld tools. Use of the thermal-mechanical model and a combined approach enabled the design of weld tools for various sizes but similar qualities. Three weld tools for weld radii of 4, 5, and 6 mm were made to join 6061-T6 aluminum sheets. Performance evaluations of the three weld tools compared fracture behavior, microstructure, micro-hardness distribution, and welding temperature of welds in lap-shear specimens. For welds made by the three weld tools under identical processing conditions, failure loads were approximately proportional to tool size. Failure modes, microstructures, and micro-hardness distributions were similar. Welding temperatures correlated with frictional heat generation rate densities. Because the three weld tools sufficiently met all design objectives, the proposed approach is considered a simple and feasible guideline for preliminary tool design. PMID:28773800

Design of Friction Stir Spot Welding Tools by Using a Novel Thermal-Mechanical Approach.

PubMed

Su, Zheng-Ming; Qiu, Qi-Hong; Lin, Pai-Chen

2016-08-09

A simple thermal-mechanical model for friction stir spot welding (FSSW) was developed to obtain similar weld performance for different weld tools. Use of the thermal-mechanical model and a combined approach enabled the design of weld tools for various sizes but similar qualities. Three weld tools for weld radii of 4, 5, and 6 mm were made to join 6061-T6 aluminum sheets. Performance evaluations of the three weld tools compared fracture behavior, microstructure, micro-hardness distribution, and welding temperature of welds in lap-shear specimens. For welds made by the three weld tools under identical processing conditions, failure loads were approximately proportional to tool size. Failure modes, microstructures, and micro-hardness distributions were similar. Welding temperatures correlated with frictional heat generation rate densities. Because the three weld tools sufficiently met all design objectives, the proposed approach is considered a simple and feasible guideline for preliminary tool design.

Validation of Tendril TrueHome Using Software-to-Software Comparison

DOE Office of Scientific and Technical Information (OSTI.GOV)

Maguire, Jeffrey B; Horowitz, Scott G; Moore, Nathan

This study performed comparative evaluation of EnergyPlus version 8.6 and Tendril TrueHome, two physics-based home energy simulation models, to identify differences in energy consumption predictions between the two programs and resolve discrepancies between them. EnergyPlus is considered a benchmark, best-in-class software tool for building energy simulation. This exercise sought to improve both software tools through additional evaluation/scrutiny.

Developing a practical evaluation tool for preceptor use.

PubMed

Walsh, Catherine M; Seldomridge, Lisa A; Badros, Karen K

2008-01-01

After years of dissatisfaction with existing instruments, a tool for preceptors to evaluate an undergraduate student's clinical performance was developed, with preceptors' input in its construction. A 2-year pilot evaluation revealed notable problems including excessively high preceptor ratings and significant disparities between faculty and preceptor ratings. Further revisions were made, reducing indicators to those which the preceptors can actually evaluate on an everyday basis and developing a rubric. Additional recommendations to bolster the quality of ratings are improving orientation and guidance of preceptors and modifying procedures for giving feedback.

Electrochemical impedance spectroscopy (EIS) as a tool for measuring corrosion of polymer-coated fasteners used in treated wood

Treesearch

Samuel L. Zelinka; Lorraine Ortiz-Candelaria; Donald S. Stone; Douglas R. Rammer

2009-01-01

Currently, many of the polymer-coated fasteners on the market are designed for improved corrosion performance in treated wood; yet, there is no way to evaluate their corrosion performance. In this study, a common technique for measuring the corrosion performance of polymer-coated metals, electrochemical impedance spectroscopy (EIS), was used to evaluate commercial...

High-Speed Edge Trimming of CFRP and Online Monitoring of Performance of Router Tools Using Acoustic Emission

PubMed Central

Prakash, Rangasamy; Krishnaraj, Vijayan; Zitoune, Redouane; Sheikh-Ahmad, Jamal

2016-01-01

Carbon fiber reinforced polymers (CFRPs) have found wide-ranging applications in numerous industrial fields such as aerospace, automotive, and shipping industries due to their excellent mechanical properties that lead to enhanced functional performance. In this paper, an experimental study on edge trimming of CFRP was done with various cutting conditions and different geometry of tools such as helical-, fluted-, and burr-type tools. The investigation involves the measurement of cutting forces for the different machining conditions and its effect on the surface quality of the trimmed edges. The modern cutting tools (router tools or burr tools) selected for machining CFRPs, have complex geometries in cutting edges and surfaces, and therefore a traditional method of direct tool wear evaluation is not applicable. An acoustic emission (AE) sensing was employed for on-line monitoring of the performance of router tools to determine the relationship between AE signal and length of machining for different kinds of geometry of tools. The investigation showed that the router tool with a flat cutting edge has better performance by generating lower cutting force and better surface finish with no delamination on trimmed edges. The mathematical modeling for the prediction of cutting forces was also done using Artificial Neural Network and Regression Analysis. PMID:28773919

Evaluation of a novel Serious Game based assessment tool for patients with Alzheimer's disease.

PubMed

Vallejo, Vanessa; Wyss, Patric; Rampa, Luca; Mitache, Andrei V; Müri, René M; Mosimann, Urs P; Nef, Tobias

2017-01-01

Despite growing interest in developing ecological assessment of difficulties in patients with Alzheimer's disease new methods assessing the cognitive difficulties related to functional activities are missing. To complete current evaluation, the use of Serious Games can be a promising approach as it offers the possibility to recreate a virtual environment with daily living activities and a precise and complete cognitive evaluation. The aim of the present study was to evaluate the usability and the screening potential of a new ecological tool for assessment of cognitive functions in patients with Alzheimer's disease. Eighteen patients with Alzheimer's disease and twenty healthy controls participated to the study. They were asked to complete six daily living virtual tasks assessing several cognitive functions: three navigation tasks, one shopping task, one cooking task and one table preparation task following a one-day scenario. Usability of the game was evaluated through a questionnaire and through the analysis of the computer interactions for the two groups. Furthermore, the performances in terms of time to achieve the task and percentage of completion on the several tasks were recorded. Results indicate that both groups subjectively found the game user friendly and they were objectively able to play the game without computer interactions difficulties. Comparison of the performances between the two groups indicated a significant difference in terms of percentage of achievement of the several tasks and in terms of time they needed to achieve the several tasks. This study suggests that this new Serious Game based assessment tool is a user-friendly and ecological method to evaluate the cognitive abilities related to the difficulties patients can encounter in daily living activities and can be used as a screening tool as it allowed to distinguish Alzheimer's patient's performance from healthy controls.

Integrated modeling tool for performance engineering of complex computer systems

NASA Technical Reports Server (NTRS)

Wright, Gary; Ball, Duane; Hoyt, Susan; Steele, Oscar

1989-01-01

This report summarizes Advanced System Technologies' accomplishments on the Phase 2 SBIR contract NAS7-995. The technical objectives of the report are: (1) to develop an evaluation version of a graphical, integrated modeling language according to the specification resulting from the Phase 2 research; and (2) to determine the degree to which the language meets its objectives by evaluating ease of use, utility of two sets of performance predictions, and the power of the language constructs. The technical approach followed to meet these objectives was to design, develop, and test an evaluation prototype of a graphical, performance prediction tool. The utility of the prototype was then evaluated by applying it to a variety of test cases found in the literature and in AST case histories. Numerous models were constructed and successfully tested. The major conclusion of this Phase 2 SBIR research and development effort is that complex, real-time computer systems can be specified in a non-procedural manner using combinations of icons, windows, menus, and dialogs. Such a specification technique provides an interface that system designers and architects find natural and easy to use. In addition, PEDESTAL's multiview approach provides system engineers with the capability to perform the trade-offs necessary to produce a design that meets timing performance requirements. Sample system designs analyzed during the development effort showed that models could be constructed in a fraction of the time required by non-visual system design capture tools.

A flexible tool for hydraulic and water quality performance analysis of green infrastructure

NASA Astrophysics Data System (ADS)

Massoudieh, A.; Alikhani, J.

2017-12-01

Models that allow for design considerations of green infrastructure (GI) practices to control stormwater runoff and associated contaminants have received considerable attention in recent years. To be used to evaluate the effect design configurations on the long-term performance of GIs, models should be able to consider processes within GIs with good fidelity. In this presentation, a sophisticated, yet flexible tool for hydraulic and water quality assessment of GIs will be introduced. The tool can be used by design engineers and researchers to capture and explore the effect of design factors and properties of the media employed in the performance of GI systems at a relatively small scale. We deemed it essential to have a flexible GI modeling tool that is capable of simulating GI system components and specific biogeochemical processes affecting contaminants such as evapotranspiration, plant uptake, reactions, and particle-associated transport accurately while maintaining a high degree of flexibility to account for the myriad of GI alternatives. The mathematical framework for a stand-alone GI performance assessment tool has been developed and will be demonstrated. The process-based model framework developed here can be used to model a diverse range of GI practices such as stormwater ponds, green roofs, retention ponds, bioretention systems, infiltration trench, permeable pavement and other custom-designed combinatory systems. An example of the application of the system to evaluate the performance of a rain-garden system will be demonstrated.

Applications of an architecture design and assessment system (ADAS)

NASA Technical Reports Server (NTRS)

Gray, F. Gail; Debrunner, Linda S.; White, Tennis S.

1988-01-01

A new Architecture Design and Assessment System (ADAS) tool package is introduced, and a range of possible applications is illustrated. ADAS was used to evaluate the performance of an advanced fault-tolerant computer architecture in a modern flight control application. Bottlenecks were identified and possible solutions suggested. The tool was also used to inject faults into the architecture and evaluate the synchronization algorithm, and improvements are suggested. Finally, ADAS was used as a front end research tool to aid in the design of reconfiguration algorithms in a distributed array architecture.

A quality assessment tool for markup-based clinical guidelines.

PubMed

Shalom, Erez; Shahar, Yuval; Taieb-Maimon, Meirav; Lunenfeld, Eitan

2008-11-06

We introduce a tool for quality assessment of procedural and declarative knowledge. We developed this tool for evaluating the specification of mark-up-based clinical GLs. Using this graphical tool, the expert physician and knowledge engineer collaborate to perform scoring, using pre-defined scoring scale, each of the knowledge roles of the mark-ups, comparing it to a gold standard. The tool enables scoring the mark-ups simultaneously at different sites by different users at different locations.

Comparison of in silico models for prediction of mutagenicity.

PubMed

Bakhtyari, Nazanin G; Raitano, Giuseppa; Benfenati, Emilio; Martin, Todd; Young, Douglas

2013-01-01

Using a dataset with more than 6000 compounds, the performance of eight quantitative structure activity relationships (QSAR) models was evaluated: ACD/Tox Suite, Absorption, Distribution, Metabolism, Elimination, and Toxicity of chemical substances (ADMET) predictor, Derek, Toxicity Estimation Software Tool (T.E.S.T.), TOxicity Prediction by Komputer Assisted Technology (TOPKAT), Toxtree, CEASAR, and SARpy (SAR in python). In general, the results showed a high level of performance. To have a realistic estimate of the predictive ability, the results for chemicals inside and outside the training set for each model were considered. The effect of applicability domain tools (when available) on the prediction accuracy was also evaluated. The predictive tools included QSAR models, knowledge-based systems, and a combination of both methods. Models based on statistical QSAR methods gave better results.

RdTools: An Open Source Python Library for PV Degradation Analysis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Deceglie, Michael G; Jordan, Dirk; Nag, Ambarish

RdTools is a set of Python tools for analysis of photovoltaic data. In particular, PV production data is evaluated over several years to obtain rates of performance degradation over time. Rdtools can handle both high frequency (hourly or better) or low frequency (daily, weekly, etc.) datasets. Best results are obtained with higher frequency data.

Performance evaluation of Bragg coherent diffraction imaging

DOE PAGES

Ozturk, Hande; Huang, X.; Yan, H.; ...

2017-10-03

In this study, we present a numerical framework for modeling three-dimensional (3D) diffraction data in Bragg coherent diffraction imaging (Bragg CDI) experiments and evaluating the quality of obtained 3D complex-valued real-space images recovered by reconstruction algorithms under controlled conditions. The approach is used to systematically explore the performance and the detection limit of this phase-retrieval-based microscopy tool. The numerical investigation suggests that the superb performance of Bragg CDI is achieved with an oversampling ratio above 30 and a detection dynamic range above 6 orders. The observed performance degradation subject to the data binning processes is also studied. Furthermore, this numericalmore » tool can be used to optimize experimental parameters and has the potential to significantly improve the throughput of Bragg CDI method.« less

An Electronic Competency-Based Evaluation Tool for Assessing Humanitarian Competencies in a Simulated Exercise.

PubMed

Evans, Andrea B; Hulme, Jennifer M; Nugus, Peter; Cranmer, Hilarie H; Coutu, Melanie; Johnson, Kirsten

2017-06-01

The evaluation tool was first derived from the formerly Consortium of British Humanitarian Agencies' (CBHA; United Kingdom), now "Start Network's," Core Humanitarian Competency Framework and formatted in an electronic data capture tool that allowed for offline evaluation. During a 3-day humanitarian simulation event, participants in teams of eight to 10 were evaluated individually at multiple injects by trained evaluators. Participants were assessed on five competencies and a global rating scale. Participants evaluated both themselves and their team members using the same tool at the end of the simulation exercise (SimEx). All participants (63) were evaluated. A total of 1,008 individual evaluations were completed. There were 90 (9.0%) missing evaluations. All 63 participants also evaluated themselves and each of their teammates using the same tool. Self-evaluation scores were significantly lower than peer-evaluations, which were significantly lower than evaluators' assessments. Participants with a medical degree, and those with humanitarian work experience of one month or more, scored significantly higher on all competencies assessed by evaluators compared to other participants. Participants with prior humanitarian experience scored higher on competencies regarding operating safely and working effectively as a team member. This study presents a novel electronic evaluation tool to assess individual performance in five of six globally recognized humanitarian competency domains in a 3-day humanitarian SimEx. The evaluation tool provides a standardized approach to the assessment of humanitarian competencies that cannot be evaluated through knowledge-based testing in a classroom setting. When combined with testing knowledge-based competencies, this presents an approach to a comprehensive competency-based assessment that provides an objective measurement of competency with respect to the competencies listed in the Framework. There is an opportunity to advance the use of this tool in future humanitarian training exercises and potentially in real time, in the field. This could impact the efficiency and effectiveness of humanitarian operations. Evans AB , Hulme JM , Nugus P , Cranmer HH , Coutu M , Johnson K . An electronic competency-based evaluation tool for assessing humanitarian competencies in a simulated exercise. Prehosp Disaster Med. 2017;32(3):253-260.

Report Central: Quality Reporting Tool in an Electronic Health Record

PubMed Central

Jung, Eunice; Li, Qi; Mangalampalli, Anil; Greim, Julie; Eskin, Michael S.; Housman, Dan; Isikoff, Jeremy; Abend, Aaron H.; Middleton, Blackford; Einbinder, Jonathan S.

2006-01-01

Quality reporting tools, integrated with ambulatory electronic health records, can help clinicians and administrators understand performance, manage populations, and improve quality. Report Central is a secure web report delivery tool built on Crystal Reports XI™ and ASP.NET technologies. Pilot evaluation of Report Central indicates that clinicians prefer a quality reporting tool that is integrated with our home-grown EHR to support clinical workflow. PMID:17238590

The current state of cancer family history collection tools in primary care: a systematic review.

PubMed

Qureshi, Nadeem; Carroll, June C; Wilson, Brenda; Santaguida, Pasqualina; Allanson, Judith; Brouwers, Melissa; Raina, Parminder

2009-07-01

Systematic collection of family history is a prerequisite for identifying genetic risk. This study reviewed tools applicable to the primary care assessment of family history of breast, colorectal, ovarian, and prostate cancer. MEDLINE, EMBASE, CINAHL, and Cochrane Central were searched for publications. All primary study designs were included. Characteristics of the studies, the family history collection tools, and the setting were evaluated. Of 40 eligible studies, 18 relevant family history tools were identified, with 11 developed for use in primary care. Most collected information on more than one cancer and on affected relatives used self-administered questionnaires and paper-based formats. Eleven tools had been evaluated relative to current practice, demonstrating 46-78% improvement in data recording over family history recording in patient charts and 75-100% agreement with structured genetic interviews. Few tools have been developed specifically for primary care settings. The few that have been evaluated performed well. The very limited evidence, which depends in part on extrapolation from studies in settings other than primary care, suggests that systematic tools may add significant family health information compared with current primary care practice. The effect of their use on health outcomes has not been evaluated.

Prospective Comparison of Live Evaluation and Video Review in the Evaluation of Operator Performance in a Pediatric Emergency Airway Simulation

PubMed Central

House, Joseph B.; Dooley-Hash, Suzanne; Kowalenko, Terry; Sikavitsas, Athina; Seeyave, Desiree M.; Younger, John G.; Hamstra, Stanley J.; Nypaver, Michele M.

2012-01-01

Introduction Real-time assessment of operator performance during procedural simulation is a common practice that requires undivided attention by 1 or more reviewers, potentially over many repetitions of the same case. Objective To determine whether reviewers display better interrater agreement of procedural competency when observing recorded, rather than live, performance; and to develop an assessment tool for pediatric rapid sequence intubation (pRSI). Methods A framework of a previously established Objective Structured Assessment of Technical Skills (OSATS) tool was modified for pRSI. Emergency medicine residents (postgraduate year 1–4) were prospectively enrolled in a pRSI simulation scenario and evaluated by 2 live raters using the modified tool. Sessions were videotaped and reviewed by the same raters at least 4 months later. Raters were blinded to their initial rating. Interrater agreement was determined by using the Krippendorff generalized concordance method. Results Overall interrater agreement for live review was 0.75 (95% confidence interval [CI], 0.72–0.78) and for video was 0.79 (95% CI, 0.73–0.82). Live review was significantly superior to video review in only 1 of the OSATS domains (Preparation) and was equivalent in the other domains. Intrarater agreement between the live and video evaluation was very good, greater than 0.75 for all raters, with a mean of 0.81 (95% CI, 0.76–0.85). Conclusion The modified OSATS assessment tool demonstrated some evidence of validity in discriminating among levels of resident experience and high interreviewer reliability. With this tool, intrareviewer reliability was high between live and 4-months' delayed video review of the simulated procedure, which supports feasibility of delayed video review in resident assessment. PMID:23997874

Software Tools to Support the Assessment of System Health

NASA Technical Reports Server (NTRS)

Melcher, Kevin J.

2013-01-01

This presentation provides an overview of three software tools that were developed by the NASA Glenn Research Center to support the assessment of system health: the Propulsion Diagnostic Method Evaluation Strategy (ProDIMES), the Systematic Sensor Selection Strategy (S4), and the Extended Testability Analysis (ETA) tool. Originally developed to support specific NASA projects in aeronautics and space, these software tools are currently available to U.S. citizens through the NASA Glenn Software Catalog. The ProDiMES software tool was developed to support a uniform comparison of propulsion gas path diagnostic methods. Methods published in the open literature are typically applied to dissimilar platforms with different levels of complexity. They often address different diagnostic problems and use inconsistent metrics for evaluating performance. As a result, it is difficult to perform a one ]to ]one comparison of the various diagnostic methods. ProDIMES solves this problem by serving as a theme problem to aid in propulsion gas path diagnostic technology development and evaluation. The overall goal is to provide a tool that will serve as an industry standard, and will truly facilitate the development and evaluation of significant Engine Health Management (EHM) capabilities. ProDiMES has been developed under a collaborative project of The Technical Cooperation Program (TTCP) based on feedback provided by individuals within the aircraft engine health management community. The S4 software tool provides a framework that supports the optimal selection of sensors for health management assessments. S4 is structured to accommodate user ]defined applications, diagnostic systems, search techniques, and system requirements/constraints. One or more sensor suites that maximize this performance while meeting other user ]defined system requirements that are presumed to exist. S4 provides a systematic approach for evaluating combinations of sensors to determine the set or sets of sensors that optimally meet the performance goals and the constraints. It identifies optimal sensor suite solutions by utilizing a merit (i.e., cost) function with one of several available optimization approaches. As part of its analysis, S4 can expose fault conditions that are difficult to diagnose due to an incomplete diagnostic philosophy and/or a lack of sensors. S4 was originally developed and applied to liquid rocket engines. It was subsequently used to study the optimized selection of sensors for a simulation ]based aircraft engine diagnostic system. The ETA Tool is a software ]based analysis tool that augments the testability analysis and reporting capabilities of a commercial ]off ]the ]shelf (COTS) package. An initial diagnostic assessment is performed by the COTS software using a user ]developed, qualitative, directed ]graph model of the system being analyzed. The ETA Tool accesses system design information captured within the model and the associated testability analysis output to create a series of six reports for various system engineering needs. These reports are highlighted in the presentation. The ETA Tool was developed by NASA to support the verification of fault management requirements early in the Launch Vehicle process. Due to their early development during the design process, the TEAMS ]based diagnostic model and the ETA Tool were able to positively influence the system design by highlighting gaps in failure detection, fault isolation, and failure recovery.

Computer systems performance measurement techniques.

DOT National Transportation Integrated Search

1971-06-01

Computer system performance measurement techniques, tools, and approaches are presented as a foundation for future recommendations regarding the instrumentation of the ARTS ATC data processing subsystem for purposes of measurement and evaluation.

A review and evaluation of numerical tools for fractional calculus and fractional order controls

NASA Astrophysics Data System (ADS)

Li, Zhuo; Liu, Lu; Dehghan, Sina; Chen, YangQuan; Xue, Dingyü

2017-06-01

In recent years, as fractional calculus becomes more and more broadly used in research across different academic disciplines, there are increasing demands for the numerical tools for the computation of fractional integration/differentiation, and the simulation of fractional order systems. Time to time, being asked about which tool is suitable for a specific application, the authors decide to carry out this survey to present recapitulative information of the available tools in the literature, in hope of benefiting researchers with different academic backgrounds. With this motivation, the present article collects the scattered tools into a dashboard view, briefly introduces their usage and algorithms, evaluates the accuracy, compares the performance, and provides informative comments for selection.

Modified Advanced Crew Escape Suit Intravehicular Activity Suit for Extravehicular Activity Mobility Evaluations

NASA Technical Reports Server (NTRS)

Watson, Richard D.

2014-01-01

The use of an intravehicular activity (IVA) suit for a spacewalk or extravehicular activity (EVA) was evaluated for mobility and usability in the Neutral Buoyancy Laboratory (NBL) environment at the Sonny Carter Training Facility near NASA Johnson Space Center in Houston, Texas. The Space Shuttle Advanced Crew Escape Suit was modified to integrate with the Orion spacecraft. The first several missions of the Orion Multi-Purpose Crew Vehicle will not have mass available to carry an EVA-specific suit; therefore, any EVA required will have to be performed by the Modified Advanced Crew Escape Suit (MACES). Since the MACES was not designed with EVA in mind, it was unknown what mobility the suit would be able to provide for an EVA or whether a person could perform useful tasks for an extended time inside the pressurized suit. The suit was evaluated in multiple NBL runs by a variety of subjects, including crewmembers with significant EVA experience. Various functional mobility tasks performed included: translation, body positioning, tool carrying, body stabilization, equipment handling, and tool usage. Hardware configurations included with and without Thermal Micrometeoroid Garment, suit with IVA gloves and suit with EVA gloves. Most tasks were completed on International Space Station mock-ups with existing EVA tools. Some limited tasks were completed with prototype tools on a simulated rocky surface. Major findings include: demonstrating the ability to weigh-out the suit, understanding the need to have subjects perform multiple runs prior to getting feedback, determining critical sizing factors, and need for adjusting suit work envelope. Early testing demonstrated the feasibility of EVA's limited duration and limited scope. Further testing is required with more flight-like tasking and constraints to validate these early results. If the suit is used for EVA, it will require mission-specific modifications for umbilical management or Primary Life Support System integration, safety tether attachment, and tool interfaces. These evaluations are continuing through calendar year 2014.

Effects of structured written feedback by cards on medical students' performance at Mini Clinical Evaluation Exercise (Mini-CEX) in an outpatient clinic.

PubMed

Haghani, Fariba; Hatef Khorami, Mohammad; Fakhari, Mohammad

2016-07-01

Feedback cards are recommended as a feasible tool for structured written feedback delivery in clinical education while effectiveness of this tool on the medical students' performance is still questionable.  The purpose of this study was to compare the effects of structured written feedback by cards as well as verbal feedback versus verbal feedback alone on the clinical performance of medical students at the Mini Clinical Evaluation Exercise (Mini-CEX) test in an outpatient clinic. This is a quasi-experimental study with pre- and post-test comprising four groups in two terms of medical students' externship. The students' performance was assessed through the Mini-Clinical Evaluation Exercise (Mini-CEX) as a clinical performance evaluation tool. Structured written feedbacks were given to two experimental groups by designed feedback cards as well as verbal feedback, while in the two control groups feedback was delivered verbally as a routine approach in clinical education. By consecutive sampling method, 62 externship students were enrolled in this study and seven students were excluded from the final analysis due to their absence for three days. According to the ANOVA analysis and Post Hoc Tukey test,  no statistically significant difference was observed among the four groups at the pre-test, whereas a statistically significant difference was observed between the experimental and control groups at the post-test  (F = 4.023, p =0.012). The effect size of the structured written feedbacks on clinical performance was 0.19. Structured written feedback by cards could improve the performance of medical students in a statistical sense. Further studies must be conducted in other clinical courses with longer durations.

Abrasive Wear Resistance of Tool Steels Evaluated by the Pin-on-Disc Testing

NASA Astrophysics Data System (ADS)

Bressan, José Divo; Schopf, Roberto Alexandre

2011-05-01

Present work examines tool steels abrasion wear resistance and the abrasion mechanisms which are one main contributor to failure of tooling in metal forming industry. Tooling used in cutting and metal forming processes without lubrication fails due to this type of wear. In the workshop and engineering practice, it is common to relate wear resistance as function of material hardness only. However, there are others parameters which influences wear such as: fracture toughness, type of crystalline structure and the occurrence of hard precipitate in the metallic matrix and also its nature. In the present investigation, the wear mechanisms acting in tool steels were analyzed and, by normalized tests, wear resistance performance of nine different types of tool steels were evaluated by pin-on-disc testing. Conventional tool steels commonly used in tooling such as AISI H13 and AISI A2 were compared in relation to tool steels fabricated by sintering process such as Crucible CPM 3V, CPM 9V and M4 steels. Friction and wear testing were carried out in a pin-on-disc automated equipment which pin was tool steel and the counter-face was a abrasive disc of silicon carbide. Normal load of 5 N, sliding velocity of 0.45 m/s, total sliding distance of 3000 m and room temperature were employed. The wear rate was calculated by the Archard's equation and from the plotted graphs of pin cumulated volume loss versus sliding distance. Specimens were appropriately heat treated by quenching and three tempering cycles. Percentage of alloying elements, metallographic analyses of microstructure and Vickers microhardness of specimens were performed, analyzed and correlated with wear rate. The work is concluded by the presentation of a rank of tool steel wear rate, comparing the different tool steel abrasion wear resistance: the best tool steel wear resistance evaluated was the Crucible CPM 9V steel.

PredictSNP: Robust and Accurate Consensus Classifier for Prediction of Disease-Related Mutations

PubMed Central

Bendl, Jaroslav; Stourac, Jan; Salanda, Ondrej; Pavelka, Antonin; Wieben, Eric D.; Zendulka, Jaroslav; Brezovsky, Jan; Damborsky, Jiri

2014-01-01

Single nucleotide variants represent a prevalent form of genetic variation. Mutations in the coding regions are frequently associated with the development of various genetic diseases. Computational tools for the prediction of the effects of mutations on protein function are very important for analysis of single nucleotide variants and their prioritization for experimental characterization. Many computational tools are already widely employed for this purpose. Unfortunately, their comparison and further improvement is hindered by large overlaps between the training datasets and benchmark datasets, which lead to biased and overly optimistic reported performances. In this study, we have constructed three independent datasets by removing all duplicities, inconsistencies and mutations previously used in the training of evaluated tools. The benchmark dataset containing over 43,000 mutations was employed for the unbiased evaluation of eight established prediction tools: MAPP, nsSNPAnalyzer, PANTHER, PhD-SNP, PolyPhen-1, PolyPhen-2, SIFT and SNAP. The six best performing tools were combined into a consensus classifier PredictSNP, resulting into significantly improved prediction performance, and at the same time returned results for all mutations, confirming that consensus prediction represents an accurate and robust alternative to the predictions delivered by individual tools. A user-friendly web interface enables easy access to all eight prediction tools, the consensus classifier PredictSNP and annotations from the Protein Mutant Database and the UniProt database. The web server and the datasets are freely available to the academic community at http://loschmidt.chemi.muni.cz/predictsnp. PMID:24453961

System analysis tools for an ELT at ESO

NASA Astrophysics Data System (ADS)

Mueller, Michael; Koch, Franz

2006-06-01

Engineering of complex, large scale systems like the ELT designs currently investigated and developed in Europe and Northern America require powerful and sophisticated tools within specific technical disciplines such as mechanics, optics and control engineering. However, even analyzing a certain component of the telescope like the telescope structure necessitates a system approach to evaluate the structural effects onto the optical performance. This paper shows several software tools developed by the European Southern Observatory (ESO) which focus onto the system approach in the analyses: Using modal results of a finite element analysis the SMI-toolbox allows an easy generation of structural models with different sizes and levels of accuracy for the control design and closed-loop simulations. The optical modeling code BeamWarrior was developed by ESO and Astrium GmbH, Germany) especially for integrated modeling and interfering with a structural model. Within BeamWarrior displacements and deformations can be applied in an arbitrary coordinate system, and hence also in the global coordinates of the FE model avoiding error prone transformations. In addition to this, a sparse state space model object was developed for Matlab to gain in computational efficiency and reduced memory requirements due to the sparsity pattern of both the structural models and the control architecture. As one result these tools allow building an integrated model in order to reliably simulate interactions, cross-coupling effects, system responses, and to evaluate global performance. In order to evaluate disturbance effects on the optical performance in openloop more efficiently, an optical evaluation toolbox was built in the FE software ANSYS which performs Zernike decomposition and best-fit computation of the deformations directly in the FE analysis.

METAPHOR: Programmer's guide, Version 1

NASA Technical Reports Server (NTRS)

Furchtgott, D. G.

1979-01-01

The internal structure of the Michigan Evaluation Aid for Perphormability (METAPHOR), an interactive software package to facilitate performability modeling and evaluation is described. Revised supplemented guides are prepared in order to maintain an up-to-date documentation of the system. Programmed tools to facilitate each step of performability model construction and model solution are given.

Mobile Formative Assessment Tool Based on Data Mining Techniques for Supporting Web-Based Learning

ERIC Educational Resources Information Center

Chen, Chih-Ming; Chen, Ming-Chuan

2009-01-01

Current trends clearly indicate that online learning has become an important learning mode. However, no effective assessment mechanism for learning performance yet exists for e-learning systems. Learning performance assessment aims to evaluate what learners learned during the learning process. Traditional summative evaluation only considers final…

Improving evaluation at two medical schools.

PubMed

Schiekirka-Schwake, Sarah; Dreiling, Katharina; Pyka, Katharina; Anders, Sven; von Steinbüchel, Nicole; Raupach, Tobias

2017-08-03

Student evaluations of teaching can provide useful feedback for teachers and programme coordinators alike. We have designed a novel evaluation tool assessing teacher performance and student learning outcome. This tool was implemented at two German medical schools. In this article, we report student and teacher perceptions of the novel tool, and the implementation process. Focus group discussions as well as one-to-one interviews involving 22 teachers and 31 undergraduate medical students were conducted. Following adjustments to the feedback reports (e.g. the colour coding of results) at one medical school, 42 teachers were asked about their perceptions of the revised report and the personal benefit of the evaluation tool. Teachers appreciated the individual feedback provided by the evaluation tool and stated that they wanted to improve their teaching, based on the results; however, they missed most of the preparative communication. Students were unsure about the additional benefit of the instrument compared with traditional evaluation tools. A majority was unwilling to complete evaluation forms in their spare time, and some felt that the new questionnaire was too long and that the evaluations occurred too often. They were particularly interested in feedback on how their comments have helped to further improve teaching. Student evaluations of teaching can provide useful feedback CONCLUSION: Despite evidence of the utility of the tool for individual teachers, implementation of changes to the process of evaluation appears to have been suboptimal, mainly owing to a perceived lack of communication. In order to motivate students to provide evaluation data, feedback loops including aims and consequences should be established. © 2017 John Wiley & Sons Ltd and The Association for the Study of Medical Education.

Randomized Trial of Smartphone-Based Evaluation for an Obstetrics and Gynecology Clerkship.

PubMed

Sobhani, Nasim C; Fay, Emily E; Schiff, Melissa A; Stephenson-Famy, Alyssa; Debiec, Katherine E

2017-12-19

We hypothesized that compared to paper evaluations, a smartphone-based quick response (QR) evaluation tool would improve timeliness of feedback, enhance efficacy of giving and receiving feedback, and be as easy to use. We performed a randomized controlled trial of student and instructor experience with two evaluation tools in the OB/GYN clerkship at University of Washington School of Medicine (UWSOM). Sites were randomized to the QR or paper tool; students at QR sites received individualized QR codes at the beginning of the clerkship. Instructors and students completed postintervention surveys regarding the evaluation tool and associated feedback. We compared responses between groups using chi-squared tests. Participating clerkship sites included primary, tertiary, private practice and institutional settings affiliated with the University of Washington in the Washington, Wyoming, Alaska, Montana and Idaho region. Of the 29 OB/GYN UWSOM clerkship sites, 18 agreed to participate and were randomized. Of 29 eligible instructors, 25 (86%) completed the survey, with n = 18 using QR and n = 7 using paper. Of 161 eligible students, 102 (63%) completed the survey, with n = 54 using QR and n = 48 using paper. Compared to those using paper evaluations, instructors using QR evaluations were significantly more likely to agree that the evaluation tool was easy to understand (100% QR vs 43% paper, p = 0.002), the tool was effective in providing feedback (78% QR vs 29% paper, p = 0.002), and they felt comfortable approaching students with the tool (89% QR vs 43% paper, p = 0.002). Compared to those using paper evaluations, students using QR evaluations were less likely to agree the tool was effective in eliciting feedback (QR 43% vs paper 55%, p = 0.042). Instructors found QR evaluations superior to paper evaluations for providing feedback to medical students, whereas students found QR evaluations less effective for feedback. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Development of Reliable and Validated Tools to Evaluate Technical Resuscitation Skills in a Pediatric Simulation Setting: Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics.

PubMed

Faudeux, Camille; Tran, Antoine; Dupont, Audrey; Desmontils, Jonathan; Montaudié, Isabelle; Bréaud, Jean; Braun, Marc; Fournier, Jean-Paul; Bérard, Etienne; Berlengi, Noémie; Schweitzer, Cyril; Haas, Hervé; Caci, Hervé; Gatin, Amélie; Giovannini-Chami, Lisa

2017-09-01

To develop a reliable and validated tool to evaluate technical resuscitation skills in a pediatric simulation setting. Four Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics (RESCAPE) evaluation tools were created, following international guidelines: intraosseous needle insertion, bag mask ventilation, endotracheal intubation, and cardiac massage. We applied a modified Delphi methodology evaluation to binary rating items. Reliability was assessed comparing the ratings of 2 observers (1 in real time and 1 after a video-recorded review). The tools were assessed for content, construct, and criterion validity, and for sensitivity to change. Inter-rater reliability, evaluated with Cohen kappa coefficients, was perfect or near-perfect (>0.8) for 92.5% of items and each Cronbach alpha coefficient was ≥0.91. Principal component analyses showed that all 4 tools were unidimensional. Significant increases in median scores with increasing levels of medical expertise were demonstrated for RESCAPE-intraosseous needle insertion (P = .0002), RESCAPE-bag mask ventilation (P = .0002), RESCAPE-endotracheal intubation (P = .0001), and RESCAPE-cardiac massage (P = .0037). Significantly increased median scores over time were also demonstrated during a simulation-based educational program. RESCAPE tools are reliable and validated tools for the evaluation of technical resuscitation skills in pediatric settings during simulation-based educational programs. They might also be used for medical practice performance evaluations. Copyright © 2017 Elsevier Inc. All rights reserved.

Prototype road weather performance management (RW-PM) tool and Minnesota Department of Transportation (MnDOT) field evaluation.

DOT National Transportation Integrated Search

2017-01-01

FHWAs Road Weather Management Program developed a Prototype Road Weather Management (RW-PM) Tool to help DOTs maximize the effectiveness of their maintenance resources and efficiently adjust deployments dynamically, as road conditions and traffic ...

Development of Low Global Warming Potential Refrigerant Solutions for Commercial Refrigeration Systems using a Life Cycle Climate Performance Design Tool

DOE Office of Scientific and Technical Information (OSTI.GOV)

Abdelaziz, Omar; Fricke, Brian A; Vineyard, Edward Allan

Commercial refrigeration systems are known to be prone to high leak rates and to consume large amounts of electricity. As such, direct emissions related to refrigerant leakage and indirect emissions resulting from primary energy consumption contribute greatly to their Life Cycle Climate Performance (LCCP). In this paper, an LCCP design tool is used to evaluate the performance of a typical commercial refrigeration system with alternative refrigerants and minor system modifications to provide lower Global Warming Potential (GWP) refrigerant solutions with improved LCCP compared to baseline systems. The LCCP design tool accounts for system performance, ambient temperature, and system load; systemmore » performance is evaluated using a validated vapor compression system simulation tool while ambient temperature and system load are devised from a widely used building energy modeling tool (EnergyPlus). The LCCP design tool also accounts for the change in hourly electricity emission rate to yield an accurate prediction of indirect emissions. The analysis shows that conventional commercial refrigeration system life cycle emissions are largely due to direct emissions associated with refrigerant leaks and that system efficiency plays a smaller role in the LCCP. However, as a transition occurs to low GWP refrigerants, the indirect emissions become more relevant. Low GWP refrigerants may not be suitable for drop-in replacements in conventional commercial refrigeration systems; however some mixtures may be introduced as transitional drop-in replacements. These transitional refrigerants have a significantly lower GWP than baseline refrigerants and as such, improved LCCP. The paper concludes with a brief discussion on the tradeoffs between refrigerant GWP, efficiency and capacity.« less

Cockpit System Situational Awareness Modeling Tool

NASA Technical Reports Server (NTRS)

Keller, John; Lebiere, Christian; Shay, Rick; Latorella, Kara

2004-01-01

This project explored the possibility of predicting pilot situational awareness (SA) using human performance modeling techniques for the purpose of evaluating developing cockpit systems. The Improved Performance Research Integration Tool (IMPRINT) was combined with the Adaptive Control of Thought-Rational (ACT-R) cognitive modeling architecture to produce a tool that can model both the discrete tasks of pilots and the cognitive processes associated with SA. The techniques for using this tool to predict SA were demonstrated using the newly developed Aviation Weather Information (AWIN) system. By providing an SA prediction tool to cockpit system designers, cockpit concepts can be assessed early in the design process while providing a cost-effective complement to the traditional pilot-in-the-loop experiments and data collection techniques.

Evaluating conflation methods using uncertainty modeling

NASA Astrophysics Data System (ADS)

Doucette, Peter; Dolloff, John; Canavosio-Zuzelski, Roberto; Lenihan, Michael; Motsko, Dennis

2013-05-01

The classic problem of computer-assisted conflation involves the matching of individual features (e.g., point, polyline, or polygon vectors) as stored in a geographic information system (GIS), between two different sets (layers) of features. The classical goal of conflation is the transfer of feature metadata (attributes) from one layer to another. The age of free public and open source geospatial feature data has significantly increased the opportunity to conflate such data to create enhanced products. There are currently several spatial conflation tools in the marketplace with varying degrees of automation. An ability to evaluate conflation tool performance quantitatively is of operational value, although manual truthing of matched features is laborious and costly. In this paper, we present a novel methodology that uses spatial uncertainty modeling to simulate realistic feature layers to streamline evaluation of feature matching performance for conflation methods. Performance results are compiled for DCGIS street centerline features.

Evaluating Opportunities to Improve Material and Energy Impacts in Commodity Supply Chains.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hanes, Rebecca J.; Carpenter, Alberta

When evaluated at the process level, next-generation technologies may be more energy and emissions intensive than current technology. However, many advanced technologies have the potential to reduce material and energy consumption in upstream or downstream processing stages. In order to fully understand the benefits and consequences of technology deployment, next-generation technologies should be evaluated in context, as part of a supply chain. This work presents the Material Flows through Industry (MFI) scenario modeling tool. The MFI tool is a cradle-to-gate linear network model of the U.S. industrial sector that can model a wide range of manufacturing scenarios, including changes inmore » production technology, increases in industrial energy efficiency, and substitution between functionally equivalent materials. The MFI tool was developed to perform supply chain scale analyses in order to quantify the impacts and benefits of next-generation technologies and materials at that scale. For the analysis presented in this paper, the MFI tool is utilized to explore a case study comparing a steel supply chain to the supply chains of several functionally equivalent materials. Several of the alternatives to the baseline steel supply chain include next-generation production technologies and materials. Results of the case study show that aluminum production scenarios can out-perform the steel supply chain by using either an advanced smelting technology or an increased aluminum recycling rate. The next-generation material supply chains do not perform as well as either aluminum or steel, but may offer additional use phase reductions in energy and emissions that are outside the scope of the MFI tool. Future work will combine results from the MFI tool with a use phase analysis.« less

Integration of tools for the Design and Assessment of High-Performance, Highly Reliable Computing Systems (DAHPHRS), phase 1

NASA Technical Reports Server (NTRS)

Scheper, C.; Baker, R.; Frank, G.; Yalamanchili, S.; Gray, G.

1992-01-01

Systems for Space Defense Initiative (SDI) space applications typically require both high performance and very high reliability. These requirements present the systems engineer evaluating such systems with the extremely difficult problem of conducting performance and reliability trade-offs over large design spaces. A controlled development process supported by appropriate automated tools must be used to assure that the system will meet design objectives. This report describes an investigation of methods, tools, and techniques necessary to support performance and reliability modeling for SDI systems development. Models of the JPL Hypercubes, the Encore Multimax, and the C.S. Draper Lab Fault-Tolerant Parallel Processor (FTPP) parallel-computing architectures using candidate SDI weapons-to-target assignment algorithms as workloads were built and analyzed as a means of identifying the necessary system models, how the models interact, and what experiments and analyses should be performed. As a result of this effort, weaknesses in the existing methods and tools were revealed and capabilities that will be required for both individual tools and an integrated toolset were identified.

`G.A.T.E': Gap analysis for TTX evaluation

NASA Astrophysics Data System (ADS)

Cacciotti, Ilaria; Di Giovanni, Daniele; Pergolini, Alessandro; Malizia, Andrea; Carestia, Mariachiara; Palombi, Leonardo; Bellecci, Carlo; Gaudio, Pasquale

2016-06-01

A Table Top Exercise (TTX) gap analysis tool was developed with the aim to provide a complete, systematic and objective evaluation of TTXs organized in safety and security fields. A TTX consists in a discussion-based emergency management exercise, organized in a simulated emergency scenario, involving groups of players who are subjected to a set of solicitations (`injects'), in order to evaluate their emergency response abilities. This kind of exercise is devoted to identify strengths and shortfalls and to propose potential and promising changes in the approach to a particular situation. In order to manage the TTX derived data collection and analysis, a gap analysis tool would be very useful and functional at identifying the 'gap' between them and specific areas and actions for improvement, consisting the gap analysis in a comparison between actual performances and optimal/expected ones. In this context, a TTX gap analysis tool was designed, with the objective to provide an evaluation of Team players' competences and performances and TTX organization and structure. The influence of both the players' expertise and the reaction time (difference between expected time and time necessary to actually complete the injects) on the final evaluation of the inject responses was also taken into account.

Development of a Methodology to Conduct Usability Evaluation for Hand Tools that May Reduce the Amount of Small Parts that are Dropped During Installation while Processing Space Flight Hardware

NASA Technical Reports Server (NTRS)

Miller, Darcy

2000-01-01

Foreign object debris (FOD) is an important concern while processing space flight hardware. FOD can be defined as "The debris that is left in or around flight hardware, where it could cause damage to that flight hardware," (United Space Alliance, 2000). Just one small screw left unintentionally in the wrong place could delay a launch schedule while it is retrieved, increase the cost of processing, or cause a potentially fatal accident. At this time, there is not a single solution to help reduce the number of dropped parts such as screws, bolts, nuts, and washers during installation. Most of the effort is currently focused on training employees and on capturing the parts once they are dropped. Advances in ergonomics and hand tool design suggest that a solution may be possible, in the form of specialty hand tools, which secure the small parts while they are being handled. To assist in the development of these new advances, a test methodology was developed to conduct a usability evaluation of hand tools, while performing tasks with risk of creating FOD. The methodology also includes hardware in the form of a testing board and the small parts that can be installed onto the board during a test. The usability of new hand tools was determined based on efficiency and the number of dropped parts. To validate the methodology, participants were tested while performing a task that is representative of the type of work that may be done when processing space flight hardware. Test participants installed small parts using their hands and two commercially available tools. The participants were from three groups: (1) students, (2) engineers / managers and (3) technicians. The test was conducted to evaluate the differences in performance when using the three installation methods, as well as the difference in performance of the three participant groups.

Mixed Methods Design Study Investigating the Use of a Music Authentic Performance Assessment Tool by High School Band Directors to Measure Student Musical Growth

ERIC Educational Resources Information Center

Beason, Christine F.

2017-01-01

This research project was designed to determine if the Model Cornerstone Assessment for Performance, Proficient level, published by the National Association for Music Education would be an appropriate tool to use to demonstrate student growth as one element of teacher evaluations, specifically the T-TESS. This study focused on four main research…

The use of medical quality indices as a performance-enhancement tool for community clinics.

PubMed

Elhayany, A

2001-12-01

One of the most important issues for a country, its population and doctors is the effective use of its health system. The extensive variation in doctors' performance leads to a tremendous waste of resources. To combat this, and at the same time ensure that medical quality plays a role when making decisions on interventions, it is essential to equip doctors and clinic directors with information on the quality of the medical care they are providing. In order to assist clinic directors in maintaining medical quality, Clalit Health Services has developed comparative medical indices enabling doctors to compare their performance to that of their colleagues, as well as to the standard and their performance over time. The development of an index to evaluate the quality of medical treatment offered in clinics provides doctors and the health system with an essential tool to lessen the existing variation among doctors and to enhance and evaluate performance.

Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the 'Black Box' of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda.

PubMed

Renmans, Dimitri; Holvoet, Nathalie; Criel, Bart

2017-09-03

Increased attention on "complexity" in health systems evaluation has resulted in many different methodological responses. Theory-driven evaluations and systems thinking are two such responses that aim for better understanding of the mechanisms underlying given outcomes. Here, we studied the implementation of a performance-based financing intervention by the Belgian Technical Cooperation in Western Uganda to illustrate a methodological strategy of combining these two approaches. We utilized a systems dynamics tool called causal loop diagramming (CLD) to generate hypotheses feeding into a theory-driven evaluation. Semi-structured interviews were conducted with 30 health workers from two districts (Kasese and Kyenjojo) and with 16 key informants. After CLD, we identified three relevant hypotheses: "success to the successful", "growth and underinvestment", and "supervision conundrum". The first hypothesis leads to increasing improvements in performance, as better performance leads to more incentives, which in turn leads to better performance. The latter two hypotheses point to potential bottlenecks. Thus, the proposed methodological strategy was a useful tool for identifying hypotheses that can inform a theory-driven evaluation. The hypotheses are represented in a comprehensible way while highlighting the underlying assumptions, and are more easily falsifiable than hypotheses identified without using CLD.

Evaluation of in silico tools to predict the skin sensitization potential of chemicals.

PubMed

Verheyen, G R; Braeken, E; Van Deun, K; Van Miert, S

2017-01-01

Public domain and commercial in silico tools were compared for their performance in predicting the skin sensitization potential of chemicals. The packages were either statistical based (Vega, CASE Ultra) or rule based (OECD Toolbox, Toxtree, Derek Nexus). In practice, several of these in silico tools are used in gap filling and read-across, but here their use was limited to make predictions based on presence/absence of structural features associated to sensitization. The top 400 ranking substances of the ATSDR 2011 Priority List of Hazardous Substances were selected as a starting point. Experimental information was identified for 160 chemically diverse substances (82 positive and 78 negative). The prediction for skin sensitization potential was compared with the experimental data. Rule-based tools perform slightly better, with accuracies ranging from 0.6 (OECD Toolbox) to 0.78 (Derek Nexus), compared with statistical tools that had accuracies ranging from 0.48 (Vega) to 0.73 (CASE Ultra - LLNA weak model). Combining models increased the performance, with positive and negative predictive values up to 80% and 84%, respectively. However, the number of substances that were predicted positive or negative for skin sensitization in both models was low. Adding more substances to the dataset will increase the confidence in the conclusions reached. The insights obtained in this evaluation are incorporated in a web database www.asopus.weebly.com that provides a potential end user context for the scope and performance of different in silico tools with respect to a common dataset of curated skin sensitization data.

Development of the TeamOBS-PPH - targeting clinical performance in postpartum hemorrhage.

PubMed

Brogaard, Lise; Hvidman, Lone; Hinshaw, Kim; Kierkegaard, Ole; Manser, Tanja; Musaeus, Peter; Arafeh, Julie; Daniels, Kay I; Judy, Amy E; Uldbjerg, Niels

2018-06-01

This study aimed to develop a valid and reliable TeamOBS-PPH tool for assessing clinical performance in the management of postpartum hemorrhage (PPH). The tool was evaluated using video-recordings of teams managing PPH in both real-life and simulated settings. A Delphi panel consisting of 12 obstetricians from the UK, Norway, Sweden, Iceland, and Denmark achieved consensus on (i) the elements to include in the assessment tool, (ii) the weighting of each element, and (iii) the final tool. The validity and reliability were evaluated according to Cook and Beckman. (Level 1) Four raters scored four video-recordings of in situ simulations of PPH. (Level 2) Two raters scored 85 video-recordings of real-life teams managing patients with PPH ≥1000 mL in two Danish hospitals. (Level 3) Two raters scored 15 video-recordings of in situ simulations of PPH from a US hospital. The tool was designed with scores from 0 to 100. (Level 1) Teams of novices had a median score of 54 (95% CI 48-60), whereas experienced teams had a median score of 75 (95% CI 71-79; p < 0.001). (Level 2) The intra-rater [intra-class correlation (ICC) = 0.96] and inter-rater (ICC = 0.83) agreements for real-life PPH were strong. The tool was applicable in all cases: atony, retained placenta, and lacerations. (Level 3) The tool was easily adapted to in situ simulation settings in the USA (ICC = 0.86). The TeamOBS-PPH tool appears to be valid and reliable for assessing clinical performance in real-life and simulated settings. The tool will be shared as the free TeamOBS App. © 2018 Nordic Federation of Societies of Obstetrics and Gynecology.

Multi-phenomenology Observation Network Evaluation Tool'' (MONET)

NASA Astrophysics Data System (ADS)

Oltrogge, D.; North, P.; Vallado, D.

2014-09-01

Evaluating overall performance of an SSA "system-of-systems" observational network collecting against thousands of Resident Space Objects (RSO) is very difficult for typical tasking or scheduling-based analysis tools. This is further complicated by networks that have a wide variety of sensor types and phenomena, to include optical, radar and passive RF types, each having unique resource, ops tempo, competing customer and detectability constraints. We present details of the Multi-phenomenology Observation Network Evaluation Tool (MONET), which circumvents these difficulties by assessing the ideal performance of such a network via a digitized supply-vs-demand approach. Cells of each sensors supply time are distributed among RSO targets of interest to determine the average performance of the network against that set of RSO targets. Orbit Determination heuristics are invoked to represent observation quantity and geometry notionally required to obtain the desired orbit estimation quality. To feed this approach, we derive the detectability and collection rate performance of optical, radar and passive RF sensor physical and performance characteristics. We then prioritize the selected RSO targets according to object size, active/inactive status, orbit regime, and/or other considerations. Finally, the OD-derived tracking demands of each RSO of interest are levied against remaining sensor supply until either (a) all sensor time is exhausted; or (b) the list of RSO targets is exhausted. The outputs from MONET include overall network performance metrics delineated by sensor type, objects and orbits tracked, along with likely orbit accuracies which might result from the conglomerate network tracking.

On the question of economic efficiency and how to assess it

NASA Astrophysics Data System (ADS)

Petrosyan, M. O.; Kovalev, I. V.; Zelenkov, P. V.; Chuvashova, M. N.; Grishina, I. A.; Pershakova, K. K.

2016-04-01

Over the years, since the world moved to a system of market-based economic relations scientists and businessmen worldwide wonder how to ease and modernize the process of analyzing business performance. During this time, a variety of methods and tools have been developed to deal with this problem. After all, the productivity of the company is directly related to it’s competitive qualities, so the managers are interested in the quality of evaluation of their companies. The article deals with the traditional methods of evaluating the performance of enterprises and some of the tools used in order to improve its quality.

The GEDI Performance Tool

NASA Astrophysics Data System (ADS)

Hancock, S.; Armston, J.; Tang, H.; Patterson, P. L.; Healey, S. P.; Marselis, S.; Duncanson, L.; Hofton, M. A.; Kellner, J. R.; Luthcke, S. B.; Sun, X.; Blair, J. B.; Dubayah, R.

2017-12-01

NASA's Global Ecosystem Dynamics Investigation will mount a multi-track, full-waveform lidar on the International Space Station (ISS) that is optimised for the measurement of forest canopy height and structure. GEDI will use ten laser tracks, two 10 mJ "power beams" and eight 5 mJ "coverage beams" to produce global (51.5oS to 51.5oN) maps of above ground biomass (AGB), canopy height, vegetation structure and other biophysical parameters. The mission has a requirement to generate a 1 km AGB map with 80% of pixels with ≤ 20% standard error or 20 Mg·ha-1, whichever is greater. To assess performance and compare to mission requirements, an end-to-end simulator has been developed. The simulator brings together tools to propagate the effects of measurement and sampling error on GEDI data products. The simulator allows us to evaluate the impact of instrument performance, ISS orbits, processing algorithms and losses of data that may occur due to clouds, snow, leaf-off conditions, and areas with an insufficient signal-to-noise ratio (SNR). By evaluating the consequences of operational decisions on GEDI data products, this tool provides a quantitative framework for decision-making and mission planning. Here we demonstrate the performance tool by using it to evaluate the trade-off between measurement and sampling error on the 1 km AGB data product. Results demonstrate that the use of coverage beams during the day (lowest GEDI SNR case) over very dense forests (>95% canopy cover) will result in some measurement bias. Omitting these low SNR cases increased the sampling error. Through this an SNR threshold for a given expected canopy cover can be set. The other applications of the performance tool are also discussed, such as assessing the impact of decisions made in the AGB modelling and signal processing stages on the accuracy of final data products.

"Chair Stand Test" as Simple Tool for Sarcopenia Screening in Elderly Women.

PubMed

Pinheiro, P A; Carneiro, J A O; Coqueiro, R S; Pereira, R; Fernandes, M H

2016-01-01

To investigate the association between sarcopenia and "chair stand test" performance, and evaluate this test as a screening tool for sarcopenia in community-dwelling elderly women. Cross-sectional Survey. 173 female individuals, aged ≥ 60 years and living in the urban area of the municipality of Lafaiete Coutinho, Bahia's inland, Brazil. The association between sarcopenia (defined by muscle mass, strength and/or performance loss) and performance in the "chair stand test" was tested by binary logistic regression technique. The ROC curve parameters were used to evaluate the diagnostic power of the test in sarcopenia screening. The significance level was set at 5 %. The model showed that the time spent for the "chair stand test" was positively associated (OR = 1.08; 95% CI = 1.01 - 1.16, p = 0.024) to sarcopenia, indicating that, for each 1 second increment in the test performance, the sarcopenia's probability increased by 8% in elderly women. The cut-off point that showed the best balance between sensitivity and specificity was 13 seconds. The performance of "chair stand test" showed predictive ability for sarcopenia, being an effective and simple screening tool for sarcopenia in elderly women. This test could be used for screening sarcopenic elderly women, allowing early interventions.

Evaluation of an interactive science publishing tool: toward enabling three-dimensional analysis of medical images.

PubMed

Rinewalt, Daniel; Williams, Betsy W; Reeves, Anthony P; Shah, Palmi; Hong, Edward; Mulshine, James L

2015-03-01

Higher resolution medical imaging platforms are rapidly emerging, but there is a challenge in applying these tools in a clinically meaningful way. The purpose of the current study was to evaluate a novel three-dimensional (3D) software imaging environment, known as interactive science publishing (ISP), in appraising 3D computed tomography images and to compare this approach with traditional planar (2D) imaging in a series of lung cancer cases. Twenty-four physician volunteers at different levels of training across multiple specialties were recruited to evaluate eight lung cancer-related clinical vignettes. The volunteers were asked to compare the performance of traditional 2D versus the ISP 3D imaging in assessing different visualization environments for diagnostic and measurement processes and to further evaluate the ISP tool in terms of general satisfaction, usability, and probable applicability. Volunteers were satisfied with both imaging methods; however, the 3D environment had significantly higher ratings. Measurement performance was comparable using both traditional 2D and 3D image evaluation. Physicians not trained in 2D measurement approaches versus those with such training demonstrated better performance with ISP and preferred working in the ISP environment. Recent postgraduates with only modest self-administered training performed equally well on 3D and 2D cases. This suggests that the 3D environment has no reduction in accuracy over the conventional 2D approach, while providing the advantage of a digital environment for cross-disciplinary interaction for shared problem solving. Exploration of more effective, efficient, self-directed training could potentially result in further improvement in image evaluation proficiency and potentially decrease training costs. Copyright © 2015. Published by Elsevier Inc.

An Evaluation of the Performance Diagnostic Checklist-Human Services (PDC-HS) Across Domains.

PubMed

Wilder, David A; Lipschultz, Joshua; Gehrman, Chana

2018-06-01

The Performance Diagnostic Checklist-Human Services (PDC-HS) is an informant-based tool designed to assess the environmental variables that contribute to poor employee performance in human service settings. Although the PDC-HS has been shown to effectively identify variables contributing to problematic performance, interventions based on only two of the four PDC-HS domains have been evaluated to date. In addition, the extent to which PDC-HS-indicated interventions are more effective than nonindicated interventions for two domains remains unclear. In the current study, we administered the PDC-HS to supervisors to assess the variables contributing to infrequent teaching of verbal operants and use of a timer by therapists at a center-based autism treatment program. Each of the four PDC-HS domains was identified as contributing to poor performance for at least one therapist. We then evaluated PDC-HS-indicated interventions for each domain. In addition, to assess the predictive validity of the tool, we evaluated various nonindicated interventions prior to implementing a PDC-HS-indicated intervention for two of the four domains. Results suggest that the PDC-HS-indicated interventions were effective across all four domains and were more effective than the nonindicated interventions for the two domains for which they were evaluated. Results are discussed in terms of the utility of the PDC-HS to identify appropriate interventions to manage therapist performance in human service settings.

Evaluation in medical education: A topical review of target parameters, data collection tools and confounding factors.

PubMed

Schiekirka, Sarah; Feufel, Markus A; Herrmann-Lingen, Christoph; Raupach, Tobias

2015-01-01

Evaluation is an integral part of education in German medical schools. According to the quality standards set by the German Society for Evaluation, evaluation tools must provide an accurate and fair appraisal of teaching quality. Thus, data collection tools must be highly reliable and valid. This review summarises the current literature on evaluation of medical education with regard to the possible dimensions of teaching quality, the psychometric properties of survey instruments and potential confounding factors. We searched Pubmed, PsycINFO and PSYNDEX for literature on evaluation in medical education and included studies published up until June 30, 2011 as well as articles identified in the "grey literature". RESULTS are presented as a narrative review. We identified four dimensions of teaching quality: structure, process, teacher characteristics, and outcome. Student ratings are predominantly used to address the first three dimensions, and a number of reliable tools are available for this purpose. However, potential confounders of student ratings pose a threat to the validity of these instruments. Outcome is usually operationalised in terms of student performance on examinations, but methodological problems may limit the usability of these data for evaluation purposes. In addition, not all examinations at German medical schools meet current quality standards. The choice of tools for evaluating medical education should be guided by the dimension that is targeted by the evaluation. Likewise, evaluation results can only be interpreted within the context of the construct addressed by the data collection tool that was used as well as its specific confounding factors.

Assessing Army Values in New Soldiers: Development of a Behaviorally Anchored Peer Evaluation Form

DTIC Science & Technology

2015-02-03

evaluation has long been a valuable tool in school and workplace performance assessment, and has sparked military interest as a tool for leader...Values. Adhering to these values can affect Awards, promotions, selection to attend military schools , leadership positions, and desirable assignments (AR...by ratee and/or rater gender . Assessing the psychometric properties of AVBARS-PE The AVBARS-PE was administered to 50 BCT Soldiers (25 male and 25

Low-Level Analytical Methodology Updates to Support Decontaminant Performance Evaluations

DTIC Science & Technology

2011-06-01

from EPDM and tire rubber coupon materials that were spiked with a known amount of the chemical agent VX, treated with bleach decontaminant, and...to evaluate the performance of bleach decontaminant on EPDM and tire rubber coupons. Dose-confirmation or Tool samples were collected by delivering...components • An aging or damaged analytical column • Dirty detector • Other factors related to general instrument and/or sample analysis performance

Performance Appraisals: How to Make Them Work.

DTIC Science & Technology

1987-03-01

Choice....................8 Rank Order...................... Forced Distribution.................9 Management by Objectives.............10 Assessment... management today. Next, the many sources and causes of shortcomings with evaluation systems are explored in detail. Considerations in system design are...the response of others when I’ve evaluated them. A good performance system can be a tremendous management tool when trying to develop others

Granting Teachers the "Benefit of the Doubt" in Performance Evaluations

ERIC Educational Resources Information Center

Rogge, Nicky

2011-01-01

Purpose: This paper proposes a benefit of the doubt (BoD) approach to construct and analyse teacher effectiveness scores (i.e. SET scores). Design/methodology/approach: The BoD approach is related to data envelopment analysis (DEA), a linear programming tool for evaluating the relative efficiency performance of a set of similar units (e.g. firms,…

Development of an interactive social media tool for parents with concerns about vaccines.

PubMed

Shoup, Jo Ann; Wagner, Nicole M; Kraus, Courtney R; Narwaney, Komal J; Goddard, Kristin S; Glanz, Jason M

2015-06-01

Describe a process for designing, building, and evaluating a theory-driven social media intervention tool to help reduce parental concerns about vaccination. We developed an interactive web-based tool using quantitative and qualitative methods (e.g., survey, focus groups, individual interviews, and usability testing). Survey results suggested that social media may represent an effective intervention tool to help parents make informed decisions about vaccination for their children. Focus groups and interviews revealed four main themes for development of the tool: Parents wanted information describing both benefits and risks of vaccination, transparency of sources of information, moderation of the tool by an expert, and ethnic and racial diversity in the visual display of people. Usability testing showed that parents were satisfied with the usability of the tool but had difficulty with performing some of the informational searches. Based on focus groups, interviews, and usability evaluations, we made additional revisions to the tool's content, design, functionality, and overall look and feel. Engaging parents at all stages of development is critical when designing a tool to address concerns about childhood vaccines. Although this can be both resource- and time-intensive, the redesigned tool is more likely to be accepted and used by parents. Next steps involve a formal evaluation through a randomized trial. © 2014 Society for Public Health Education.

Use of Continuous Integration Tools for Application Performance Monitoring

DOE Office of Scientific and Technical Information (OSTI.GOV)

Vergara Larrea, Veronica G; Joubert, Wayne; Fuson, Christopher B

High performance computing systems are becom- ing increasingly complex, both in node architecture and in the multiple layers of software stack required to compile and run applications. As a consequence, the likelihood is increasing for application performance regressions to occur as a result of routine upgrades of system software components which interact in complex ways. The purpose of this study is to evaluate the effectiveness of continuous integration tools for application performance monitoring on HPC systems. In addition, this paper also describes a prototype system for application perfor- mance monitoring based on Jenkins, a Java-based continuous integration tool. The monitoringmore » system described leverages several features in Jenkins to track application performance results over time. Preliminary results and lessons learned from monitoring applications on Cray systems at the Oak Ridge Leadership Computing Facility are presented.« less

Assessing hospital disaster preparedness: a comparison of an on-site survey, directly observed drill performance, and video analysis of teamwork.

PubMed

Kaji, Amy H; Langford, Vinette; Lewis, Roger J

2008-09-01

There is currently no validated method for assessing hospital disaster preparedness. We determine the degree of correlation between the results of 3 methods for assessing hospital disaster preparedness: administration of an on-site survey, drill observation using a structured evaluation tool, and video analysis of team performance in the hospital incident command center. This was a prospective, observational study conducted during a regional disaster drill, comparing the results from an on-site survey, a structured disaster drill evaluation tool, and a video analysis of teamwork, performed at 6 911-receiving hospitals in Los Angeles County, CA. The on-site survey was conducted separately from the drill and assessed hospital disaster plan structure, vendor agreements, modes of communication, medical and surgical supplies, involvement of law enforcement, mutual aid agreements with other facilities, drills and training, surge capacity, decontamination capability, and pharmaceutical stockpiles. The drill evaluation tool, developed by Johns Hopkins University under contract from the Agency for Healthcare Research and Quality, was used to assess various aspects of drill performance, such as the availability of the hospital disaster plan, the geographic configuration of the incident command center, whether drill participants were identifiable, whether the noise level interfered with effective communication, and how often key information (eg, number of available staffed floor, intensive care, and isolation beds; number of arriving victims; expected triage level of victims; number of potential discharges) was received by the incident command center. Teamwork behaviors in the incident command center were quantitatively assessed, using the MedTeams analysis of the video recordings obtained during the disaster drill. Spearman rank correlations of the results between pair-wise groupings of the 3 assessment methods were calculated. The 3 evaluation methods demonstrated qualitatively different results with respect to each hospital's level of disaster preparedness. The Spearman rank correlation coefficient between the results of the on-site survey and the video analysis of teamwork was -0.34; between the results of the on-site survey and the structured drill evaluation tool, 0.15; and between the results of the video analysis and the drill evaluation tool, 0.82. The disparate results obtained from the 3 methods suggest that each measures distinct aspects of disaster preparedness, and perhaps no single method adequately characterizes overall hospital preparedness.

Policy options evaluation tool for managed lanes (POET-ML) users guide and methodology description : Federal Highway Administration HOV lane performance

DOT National Transportation Integrated Search

2008-12-01

Users guide for a sketch planning tool for exploring policy alternatives. It is intended for an audience of transportation professionals responsible for planning, designing, funding, operating, enforcing, monitoring, and managing HOV and HOT lanes...

29 CFR 29.6 - Program performance standards.

Code of Federal Regulations, 2012 CFR

2012-07-01

... apprenticeship programs. (1) The tools and factors to be used must include, but are not limited to: (i) Quality... rates. (2) Any additional tools and factors used by the Registration Agency in evaluating program... probationary period will not have an adverse impact on a sponsor's completion rate. ...

29 CFR 29.6 - Program performance standards.

Code of Federal Regulations, 2011 CFR

2011-07-01

... apprenticeship programs. (1) The tools and factors to be used must include, but are not limited to: (i) Quality... rates. (2) Any additional tools and factors used by the Registration Agency in evaluating program... probationary period will not have an adverse impact on a sponsor's completion rate. ...

29 CFR 29.6 - Program performance standards.

Code of Federal Regulations, 2013 CFR

2013-07-01

... apprenticeship programs. (1) The tools and factors to be used must include, but are not limited to: (i) Quality... rates. (2) Any additional tools and factors used by the Registration Agency in evaluating program... probationary period will not have an adverse impact on a sponsor's completion rate. ...

29 CFR 29.6 - Program performance standards.

Code of Federal Regulations, 2014 CFR

2014-07-01

... apprenticeship programs. (1) The tools and factors to be used must include, but are not limited to: (i) Quality... rates. (2) Any additional tools and factors used by the Registration Agency in evaluating program... probationary period will not have an adverse impact on a sponsor's completion rate. ...

Performance evaluation of infrared imaging system in field test

NASA Astrophysics Data System (ADS)

Wang, Chensheng; Guo, Xiaodong; Ren, Tingting; Zhang, Zhi-jie

2014-11-01

Infrared imaging system has been applied widely in both military and civilian fields. Since the infrared imager has various types and different parameters, for system manufacturers and customers, there is great demand for evaluating the performance of IR imaging systems with a standard tool or platform. Since the first generation IR imager was developed, the standard method to assess the performance has been the MRTD or related improved methods which are not perfect adaptable for current linear scanning imager or 2D staring imager based on FPA detector. For this problem, this paper describes an evaluation method based on the triangular orientation discrimination metric which is considered as the effective and emerging method to evaluate the synthesis performance of EO system. To realize the evaluation in field test, an experiment instrument is developed. And considering the importance of operational environment, the field test is carried in practical atmospheric environment. The test imagers include panoramic imaging system and staring imaging systems with different optics and detectors parameters (both cooled and uncooled). After showing the instrument and experiment setup, the experiment results are shown. The target range performance is analyzed and discussed. In data analysis part, the article gives the range prediction values obtained from TOD method, MRTD method and practical experiment, and shows the analysis and results discussion. The experimental results prove the effectiveness of this evaluation tool, and it can be taken as a platform to give the uniform performance prediction reference.

Freva - Freie Univ Evaluation System Framework for Scientific Infrastructures in Earth System Modeling

NASA Astrophysics Data System (ADS)

Kadow, Christopher; Illing, Sebastian; Kunst, Oliver; Schartner, Thomas; Kirchner, Ingo; Rust, Henning W.; Cubasch, Ulrich; Ulbrich, Uwe

2016-04-01

The Freie Univ Evaluation System Framework (Freva - freva.met.fu-berlin.de) is a software infrastructure for standardized data and tool solutions in Earth system science. Freva runs on high performance computers to handle customizable evaluation systems of research projects, institutes or universities. It combines different software technologies into one common hybrid infrastructure, including all features present in the shell and web environment. The database interface satisfies the international standards provided by the Earth System Grid Federation (ESGF). Freva indexes different data projects into one common search environment by storing the meta data information of the self-describing model, reanalysis and observational data sets in a database. This implemented meta data system with its advanced but easy-to-handle search tool supports users, developers and their plugins to retrieve the required information. A generic application programming interface (API) allows scientific developers to connect their analysis tools with the evaluation system independently of the programming language used. Users of the evaluation techniques benefit from the common interface of the evaluation system without any need to understand the different scripting languages. Facilitation of the provision and usage of tools and climate data automatically increases the number of scientists working with the data sets and identifying discrepancies. The integrated web-shell (shellinabox) adds a degree of freedom in the choice of the working environment and can be used as a gate to the research projects HPC. Plugins are able to integrate their e.g. post-processed results into the database of the user. This allows e.g. post-processing plugins to feed statistical analysis plugins, which fosters an active exchange between plugin developers of a research project. Additionally, the history and configuration sub-system stores every analysis performed with the evaluation system in a database. Configurations and results of the tools can be shared among scientists via shell or web system. Therefore, plugged-in tools benefit from transparency and reproducibility. Furthermore, if configurations match while starting an evaluation plugin, the system suggests to use results already produced by other users - saving CPU/h, I/O, disk space and time. The efficient interaction between different technologies improves the Earth system modeling science framed by Freva.

Interactive Tools for Measuring Visual Scanning Performance and Reaction Time

PubMed Central

Seeanner, Julia; Hennessy, Sarah; Manganelli, Joseph; Crisler, Matthew; Rosopa, Patrick; Jenkins, Casey; Anderson, Michael; Drouin, Nathalie; Belle, Leah; Truesdail, Constance; Tanner, Stephanie

2017-01-01

Occupational therapists are constantly searching for engaging, high-technology interactive tasks that provide immediate feedback to evaluate and train clients with visual scanning deficits. This study examined the relationship between two tools: the VISION COACH™ interactive light board and the Functional Object Detection© (FOD) Advanced driving simulator scenario. Fifty-four healthy drivers, ages 21–66 yr, were divided into three age groups. Participants performed braking response and visual target (E) detection tasks of the FOD Advanced driving scenario, followed by two sets of three trials using the VISION COACH Full Field 60 task. Results showed no significant effect of age on FOD Advanced performance but a significant effect of age on VISION COACH performance. Correlations showed that participants’ performance on both braking and E detection tasks were significantly positively correlated with performance on the VISION COACH (.37 < r < .40, p < .01). These tools provide new options for therapists. PMID:28218598

Tools for observational gait analysis in patients with stroke: a systematic review.

PubMed

Ferrarello, Francesco; Bianchi, Valeria Anna Maria; Baccini, Marco; Rubbieri, Gaia; Mossello, Enrico; Cavallini, Maria Chiara; Marchionni, Niccolò; Di Bari, Mauro

2013-12-01

Stroke severely affects walking ability, and assessment of gait kinematics is important in defining diagnosis, planning treatment, and evaluating interventions in stroke rehabilitation. Although observational gait analysis is the most common approach to evaluate gait kinematics, tools useful for this purpose have received little attention in the scientific literature and have not been thoroughly reviewed. The aims of this systematic review were to identify tools proposed to conduct observational gait analysis in adults with a stroke, to summarize evidence concerning their quality, and to assess their implementation in rehabilitation research and clinical practice. An extensive search was performed of original articles reporting on visual/observational tools developed to investigate gait kinematics in adults with a stroke. Two reviewers independently selected studies, extracted data, assessed quality of the included studies, and scored the metric properties and clinical utility of each tool. Rigor in reporting metric properties and dissemination of the tools also was evaluated. Five tools were identified, not all of which had been tested adequately for their metric properties. Evaluation of content validity was partially satisfactory. Reliability was poorly investigated in all but one tool. Concurrent validity and sensitivity to change were shown for 3 and 2 tools, respectively. Overall, adequate levels of quality were rarely reached. The dissemination of the tools was poor. Based on critical appraisal, the Gait Assessment and Intervention Tool shows a good level of quality, and its use in stroke rehabilitation is recommended. Rigorous studies are needed for the other tools in order to establish their usefulness.

Content Validation and Evaluation of an Endovascular Teamwork Assessment Tool.

PubMed

Hull, L; Bicknell, C; Patel, K; Vyas, R; Van Herzeele, I; Sevdalis, N; Rudarakanchana, N

2016-07-01

To modify, content validate, and evaluate a teamwork assessment tool for use in endovascular surgery. A multistage, multimethod study was conducted. Stage 1 included expert review and modification of the existing Observational Teamwork Assessment for Surgery (OTAS) tool. Stage 2 included identification of additional exemplar behaviours contributing to effective teamwork and enhanced patient safety in endovascular surgery (using real-time observation, focus groups, and semistructured interviews of multidisciplinary teams). Stage 3 included content validation of exemplar behaviours using expert consensus according to established psychometric recommendations and evaluation of structure, content, feasibility, and usability of the Endovascular Observational Teamwork Assessment Tool (Endo-OTAS) by an expert multidisciplinary panel. Stage 4 included final team expert review of exemplars. OTAS core team behaviours were maintained (communication, coordination, cooperation, leadership team monitoring). Of the 114 OTAS behavioural exemplars, 19 were modified, four removed, and 39 additional endovascular-specific behaviours identified. Content validation of these 153 exemplar behaviours showed that 113/153 (73.9%) reached the predetermined Item-Content Validity Index rating for teamwork and/or patient safety. After expert team review, 140/153 (91.5%) exemplars were deemed to warrant inclusion in the tool. More than 90% of the expert panel agreed that Endo-OTAS is an appropriate teamwork assessment tool with observable behaviours. Some concerns were noted about the time required to conduct observations and provide performance feedback. Endo-OTAS is a novel teamwork assessment tool, with evidence for content validity and relevance to endovascular teams. Endo-OTAS enables systematic objective assessment of the quality of team performance during endovascular procedures. Copyright © 2016. Published by Elsevier Ltd.

The Comprehensive, Powerful, Academic Database (CPAD): An Evaluative Study of a Predictive Tool Designed for Elementary School Personnel in Identifying At-Risk Students through Progress, Curriculum, and Performance Monitoring

ERIC Educational Resources Information Center

Chavez-Gibson, Sarah

2013-01-01

The purpose of this study is to exam in-depth, the Comprehensive, Powerful, Academic Database (CPAD), a data decision-making tool that determines and identifies students at-risk of dropping out of school, and how the CPAD assists administrators and teachers at an elementary campus to monitor progress, curriculum, and performance to improve student…

Clinical Parameters and Tools for Home-Based Assessment of Parkinson's Disease: Results from a Delphi study.

PubMed

Ferreira, Joaquim J; Santos, Ana T; Domingos, Josefa; Matthews, Helen; Isaacs, Tom; Duffen, Joy; Al-Jawad, Ahmed; Larsen, Frank; Artur Serrano, J; Weber, Peter; Thoms, Andrea; Sollinger, Stefan; Graessner, Holm; Maetzler, Walter

2015-01-01

Parkinson's disease (PD) is a neurodegenerative disorder with fluctuating symptoms. To aid the development of a system to evaluate people with PD (PwP) at home (SENSE-PARK system) there was a need to define parameters and tools to be applied in the assessment of 6 domains: gait, bradykinesia/hypokinesia, tremor, sleep, balance and cognition. To identify relevant parameters and assessment tools of the 6 domains, from the perspective of PwP, caregivers and movement disorders specialists. A 2-round Delphi study was conducted to select a core of parameters and assessment tools to be applied. This process included PwP, caregivers and movement disorders specialists. Two hundred and thirty-three PwP, caregivers and physicians completed the first round questionnaire, and 50 the second. Results allowed the identification of parameters and assessment tools to be added to the SENSE-PARK system. The most consensual parameters were: Falls and Near Falls; Capability to Perform Activities of Daily Living; Interference with Activities of Daily Living; Capability to Process Tasks; and Capability to Recall and Retrieve Information. The most cited assessment strategies included Walkers; the Evaluation of Performance Doing Fine Motor Movements; Capability to Eat; Assessment of Sleep Quality; Identification of Circumstances and Triggers for Loose of Balance and Memory Assessment. An agreed set of measuring parameters, tests, tools and devices was achieved to be part of a system to evaluate PwP at home. A pattern of different perspectives was identified for each stakeholder.

Development and Evaluation of a Novel Survey Tool Assessing Inpatient Consult Service Performance.

PubMed

Miloslavsky, Eli M; Chang, Yuchiao

2017-12-01

Subspecialty consultation in inpatient medicine is increasing, and enhancing performance of consultation services may have a broad-reaching impact. Multisource feedback is an important tool in assessing competence and improving performance. A mechanism for primary team resident feedback on performance of consult services has not been described. We developed and evaluated an instrument designed to assess internal medicine (IM) subspecialty inpatient consult service performance. We hypothesized that the instrument would be feasible to administer and provide important information to fellowship directors. The instrument was administered in 2015 and 2016 at a single academic center. All IM residents were invited to evaluate 10 IM subspecialty consult services on 4 items and an overall satisfaction rating. The instrument allowed for free-text feedback to fellows. Program directors completed another survey assessing the impact of the consult service evaluation. A total of 113 residents responded (47 in 2015 and 66 in 2016, for a combined response rate of 35%). Each of the 4 items measured (communication, professionalism, teaching, and pushback) correlated significantly with the overall satisfaction rating in univariate and multivariate analyses. There were no differences in ratings across postgraduate year or year of administration. There was considerable variation in ratings among the services evaluated. The 7 program directors who provided feedback found the survey useful and made programmatic changes following evaluation implementation. A primary team resident evaluation of inpatient medicine subspecialty consult services is feasible, provides valuable information, and is associated with changes in consult service structure and curricula.

Designing Real-time Decision Support for Trauma Resuscitations

PubMed Central

Yadav, Kabir; Chamberlain, James M.; Lewis, Vicki R.; Abts, Natalie; Chawla, Shawn; Hernandez, Angie; Johnson, Justin; Tuveson, Genevieve; Burd, Randall S.

2016-01-01

Background Use of electronic clinical decision support (eCDS) has been recommended to improve implementation of clinical decision rules. Many eCDS tools, however, are designed and implemented without taking into account the context in which clinical work is performed. Implementation of the pediatric traumatic brain injury (TBI) clinical decision rule at one Level I pediatric emergency department includes an electronic questionnaire triggered when ordering a head computed tomography using computerized physician order entry (CPOE). Providers use this CPOE tool in less than 20% of trauma resuscitation cases. A human factors engineering approach could identify the implementation barriers that are limiting the use of this tool. Objectives The objective was to design a pediatric TBI eCDS tool for trauma resuscitation using a human factors approach. The hypothesis was that clinical experts will rate a usability-enhanced eCDS tool better than the existing CPOE tool for user interface design and suitability for clinical use. Methods This mixed-methods study followed usability evaluation principles. Pediatric emergency physicians were surveyed to identify barriers to using the existing eCDS tool. Using standard trauma resuscitation protocols, a hierarchical task analysis of pediatric TBI evaluation was developed. Five clinical experts, all board-certified pediatric emergency medicine faculty members, then iteratively modified the hierarchical task analysis until reaching consensus. The software team developed a prototype eCDS display using the hierarchical task analysis. Three human factors engineers provided feedback on the prototype through a heuristic evaluation, and the software team refined the eCDS tool using a rapid prototyping process. The eCDS tool then underwent iterative usability evaluations by the five clinical experts using video review of 50 trauma resuscitation cases. A final eCDS tool was created based on their feedback, with content analysis of the evaluations performed to ensure all concerns were identified and addressed. Results Among 26 EPs (76% response rate), the main barriers to using the existing tool were that the information displayed is redundant and does not fit clinical workflow. After the prototype eCDS tool was developed based on the trauma resuscitation hierarchical task analysis, the human factors engineers rated it to be better than the CPOE tool for nine of 10 standard user interface design heuristics on a three-point scale. The eCDS tool was also rated better for clinical use on the same scale, in 84% of 50 expert–video pairs, and was rated equivalent in the remainder. Clinical experts also rated barriers to use of the eCDS tool as being low. Conclusions An eCDS tool for diagnostic imaging designed using human factors engineering methods has improved perceived usability among pediatric emergency physicians. PMID:26300010

Simulator evaluation of the final approach spacing tool

NASA Technical Reports Server (NTRS)

Davis, Thomas J.; Erzberger, Heinz; Green, Steven M.

1990-01-01

The design and simulator evaluation of an automation tool for assisting terminal radar approach controllers in sequencing and spacing traffic onto the final approach course is described. The automation tool, referred to as the Final Approach Spacing Tool (FAST), displays speed and heading advisories for arrivals as well as sequencing information on the controller's radar display. The main functional elements of FAST are a scheduler that schedules and sequences the traffic, a 4-D trajectory synthesizer that generates the advisories, and a graphical interface that displays the information to the controller. FAST was implemented on a high performance workstation. It can be operated as a stand-alone in the Terminal Radar Approach Control (TRACON) Facility or as an element of a system integrated with automation tools in the Air Route Traffic Control Center (ARTCC). FAST was evaluated by experienced TRACON controllers in a real-time air traffic control simulation. Simulation results show that FAST significantly reduced controller workload and demonstrated a potential for an increase in landing rate.

[Blended-learning in psychosomatics and psychotherapy - Increasing the satisfaction and knowledge of students with a web-based e-learning tool].

PubMed

Ferber, Julia; Schneider, Gudrun; Havlik, Linda; Heuft, Gereon; Friederichs, Hendrik; Schrewe, Franz-Bernhard; Schulz-Steinel, Andrea; Burgmer, Markus

2014-01-01

To improve the synergy of established methods of teaching, the Department of Psychosomatics and Psychotherapy, University Hospital Münster, developed a web-based elearning tool using video clips of standardized patients. The effect of this blended-learning approach was evaluated. A multiple-choice test was performed by a naive (without the e-learning tool) and an experimental (with the tool) cohort of medical students to test the groups' expertise in psychosomatics. In addition, participants' satisfaction with the new tool was evaluated (numeric rating scale of 0-10). The experimental cohort was more satisfied with the curriculum and more interested in psychosomatics. Furthermore, the experimental cohort scored significantly better in the multiple-choice test. The new tool proved to be an important addition to the classical curriculum as a blended-learning approach which improves students' satisfaction and knowledge in psychosomatics.

Development of a Content-Valid Standardized Orthopedic Assessment Tool (SOAT)

ERIC Educational Resources Information Center

Lafave, Mark; Katz, Larry; Butterwick, Dale

2008-01-01

Content validation of an instrument that measures student performance in OSCE-type practical examinations is a critical step in a tool's overall validity and reliability [Hopkins (1998), "Educational and Psychological Measurement and Evaluation" (8th ed.). Toronto: Allyn & Bacon]. The purpose of the paper is to outline the process…

Innovative Approaches to Assessment of Results of Higher School Students Training

ERIC Educational Resources Information Center

Vaganova, Olga I.; Medvedeva, Tatiana Yu.; Kirdyanova, Elena R.; Kazantseva, Galina A.; Karpukova, Albina A.

2016-01-01

The basis of assessment tools selection for performance of control and evaluation of training results subject to requirements of modular-competence approach has been disclosed. The experience in implementation of assessment tools during "General and professional pedagogy" course has been observed. The objective of the study is rationale…

Design and evaluation of an air traffic control Final Approach Spacing Tool

NASA Technical Reports Server (NTRS)

Davis, Thomas J.; Erzberger, Heinz; Green, Steven M.; Nedell, William

1991-01-01

This paper describes the design and simulator evaluation of an automation tool for assisting terminal radar approach controllers in sequencing and spacing traffic onto the final approach course. The automation tool, referred to as the Final Approach Spacing Tool (FAST), displays speed and heading advisories for arriving aircraft as well as sequencing information on the controller's radar display. The main functional elements of FAST are a scheduler that schedules and sequences the traffic, a four-dimensional trajectory synthesizer that generates the advisories, and a graphical interface that displays the information to the controller. FAST has been implemented on a high-performance workstation. It can be operated as a stand-alone in the terminal radar approach control facility or as an element of a system integrated with automation tools in the air route traffic control center. FAST was evaluated by experienced air traffic controllers in a real-time air traffic control simulation. simulation results summarized in the paper show that the automation tools significantly reduced controller work load and demonstrated a potential for an increase in landing rate.

Prediction Of Abrasive And Diffusive Tool Wear Mechanisms In Machining

NASA Astrophysics Data System (ADS)

Rizzuti, S.; Umbrello, D.

2011-01-01

Tool wear prediction is regarded as very important task in order to maximize tool performance, minimize cutting costs and improve the quality of workpiece in cutting. In this research work, an experimental campaign was carried out at the varying of cutting conditions with the aim to measure both crater and flank tool wear, during machining of an AISI 1045 with an uncoated carbide tool P40. Parallel a FEM-based analysis was developed in order to study the tool wear mechanisms, taking also into account the influence of the cutting conditions and the temperature reached on the tool surfaces. The results show that, when the temperature of the tool rake surface is lower than the activation temperature of the diffusive phenomenon, the wear rate can be estimated applying an abrasive model. In contrast, in the tool area where the temperature is higher than the diffusive activation temperature, the wear rate can be evaluated applying a diffusive model. Finally, for a temperature ranges within the above cited values an adopted abrasive-diffusive wear model furnished the possibility to correctly evaluate the tool wear phenomena.

An intelligent tool for activity data collection.

PubMed

Sarkar, A M Jehad

2011-01-01

Activity recognition systems using simple and ubiquitous sensors require a large variety of real-world sensor data for not only evaluating their performance but also training the systems for better functioning. However, a tremendous amount of effort is required to setup an environment for collecting such data. For example, expertise and resources are needed to design and install the sensors, controllers, network components, and middleware just to perform basic data collections. It is therefore desirable to have a data collection method that is inexpensive, flexible, user-friendly, and capable of providing large and diverse activity datasets. In this paper, we propose an intelligent activity data collection tool which has the ability to provide such datasets inexpensively without physically deploying the testbeds. It can be used as an inexpensive and alternative technique to collect human activity data. The tool provides a set of web interfaces to create a web-based activity data collection environment. It also provides a web-based experience sampling tool to take the user's activity input. The tool generates an activity log using its activity knowledge and the user-given inputs. The activity knowledge is mined from the web. We have performed two experiments to validate the tool's performance in producing reliable datasets.

The artificial membrane insert system as predictive tool for formulation performance evaluation.

PubMed

Berben, Philippe; Brouwers, Joachim; Augustijns, Patrick

2018-02-15

In view of the increasing interest of pharmaceutical companies for cell- and tissue-free models to implement permeation into formulation testing, this study explored the capability of an artificial membrane insert system (AMI-system) as predictive tool to evaluate the performance of absorption-enabling formulations. Firstly, to explore the usefulness of the AMI-system in supersaturation assessment, permeation was monitored after induction of different degrees of loviride supersaturation. Secondly, to explore the usefulness of the AMI-system in formulation evaluation, a two-stage dissolution test was performed prior to permeation assessment. Different case examples were selected based on the availability of in vivo (intraluminal and systemic) data: (i) a suspension of posaconazole (Noxafil ® ), (ii) a cyclodextrin-based formulation of itraconazole (Sporanox ® ), and (iii) a micronized (Lipanthyl ® ) and nanosized (Lipanthylnano ® ) formulation of fenofibrate. The obtained results demonstrate that the AMI-system is able to capture the impact of loviride supersaturation on permeation. Furthermore, the AMI-system correctly predicted the effects of (i) formulation pH on posaconazole absorption, (ii) dilution on cyclodextrin-based itraconazole absorption, and (iii) food intake on fenofibrate absorption. Based on the applied in vivo/in vitro approach, the AMI-system combined with simple dissolution testing appears to be a time- and cost-effective tool for the early-stage evaluation of absorption-enabling formulations. Copyright © 2017 Elsevier B.V. All rights reserved.

MACHETE: Environment for Space Networking Evaluation

NASA Technical Reports Server (NTRS)

Jennings, Esther H.; Segui, John S.; Woo, Simon

2010-01-01

Space Exploration missions requires the design and implementation of space networking that differs from terrestrial networks. In a space networking architecture, interplanetary communication protocols need to be designed, validated and evaluated carefully to support different mission requirements. As actual systems are expensive to build, it is essential to have a low cost method to validate and verify mission/system designs and operations. This can be accomplished through simulation. Simulation can aid design decisions where alternative solutions are being considered, support trade-studies and enable fast study of what-if scenarios. It can be used to identify risks, verify system performance against requirements, and as an initial test environment as one moves towards emulation and actual hardware implementation of the systems. We describe the development of Multi-mission Advanced Communications Hybrid Environment for Test and Evaluation (MACHETE) and its use cases in supporting architecture trade studies, protocol performance and its role in hybrid simulation/emulation. The MACHETE environment contains various tools and interfaces such that users may select the set of tools tailored for the specific simulation end goal. The use cases illustrate tool combinations for simulating space networking in different mission scenarios. This simulation environment is useful in supporting space networking design for planned and future missions as well as evaluating performance of existing networks where non-determinism exist in data traffic and/or link conditions.

A novel adjuvant to the resident selection process: the hartman value profile.

PubMed

Cone, Jeffrey D; Byrum, C Stephen; Payne, Wyatt G; Smith, David J

2012-01-01

The goal of resident selection is twofold: (1) select candidates who will be successful residents and eventually successful practitioners and (2) avoid selecting candidates who will be unsuccessful residents and/or eventually unsuccessful practitioners. Traditional tools used to select residents have well-known limitations. The Hartman Value Profile (HVP) is a proven adjuvant tool to predicting future performance in candidates for advanced positions in the corporate setting. No literature exists to indicate use of the HVP for resident selection. The HVP evaluates the structure and the dynamics of an individual value system. Given the potential impact, we implemented its use beginning in 2007 as an adjuvant tool to the traditional selection process. Experience gained from incorporating the HVP into the residency selection process suggests that it may add objectivity and refinement in predicting resident performance. Further evaluation is warranted with longer follow-up times.

A Novel Adjuvant to the Resident Selection Process: the Hartman Value Profile

PubMed Central

Cone, Jeffrey D.; Byrum, C. Stephen; Payne, Wyatt G.; Smith, David J.

2012-01-01

Objectives: The goal of resident selection is twofold: (1) select candidates who will be successful residents and eventually successful practitioners and (2) avoid selecting candidates who will be unsuccessful residents and/or eventually unsuccessful practitioners. Traditional tools used to select residents have well-known limitations. The Hartman Value Profile (HVP) is a proven adjuvant tool to predicting future performance in candidates for advanced positions in the corporate setting. Methods: No literature exists to indicate use of the HVP for resident selection. Results: The HVP evaluates the structure and the dynamics of an individual value system. Given the potential impact, we implemented its use beginning in 2007 as an adjuvant tool to the traditional selection process. Conclusions: Experience gained from incorporating the HVP into the residency selection process suggests that it may add objectivity and refinement in predicting resident performance. Further evaluation is warranted with longer follow-up times. PMID:22720114

Teacher Evaluation, Pay for Performance, and Learning around Instruction: Between Dissonant Incentives and Resonant Procedures

ERIC Educational Resources Information Center

Mintrop, Rick; Ordenes, Miguel; Coghlan, Erin; Pryor, Laura; Madero, Cristobal

2018-01-01

Purpose: The study examines why the logic of a performance management system, supported by the federal Teacher Incentive Fund, might be faulty. It does this by exploring the nuances of the interplay between teaching evaluations as formative and summative, the use of procedures, tools, and artifacts obligated by the local Teacher Incentive Fund…

Development and evaluation of a patient-centred measurement tool for surgeons' non-technical skills.

PubMed

Yule, J; Hill, K; Yule, S

2018-06-01

Non-technical skills are essential for safe and effective surgery. Several tools to assess surgeons' non-technical skills from the clinician's perspective have been developed. However, a reliable measurement tool using a patient-centred approach does not currently exist. The aim of this study was to translate the existing Non-Technical Skills for Surgeons (NOTSS) tool into a patient-centred evaluation tool. Data were gathered from four cohorts of patients using an iterative four-stage mixed-methods research design. Exploratory and confirmatory factor analyses were performed to establish the psychometric properties of the tool, focusing on validity, reliability, usability and parsimony. Some 534 patients were recruited to the study. A total of 24 patient-centred non-technical skill items were developed in stage 1, and reduced to nine items in stage 2 using exploratory factor analysis. In stage 3, confirmatory factor analysis demonstrated that these nine items each loaded on to one of three factors, with excellent internal consistency: decision-making, leadership, and communication and teamwork. In stage 4, validity testing established that the new tool was independent of physician empathy and predictive of surgical quality. Surgical leadership emerged as the most dominant skill that patients could recognize and evaluate. A novel nine-item assessment tool has been developed. The Patients' Evaluation of Non-Technical Skills (PENTS) tool allows valid and reliable measurement of surgeons' non-technical skills from the patient perspective. © 2018 BJS Society Ltd Published by John Wiley & Sons Ltd.

Computer assisted blast design and assessment tools

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cameron, A.R.; Kleine, T.H.; Forsyth, W.W.

1995-12-31

In general the software required by a blast designer includes tools that graphically present blast designs (surface and underground), can analyze a design or predict its result, and can assess blasting results. As computers develop and computer literacy continues to rise the development of and use of such tools will spread. An example of the tools that are becoming available includes: Automatic blast pattern generation and underground ring design; blast design evaluation in terms of explosive distribution and detonation simulation; fragmentation prediction; blast vibration prediction and minimization; blast monitoring for assessment of dynamic performance; vibration measurement, display and signal processing;more » evaluation of blast results in terms of fragmentation; and risk and reliability based blast assessment. The authors have identified a set of criteria that are essential in choosing appropriate software blasting tools.« less

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bunshah, R.F.; Shabaik, A.H.

The process of Activated Reactive Evaporation is used to synthesize superhard materials like carbides, oxides, nitrides and ultrafine grain cermets. The deposits are characterized by hardness, microstructure, microprobe analysis for chemistry and lattice parameter measurements. The synthesis and characterization of TiC-Ni cermets and Al/sub 2/O/sub 3/ are given. High speed steel tool coated with TiC, TiC-Ni and TaC are tested for machining performance at different speeds and feeds. The machining evaluation and the selection of coatings is based on the rate of deterioration of the coating tool temperature, and cutting forces. Tool life tests show coated high speed steel toolsmore » having 150 to 300% improvement in tool life compared to uncoated tools. Variability in the quality of the ground edge on high speed steel inserts produce a great scatter in the machining evaluation data.« less

Prospective performance evaluation of selected common virtual screening tools. Case study: Cyclooxygenase (COX) 1 and 2.

PubMed

Kaserer, Teresa; Temml, Veronika; Kutil, Zsofia; Vanek, Tomas; Landa, Premysl; Schuster, Daniela

2015-01-01

Computational methods can be applied in drug development for the identification of novel lead candidates, but also for the prediction of pharmacokinetic properties and potential adverse effects, thereby aiding to prioritize and identify the most promising compounds. In principle, several techniques are available for this purpose, however, which one is the most suitable for a specific research objective still requires further investigation. Within this study, the performance of several programs, representing common virtual screening methods, was compared in a prospective manner. First, we selected top-ranked virtual screening hits from the three methods pharmacophore modeling, shape-based modeling, and docking. For comparison, these hits were then additionally predicted by external pharmacophore- and 2D similarity-based bioactivity profiling tools. Subsequently, the biological activities of the selected hits were assessed in vitro, which allowed for evaluating and comparing the prospective performance of the applied tools. Although all methods performed well, considerable differences were observed concerning hit rates, true positive and true negative hits, and hitlist composition. Our results suggest that a rational selection of the applied method represents a powerful strategy to maximize the success of a research project, tightly linked to its aims. We employed cyclooxygenase as application example, however, the focus of this study lied on highlighting the differences in the virtual screening tool performances and not in the identification of novel COX-inhibitors. Copyright © 2015 The Authors. Published by Elsevier Masson SAS.. All rights reserved.

Managing the "Performance" in Performance Management.

ERIC Educational Resources Information Center

Repinski, Marilyn; Bartsch, Maryjo

1996-01-01

Describes a five-step approach to performance management which includes (1) redefining tasks; (2) identifying skills; (3) determining what development tools are necessary; (4) prioritizing skills development; and (5) developing an action plan. Presents a hiring model that includes job analysis, job description, selection, goal setting, evaluation,…

A Process-Centered Tool for Evaluating Patient Safety Performance and Guiding Strategic Improvement

DTIC Science & Technology

2005-01-01

next patient safety steps in individual health care organizations. The low priority given to Category 3 (Focus on patients , other customers , and...presents a patient safety applicator tool for implementing and assessing patient safety systems in health care institutions. The applicator tool consists...the survey rounds. The study addressed three research questions: 1. What critical processes should be included in health care patient safety systems

Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the ‘Black Box’ of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda

PubMed Central

Holvoet, Nathalie; Criel, Bart

2017-01-01

Increased attention on “complexity” in health systems evaluation has resulted in many different methodological responses. Theory-driven evaluations and systems thinking are two such responses that aim for better understanding of the mechanisms underlying given outcomes. Here, we studied the implementation of a performance-based financing intervention by the Belgian Technical Cooperation in Western Uganda to illustrate a methodological strategy of combining these two approaches. We utilized a systems dynamics tool called causal loop diagramming (CLD) to generate hypotheses feeding into a theory-driven evaluation. Semi-structured interviews were conducted with 30 health workers from two districts (Kasese and Kyenjojo) and with 16 key informants. After CLD, we identified three relevant hypotheses: “success to the successful”, “growth and underinvestment”, and “supervision conundrum”. The first hypothesis leads to increasing improvements in performance, as better performance leads to more incentives, which in turn leads to better performance. The latter two hypotheses point to potential bottlenecks. Thus, the proposed methodological strategy was a useful tool for identifying hypotheses that can inform a theory-driven evaluation. The hypotheses are represented in a comprehensible way while highlighting the underlying assumptions, and are more easily falsifiable than hypotheses identified without using CLD. PMID:28869518

Performance Evaluation of the NASA/KSC Transmission System

NASA Technical Reports Server (NTRS)

Christensen, Kenneth J.

2000-01-01

NASA-KSC currently uses three bridged 100-Mbps FDDI segments as its backbone for data traffic. The FDDI Transmission System (FTXS) connects the KSC industrial area, KSC launch complex 39 area, and the Cape Canaveral Air Force Station. The report presents a performance modeling study of the FTXS and the proposed ATM Transmission System (ATXS). The focus of the study is on performance of MPEG video transmission on these networks. Commercial modeling tools - the CACI Predictor and Comnet tools - were used. In addition, custom software tools were developed to characterize conversation pairs in Sniffer trace (capture) files to use as input to these tools. A baseline study of both non-launch and launch day data traffic on the FTXS is presented. MPEG-1 and MPEG-2 video traffic was characterized and the shaping of it evaluated. It is shown that the characteristics of a video stream has a direct effect on its performance in a network. It is also shown that shaping of video streams is necessary to prevent overflow losses and resulting poor video quality. The developed models can be used to predict when the existing FTXS will 'run out of room' and for optimizing the parameters of ATM links used for transmission of MPEG video. Future work with these models can provide useful input and validation to set-top box projects within the Advanced Networks Development group in NASA-KSC Development Engineering.

The identification of complete domains within protein sequences using accurate E-values for semi-global alignment

PubMed Central

Kann, Maricel G.; Sheetlin, Sergey L.; Park, Yonil; Bryant, Stephen H.; Spouge, John L.

2007-01-01

The sequencing of complete genomes has created a pressing need for automated annotation of gene function. Because domains are the basic units of protein function and evolution, a gene can be annotated from a domain database by aligning domains to the corresponding protein sequence. Ideally, complete domains are aligned to protein subsequences, in a ‘semi-global alignment’. Local alignment, which aligns pieces of domains to subsequences, is common in high-throughput annotation applications, however. It is a mature technique, with the heuristics and accurate E-values required for screening large databases and evaluating the screening results. Hidden Markov models (HMMs) provide an alternative theoretical framework for semi-global alignment, but their use is limited because they lack heuristic acceleration and accurate E-values. Our new tool, GLOBAL, overcomes some limitations of previous semi-global HMMs: it has accurate E-values and the possibility of the heuristic acceleration required for high-throughput applications. Moreover, according to a standard of truth based on protein structure, two semi-global HMM alignment tools (GLOBAL and HMMer) had comparable performance in identifying complete domains, but distinctly outperformed two tools based on local alignment. When searching for complete protein domains, therefore, GLOBAL avoids disadvantages commonly associated with HMMs, yet maintains their superior retrieval performance. PMID:17596268

The relationship between the managerial skills and results of "performance evaluation "tool among nursing managers in teaching hospitals of Iran University of Medical Science.

PubMed

Isfahani, Haleh Mousavi; Aryankhesal, Aidin; Haghani, Hamid

2014-09-25

Performance of different organizations, such as hospitals is mainly influenced by their managers' performance. Nursing managers have an important role in hospital performance and their managerial skills can improve the quality of the services. Hence, the present study was conducted in order to assess the relationship between the managerial skills and the results of their performance evaluation in Teaching Hospitals of Iran University of Medical Science in 2013. The research used the cross sectional method in 2013. It was done by distributing a managerial skills assessment questionnaire, with close-ended questions in 5 choice Likert scale, among 181 managers and head nurses of hospitals of Iran university of Medical Sciences; among which 131 answered the questions. Another data collection tools was a forms to record evaluation marks from the personnel records. We used Pearson and Spearman correlation tests and SPSS for analysis and description (frequency, mean and standard deviation). Results showed that the managerial skills of the nursing mangers were fair (2.57 out of 5) and the results of the performance evaluation were in a good condition (98.44). The mangers' evaluation results and the managerial skills scores were not in a meaningful correlation (r=0.047 np=0.856). The research showed no correlation between different domains of managerial skills and the performance evaluation marks: decision making skills (r=0.074 and p=0.399), leadership (correlation coefficient 0.028 and p=0.654), motivation (correlation coefficient 0.118 and p=0.163), communication  (correlation coefficient 0.116 and p=0.122), systematic thinking  (correlation coefficient 0.028 and p=0.828), time management (correlation coefficient 0.077 and p=0.401) and strategic thinking  (correlation coefficient 0.041 and p=0.756). Lack of any correlation and relation between managers' managerial skills and their performance evaluation results shows need to a fundamental revision at managers' performance evaluation form.

Methods for assessing fracture risk prediction models: experience with FRAX in a large integrated health care delivery system.

PubMed

Pressman, Alice R; Lo, Joan C; Chandra, Malini; Ettinger, Bruce

2011-01-01

Area under the receiver operating characteristics (AUROC) curve is often used to evaluate risk models. However, reclassification tests provide an alternative assessment of model performance. We performed both evaluations on results from FRAX (World Health Organization Collaborating Centre for Metabolic Bone Diseases, University of Sheffield, UK), a fracture risk tool, using Kaiser Permanente Northern California women older than 50yr with bone mineral density (BMD) measured during 1997-2003. We compared FRAX performance with and without BMD in the model. Among 94,489 women with mean follow-up of 6.6yr, 1579 (1.7%) sustained a hip fracture. Overall, AUROCs were 0.83 and 0.84 for FRAX without and with BMD, suggesting that BMD did not contribute to model performance. AUROC decreased with increasing age, and BMD contributed significantly to higher AUROC among those aged 70yr and older. Using an 81% sensitivity threshold (optimum level from receiver operating characteristic curve, corresponding to 1.2% cutoff), 35% of those categorized above were reassigned below when BMD was added. In contrast, only 10% of those categorized below were reassigned to the higher risk category when BMD was added. The net reclassification improvement was 5.5% (p<0.01). Two versions of this risk tool have similar AUROCs, but alternative assessments indicate that addition of BMD improves performance. Multiple methods should be used to evaluate risk tool performance with less reliance on AUROC alone. Copyright © 2011 The International Society for Clinical Densitometry. Published by Elsevier Inc. All rights reserved.

Distributed Space Mission Design for Earth Observation Using Model-Based Performance Evaluation

NASA Technical Reports Server (NTRS)

Nag, Sreeja; LeMoigne-Stewart, Jacqueline; Cervantes, Ben; DeWeck, Oliver

2015-01-01

Distributed Space Missions (DSMs) are gaining momentum in their application to earth observation missions owing to their unique ability to increase observation sampling in multiple dimensions. DSM design is a complex problem with many design variables, multiple objectives determining performance and cost and emergent, often unexpected, behaviors. There are very few open-access tools available to explore the tradespace of variables, minimize cost and maximize performance for pre-defined science goals, and therefore select the most optimal design. This paper presents a software tool that can multiple DSM architectures based on pre-defined design variable ranges and size those architectures in terms of predefined science and cost metrics. The tool will help a user select Pareto optimal DSM designs based on design of experiments techniques. The tool will be applied to some earth observation examples to demonstrate its applicability in making some key decisions between different performance metrics and cost metrics early in the design lifecycle.

An iterative learning control method with application for CNC machine tools

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kim, D.I.; Kim, S.

1996-01-01

A proportional, integral, and derivative (PID) type iterative learning controller is proposed for precise tracking control of industrial robots and computer numerical controller (CNC) machine tools performing repetitive tasks. The convergence of the output error by the proposed learning controller is guaranteed under a certain condition even when the system parameters are not known exactly and unknown external disturbances exist. As the proposed learning controller is repeatedly applied to the industrial robot or the CNC machine tool with the path-dependent repetitive task, the distance difference between the desired path and the actual tracked or machined path, which is one ofmore » the most significant factors in the evaluation of control performance, is progressively reduced. The experimental results demonstrate that the proposed learning controller can improve machining accuracy when the CNC machine tool performs repetitive machining tasks.« less

Refining prognosis in lung cancer: A report on the quality and relevance of clinical prognostic tools

PubMed Central

Mahar, Alyson L.; Compton, Carolyn; McShane, Lisa M.; Halabi, Susan; Asamura, Hisao; Rami-Porta, Ramon; Groome, Patti A.

2015-01-01

Introduction Accurate, individualized prognostication for lung cancer patients requires the integration of standard patient and pathologic factors, biologic, genetic, and other molecular characteristics of the tumor. Clinical prognostic tools aim to aggregate information on an individual patient to predict disease outcomes such as overall survival, but little is known about their clinical utility and accuracy in lung cancer. Methods A systematic search of the scientific literature for clinical prognostic tools in lung cancer published Jan 1, 1996-Jan 27, 2015 was performed. In addition, web-based resources were searched. A priori criteria determined by the Molecular Modellers Working Group of the American Joint Committee on Cancer were used to investigate the quality and usefulness of tools. Criteria included clinical presentation, model development approaches, validation strategies, and performance metrics. Results Thirty-two prognostic tools were identified. Patients with metastases were the most frequently considered population in non-small cell lung cancer. All tools for small cell lung cancer covered that entire patient population. Included prognostic factors varied considerably across tools. Internal validity was not formally evaluated for most tools and only eleven were evaluated for external validity. Two key considerations were highlighted for tool development: identification of an explicit purpose related to a relevant clinical population and clear decision-points, and prioritized inclusion of established prognostic factors over emerging factors. Conclusions Prognostic tools will contribute more meaningfully to the practice of personalized medicine if better study design and analysis approaches are used in their development and validation. PMID:26313682

Ceramic tools insert assesment based on vickers indentation methodology

NASA Astrophysics Data System (ADS)

Husni; Rizal, Muhammad; Aziz M, M.; Wahyu, M.

2018-05-01

In the interrupted cutting process, the risk of tool chipping or fracture is higher than continues cutting. Therefore, the selection of suitable ceramic tools for interrupted cutting application become an important issue to assure that the cutting process is running effectively. At present, the performance of ceramics tools is assessed by conducting some cutting tests, which is required time and cost consuming. In this study, the performance of ceramic tools evaluated using hardness tester machine. The technique, in general, has a certain advantage compare with the more conventional methods; the experimental is straightforward involving minimal specimen preparation and the amount of material needed is small. Three types of ceramic tools AS10, CC650 and K090 have been used, each tool was polished then Vickers indentation test were performed with the load were 0.2, 0.5, 1, 2.5, 5 and 10 kgf. The results revealed that among the load used in the tests, the indentation loads of 5 kgf always produce well cracks as compared with others. Among the cutting tool used in the tests, AS10 has produced the shortest crack length and follow by CC 670, and K090. It is indicated that the shortest crack length of AS10 reflected that the tool has a highest dynamic load resistance among others insert.

DeltaSA tool for source apportionment benchmarking, description and sensitivity analysis

NASA Astrophysics Data System (ADS)

Pernigotti, D.; Belis, C. A.

2018-05-01

DeltaSA is an R-package and a Java on-line tool developed at the EC-Joint Research Centre to assist and benchmark source apportionment applications. Its key functionalities support two critical tasks in this kind of studies: the assignment of a factor to a source in factor analytical models (source identification) and the model performance evaluation. The source identification is based on the similarity between a given factor and source chemical profiles from public databases. The model performance evaluation is based on statistical indicators used to compare model output with reference values generated in intercomparison exercises. The references values are calculated as the ensemble average of the results reported by participants that have passed a set of testing criteria based on chemical profiles and time series similarity. In this study, a sensitivity analysis of the model performance criteria is accomplished using the results of a synthetic dataset where "a priori" references are available. The consensus modulated standard deviation punc gives the best choice for the model performance evaluation when a conservative approach is adopted.

Evaluation methodology for comparing memory and communication of analytic processes in visual analytics

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ragan, Eric D; Goodall, John R

2014-01-01

Provenance tools can help capture and represent the history of analytic processes. In addition to supporting analytic performance, provenance tools can be used to support memory of the process and communication of the steps to others. Objective evaluation methods are needed to evaluate how well provenance tools support analyst s memory and communication of analytic processes. In this paper, we present several methods for the evaluation of process memory, and we discuss the advantages and limitations of each. We discuss methods for determining a baseline process for comparison, and we describe various methods that can be used to elicit processmore » recall, step ordering, and time estimations. Additionally, we discuss methods for conducting quantitative and qualitative analyses of process memory. By organizing possible memory evaluation methods and providing a meta-analysis of the potential benefits and drawbacks of different approaches, this paper can inform study design and encourage objective evaluation of process memory and communication.« less

Using Peer Assessment to Evaluate Teamwork from a Multidisciplinary Perspective

ERIC Educational Resources Information Center

Planas-Lladó, Anna; Feliu, Lidia; Castro, Francesc; Fraguell, Rosa Maria; Arbat, Gerard; Pujol, Joan; Suñol, Joan Josep; Daunis-i-Estadella, Pepus

2018-01-01

This article analyses the use of peer evaluation as a tool for evaluating teamwork and students' perceptions of this type of evaluation. A study was conducted of six subjects included on five degree courses at the University of Girona. In all of these subjects, students carried out a team activity, evaluated the performance of the team and the…

Relations between mental health team characteristics and work role performance.

PubMed

Fleury, Marie-Josée; Grenier, Guy; Bamvita, Jean-Marie; Farand, Lambert

2017-01-01

Effective mental health care requires a high performing, interprofessional team. Among 79 mental health teams in Quebec (Canada), this exploratory study aims to 1) determine the association between work role performance and a wide range of variables related to team effectiveness according to the literature, and to 2) using structural equation modelling, assess the covariance between each of these variables as well as the correlation with other exogenous variables. Work role performance was measured with an adapted version of a work role questionnaire. Various independent variables including team manager characteristics, user characteristics, team profiles, clinical activities, organizational culture, network integration strategies and frequency/satisfaction of interactions with other teams or services were analyzed under the structural equation model. The later provided a good fit with the data. Frequent use of standardized procedures and evaluation tools (e.g. screening and assessment tools for mental health disorders) and team manager seniority exerted the most direct effect on work role performance. While network integration strategies had little effect on work role performance, there was a high covariance between this variable and those directly affecting work role performance among mental health teams. The results suggest that the mental healthcare system should apply standardized procedures and evaluation tools and, to a lesser extent, clinical approaches to improve work role performance in mental health teams. Overall, a more systematic implementation of network integration strategies may contribute to improved work role performance in mental health care.

Relations between mental health team characteristics and work role performance

PubMed Central

Grenier, Guy; Bamvita, Jean-Marie; Farand, Lambert

2017-01-01

Effective mental health care requires a high performing, interprofessional team. Among 79 mental health teams in Quebec (Canada), this exploratory study aims to 1) determine the association between work role performance and a wide range of variables related to team effectiveness according to the literature, and to 2) using structural equation modelling, assess the covariance between each of these variables as well as the correlation with other exogenous variables. Work role performance was measured with an adapted version of a work role questionnaire. Various independent variables including team manager characteristics, user characteristics, team profiles, clinical activities, organizational culture, network integration strategies and frequency/satisfaction of interactions with other teams or services were analyzed under the structural equation model. The later provided a good fit with the data. Frequent use of standardized procedures and evaluation tools (e.g. screening and assessment tools for mental health disorders) and team manager seniority exerted the most direct effect on work role performance. While network integration strategies had little effect on work role performance, there was a high covariance between this variable and those directly affecting work role performance among mental health teams. The results suggest that the mental healthcare system should apply standardized procedures and evaluation tools and, to a lesser extent, clinical approaches to improve work role performance in mental health teams. Overall, a more systematic implementation of network integration strategies may contribute to improved work role performance in mental health care. PMID:28991923

Performance evaluation of nonhomogeneous hospitals: the case of Hong Kong hospitals.

PubMed

Li, Yongjun; Lei, Xiyang; Morton, Alec

2018-02-14

Throughout the world, hospitals are under increasing pressure to become more efficient. Efficiency analysis tools can play a role in giving policymakers insight into which units are less efficient and why. Many researchers have studied efficiencies of hospitals using data envelopment analysis (DEA) as an efficiency analysis tool. However, in the existing literature on DEA-based performance evaluation, a standard assumption of the constant returns to scale (CRS) or the variable returns to scale (VRS) DEA models is that decision-making units (DMUs) use a similar mix of inputs to produce a similar set of outputs. In fact, hospitals with different primary goals supply different services and provide different outputs. That is, hospitals are nonhomogeneous and the standard assumption of the DEA model is not applicable to the performance evaluation of nonhomogeneous hospitals. This paper considers the nonhomogeneity among hospitals in the performance evaluation and takes hospitals in Hong Kong as a case study. An extension of Cook et al. (2013) [1] based on the VRS assumption is developed to evaluated nonhomogeneous hospitals' efficiencies since inputs of hospitals vary greatly. Following the philosophy of Cook et al. (2013) [1], hospitals are divided into homogeneous groups and the product process of each hospital is divided into subunits. The performance of hospitals is measured on the basis of subunits. The proposed approach can be applied to measure the performance of other nonhomogeneous entities that exhibit variable return to scale.

Integrated performance and reliability specification for digital avionics systems

NASA Technical Reports Server (NTRS)

Brehm, Eric W.; Goettge, Robert T.

1995-01-01

This paper describes an automated tool for performance and reliability assessment of digital avionics systems, called the Automated Design Tool Set (ADTS). ADTS is based on an integrated approach to design assessment that unifies traditional performance and reliability views of system designs, and that addresses interdependencies between performance and reliability behavior via exchange of parameters and result between mathematical models of each type. A multi-layer tool set architecture has been developed for ADTS that separates the concerns of system specification, model generation, and model solution. Performance and reliability models are generated automatically as a function of candidate system designs, and model results are expressed within the system specification. The layered approach helps deal with the inherent complexity of the design assessment process, and preserves long-term flexibility to accommodate a wide range of models and solution techniques within the tool set structure. ADTS research and development to date has focused on development of a language for specification of system designs as a basis for performance and reliability evaluation. A model generation and solution framework has also been developed for ADTS, that will ultimately encompass an integrated set of analytic and simulated based techniques for performance, reliability, and combined design assessment.

MiRduplexSVM: A High-Performing MiRNA-Duplex Prediction and Evaluation Methodology

PubMed Central

Karathanasis, Nestoras; Tsamardinos, Ioannis; Poirazi, Panayiota

2015-01-01

We address the problem of predicting the position of a miRNA duplex on a microRNA hairpin via the development and application of a novel SVM-based methodology. Our method combines a unique problem representation and an unbiased optimization protocol to learn from mirBase19.0 an accurate predictive model, termed MiRduplexSVM. This is the first model that provides precise information about all four ends of the miRNA duplex. We show that (a) our method outperforms four state-of-the-art tools, namely MaturePred, MiRPara, MatureBayes, MiRdup as well as a Simple Geometric Locator when applied on the same training datasets employed for each tool and evaluated on a common blind test set. (b) In all comparisons, MiRduplexSVM shows superior performance, achieving up to a 60% increase in prediction accuracy for mammalian hairpins and can generalize very well on plant hairpins, without any special optimization. (c) The tool has a number of important applications such as the ability to accurately predict the miRNA or the miRNA*, given the opposite strand of a duplex. Its performance on this task is superior to the 2nts overhang rule commonly used in computational studies and similar to that of a comparative genomic approach, without the need for prior knowledge or the complexity of performing multiple alignments. Finally, it is able to evaluate novel, potential miRNAs found either computationally or experimentally. In relation with recent confidence evaluation methods used in miRBase, MiRduplexSVM was successful in identifying high confidence potential miRNAs. PMID:25961860

48 CFR 642.1503-70 - Contractor Performance System (CPS).

Code of Federal Regulations, 2011 CFR

2011-10-01

... Performance System (CPS) maintained by the National Institutes of Health. CPS is an Internet-based tool... with access to the Internet shall use CPS to evaluate contractor's past performance for all contracts... the CPS; or (2) At overseas locations where access to the Internet is not practicable. (d) Heads of...

Design tool for multiprocessor scheduling and evaluation of iterative dataflow algorithms

NASA Technical Reports Server (NTRS)

Jones, Robert L., III

1995-01-01

A graph-theoretic design process and software tool is defined for selecting a multiprocessing scheduling solution for a class of computational problems. The problems of interest are those that can be described with a dataflow graph and are intended to be executed repetitively on a set of identical processors. Typical applications include signal processing and control law problems. Graph-search algorithms and analysis techniques are introduced and shown to effectively determine performance bounds, scheduling constraints, and resource requirements. The software tool applies the design process to a given problem and includes performance optimization through the inclusion of additional precedence constraints among the schedulable tasks.

The coronary artery disease quality dashboard: a chronic care disease management tool in an electronic health record.

PubMed

Jung, Eunice; Schnipper, Jeffrey L; Li, Qi; Linder, Jeffrey A; Rose, Alan F; Li, Ruzhuo; Eskin, Michael S; Housman, Dan; Middleton, Blackford; Einbinder, Jonathan S

2007-10-11

Quality reporting tools, integrated with ambulatory electronic health records (EHRs), may help clinicians understand performance, manage populations, and improve quality. The Coronary Artery Disease Quality Dash board (CAD QD) is a secure web report for performance measurement of a chronic care condition delivered through a central data warehouse and custom-built reporting tool. Pilot evaluation of the CAD Quality Dash board indicates that clinicians prefer a quality report that combines not only structured data from EHRs but one that facilitates actions to be taken on individual patients or on a population, i.e., for case management.

Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction.

PubMed

Park, Seong Ho; Han, Kyunghwa

2018-03-01

The use of artificial intelligence in medicine is currently an issue of great interest, especially with regard to the diagnostic or predictive analysis of medical images. Adoption of an artificial intelligence tool in clinical practice requires careful confirmation of its clinical utility. Herein, the authors explain key methodology points involved in a clinical evaluation of artificial intelligence technology for use in medicine, especially high-dimensional or overparameterized diagnostic or predictive models in which artificial deep neural networks are used, mainly from the standpoints of clinical epidemiology and biostatistics. First, statistical methods for assessing the discrimination and calibration performances of a diagnostic or predictive model are summarized. Next, the effects of disease manifestation spectrum and disease prevalence on the performance results are explained, followed by a discussion of the difference between evaluating the performance with use of internal and external datasets, the importance of using an adequate external dataset obtained from a well-defined clinical cohort to avoid overestimating the clinical performance as a result of overfitting in high-dimensional or overparameterized classification model and spectrum bias, and the essentials for achieving a more robust clinical evaluation. Finally, the authors review the role of clinical trials and observational outcome studies for ultimate clinical verification of diagnostic or predictive artificial intelligence tools through patient outcomes, beyond performance metrics, and how to design such studies. © RSNA, 2018.

Developing the Thai Siriraj Psoriatic Arthritis Screening Tool and validating the Thai Psoriasis Epidemiology Screening Tool and the Early Arthritis for Psoriatic Patients questionnaire.

PubMed

Chiowchanwisawakit, Praveena; Wattanamongkolsil, Luksame; Srinonprasert, Varalak; Petcharat, Chonachan; Siriwanarangsun, Palanan; Katchamart, Wanruchada

2016-10-01

To validate the Thai language version of the Psoriasis Epidemiology Screening Tool (PEST) and the Early Arthritis for Psoriatic Patients Questionnaire (EARP), as well as also to develop a new tool for screening psoriatic arthritis (PsA) among psoriasis (Ps) patients. This was a cross-sectional study. Ps patients visiting the psoriasis clinic at Siriraj Hospital were recruited. They completed the EARP and PEST. Full musculoskeletal history, examination, and radiography were evaluated. PsA was diagnosed by a rheumatologist's evaluation and fulfillment of the classification criteria for psoriatic arthritis. Receiver operator characteristic (ROC) curves, sensitivity, and specificity were used to evaluate the performances of the tools. The Siriraj Psoriatic Arthritis Screening Tool (SiPAT) contained questions most relevant to peripheral arthritis, axial inflammation, and enthesitis, selected from multivariate analysis. Of a total of 159 patients, the prevalence of PsA was 78.6 %. The ROC curve analyses of Thai EARP, PEST, and SiPAT were 0.90 (95 % CI 0.84, 0.96), 0.85 (0.78, 0.92), and 0.89 (0.83, 0.95), respectively. The sensitivities of SiPAT, Thai EARP, and PEST were 91.0, 83.0, and 72.0 %, respectively, while the specificities were 69.0, 79.3, and 89.7 %, respectively. All screening questionnaires showed good diagnostic performances. SiPAT could be considered as a screening tool with its desirable properties: higher sensitivity and taking less time. Thai PEST and EARP could possibly be sequentially administered for people with a positive test from SiPAT to reduce the number of false positives.

A decision support tool to determine cost-to-benefit of a family-centered in-home program for at-risk adolescents.

PubMed

Wilson, Fernando A; Araz, Ozgur M; Thompson, Ronald W; Ringle, Jay L; Mason, W Alex; Stimpson, Jim P

2016-06-01

Family-centered program research has demonstrated its effectiveness in improving adolescent outcomes. However, given current fiscal constraints faced by governmental agencies, a recent report from the Institute of Medicine and National Research Council highlighted the need for cost-benefit analyses to inform decision making by policymakers. Furthermore, performance management tools such as balanced scorecards and dashboards do not generally include cost-benefit analyses. In this paper, we describe the development of an Excel-based decision support tool that can be used to evaluate a selected family-based program for at-risk children and adolescents relative to a comparison program or the status quo. This tool incorporates the use of an efficient, user-friendly interface with results provided in concise tabular and graphical formats that may be interpreted without need for substantial training in economic evaluation. To illustrate, we present an application of this tool to evaluate use of Boys Town's In-Home Family Services (IHFS) relative to detention and out-of-home placement in New York City. Use of the decision support tool can help mitigate the need for programs to contract experts in economic evaluation, especially when there are financial or time constraints. Copyright © 2016 Elsevier Ltd. All rights reserved.

USING A LIFE-CYCLE APPROACH TO ACHIEVE SUSTAINABLE MUNICIPAL SOLID WASTE MANAGEMENT STRATEGIES IN THE UNITED STATES

EPA Science Inventory

The paper discusses a computer-based decision support tool that has been developed to assist local governments in evaluating the cost and environmental performance of integrated municipal solid waste (MSW) managment systems. ongoing case studies of the tool at the local level are...

Power Systems Design and Studies | Grid Modernization | NREL

Science.gov Websites

Design and Studies Power Systems Design and Studies NREL develops new tools, algorithms, and market design and performance evaluations; and planning, operations, and protection studies. Photo of two researchers looking at a screen showing a distribution grid map Current design and planning tools for the

Comparing Two CBM Maze Selection Tools: Considering Scoring and Interpretive Metrics for Universal Screening

ERIC Educational Resources Information Center

Ford, Jeremy W.; Missall, Kristen N.; Hosp, John L.; Kuhle, Jennifer L.

2016-01-01

Advances in maze selection curriculum-based measurement have led to several published tools with technical information for interpretation (e.g., norms, benchmarks, cut-scores, classification accuracy) that have increased their usefulness for universal screening. A range of scoring practices have emerged for evaluating student performance on maze…

Useful Interactive Teaching Tool for Learning: Clickers in Higher Education

ERIC Educational Resources Information Center

Camacho-Miñano, María-del-Mar; del Campo, Cristina

2016-01-01

Many university lecturers are encouraged to implement innovative teaching tools and methodologies such as clickers in order to create an interactive learning environment and improve student learning, but its performance must be evaluated. The aim of this paper is to test empirically the impact of the use of clickers on students' learning…

A Web-Based Learning Tool Improves Student Performance in Statistics: A Randomized Masked Trial

ERIC Educational Resources Information Center

Gonzalez, Jose A.; Jover, Lluis; Cobo, Erik; Munoz, Pilar

2010-01-01

Background: e-status is a web-based tool able to generate different statistical exercises and to provide immediate feedback to students' answers. Although the use of Information and Communication Technologies (ICTs) is becoming widespread in undergraduate education, there are few experimental studies evaluating its effects on learning. Method: All…

Fast analysis of radionuclide decay chain migration

NASA Astrophysics Data System (ADS)

Chen, J. S.; Liang, C. P.; Liu, C. W.; Li, L.

2014-12-01

A novel tool for rapidly predicting the long-term plume behavior of an arbitrary length radionuclide decay chain is presented in this study. This fast tool is achieved based on generalized analytical solutions in compact format derived for a set of two-dimensional advection-dispersion equations coupled with sequential first-order decay reactions in groundwater system. The performance of the developed tool is evaluated by a numerical model using a Laplace transform finite difference scheme. The results of performance evaluation indicate that the developed model is robust and accurate. The developed model is then used to fast understand the transport behavior of a four-member radionuclide decay chain. Results show that the plume extents and concentration levels of any target radionuclide are very sensitive to longitudinal, transverse dispersion, decay rate constant and retardation factor. The developed model are useful tools for rapidly assessing the ecological and environmental impact of the accidental radionuclide releases such as the Fukushima nuclear disaster where multiple radionuclides leaked through the reactor, subsequently contaminating the local groundwater and ocean seawater in the vicinity of the nuclear plant.

Development and psychometric evaluation of the "Neurosurgical Evaluation of Attitudes towards simulation Training" (NEAT) tool for use in neurosurgical education and training.

PubMed

Kirkman, Matthew A; Muirhead, William; Nandi, Dipankar; Sevdalis, Nick

2014-01-01

Neurosurgical simulation training is becoming increasingly popular. Attitudes toward simulation among residents can contribute to the effectiveness of simulation training, but such attitudes remain poorly explored in neurosurgery with no psychometrically proven measure in the literature. The aim of the present study was to evaluate prospectively a newly developed tool for this purpose: the Neurosurgical Evaluation of Attitudes towards simulation Training (NEAT). The NEAT tool was prospectively developed in 2 stages and psychometrically evaluated (validity and reliability) in 2 administrations with the same participants. The tool comprises a questionnaire with 9 Likert scale items and 2 free-text sections assessing attitudes toward simulation in neurosurgery. The evaluation was completed with 31 neurosurgery residents in London, United Kingdom, who were generally favorable toward neurosurgical simulation. The internal consistency of the questionnaire was high, as demonstrated by the overall Cronbach α values (α=0.899 and α=0.955). All but 2 questionnaire items had "substantial" or "almost perfect" test-retest reliability following repeated survey administrations (median Pearson r correlation=0.688; range, 0.248-0.841). NEAT items were well correlated with each other on both occasions, showing good validity of content within the NEAT tool. There was no significant relationship between either gender or length of neurosurgical experience and item ratings. NEAT is the first psychometrically evaluated tool for evaluating attitudes toward simulation in neurosurgery. Further implementation of NEAT is required in wider neurosurgical populations to establish whether specific population groups differ. Use of NEAT in studies of neurosurgical simulation could offer an additional outcome measure to performance metrics, permitting evaluation of the impact of neurosurgical simulation on attitudes toward simulation both between participants and within the same participants over time. Copyright © 2014 Elsevier Inc. All rights reserved.

Value engineering on the designed operator work tools for brick and rings wells production

NASA Astrophysics Data System (ADS)

Ayu Bidiawati J., R.; Muchtiar, Yesmizarti; Wariza, Ragil Okta

2017-06-01

Operator working tools in making brick and ring wells were designed and made, and the value engineering was calculated to identify and develop the function of these tools in obtaining the balance between cost, reliability and appearance. This study focused on the value of functional components of the tools and attempted to increase the difference between the costs incurred by the generated values. The purpose of this study was to determine the alternatives of tools design and to determine the performance of each alternative. The technique was developed using FAST method that consisted of five stages: information, creative, analytical, development and presentation stage. The results of the analysis concluded that the designed tools have higher value and better function description. There were four alternative draft improvements for operator working tools. The best alternative was determined based on the rank by using matrix evaluation. Best performance was obtained by the alternative II, amounting to 98.92 with a value of 0.77.

Benchmarking and Self-Assessment in the Wine Industry

DOE Office of Scientific and Technical Information (OSTI.GOV)

Galitsky, Christina; Radspieler, Anthony; Worrell, Ernst

2005-12-01

Not all industrial facilities have the staff or theopportunity to perform a detailed audit of their operations. The lack ofknowledge of energy efficiency opportunities provides an importantbarrier to improving efficiency. Benchmarking programs in the U.S. andabroad have shown to improve knowledge of the energy performance ofindustrial facilities and buildings and to fuel energy managementpractices. Benchmarking provides a fair way to compare the energyintensity of plants, while accounting for structural differences (e.g.,the mix of products produced, climate conditions) between differentfacilities. In California, the winemaking industry is not only one of theeconomic pillars of the economy; it is also a large energymore » consumer, witha considerable potential for energy-efficiency improvement. LawrenceBerkeley National Laboratory and Fetzer Vineyards developed the firstbenchmarking tool for the California wine industry called "BEST(Benchmarking and Energy and water Savings Tool) Winery". BEST Wineryenables a winery to compare its energy efficiency to a best practicereference winery. Besides overall performance, the tool enables the userto evaluate the impact of implementing efficiency measures. The toolfacilitates strategic planning of efficiency measures, based on theestimated impact of the measures, their costs and savings. The tool willraise awareness of current energy intensities and offer an efficient wayto evaluate the impact of future efficiency measures.« less

Implementation of Insight Responsibilities in Process Engineering

NASA Technical Reports Server (NTRS)

Osborne, Deborah M.

1997-01-01

This report describes an approach for evaluating flight readiness (COFR) and contractor performance evaluation (award fee) as part of the insight role of NASA Process Engineering at Kennedy Space Center. Several evaluation methods are presented, including systems engineering evaluations and use of systems performance data. The transition from an oversight function to the insight function is described. The types of analytical tools appropriate for achieving the flight readiness and contractor performance evaluation goals are described and examples are provided. Special emphasis is placed upon short and small run statistical quality control techniques. Training requirements for system engineers are delineated. The approach described herein would be equally appropriate in other directorates at Kennedy Space Center.

Prioritizing guideline topics: development and evaluation of a practical tool.

PubMed

Ketola, Eeva; Toropainen, Erja; Kaila, Minna; Luoto, Riitta; Mäkelä, Marjukka

2007-08-01

A clear process for selecting and adopting clinical practice guidelines in the new topic areas is needed. The aim of this study is to design and develop a practical tool to assess guideline topics that have been suggested to the organization responsible for producing guidelines. We carried out an iterative development, feasibility and validation study of a guideline topic prioritization tool. The setting included the guideline producer organization and the tax-funded health care system. In the first stage of the tool development, participants were researchers, members of the Current Care Board and experts from health care organizations. In the second stage, the evaluation was done internally within the project by three independent reviewers. The main outcome measures were responses to an evaluation questionnaire, qualitative process feedback and analysis of the performance of the instrument on a random set of guidelines. Evaluations by three independent reviewers revealed good agreement and face validity with respect to its feasibility as a planning tool at the guideline board level. Feedback from board members suggested that the instrument is useful in prioritizing guideline topics. This instrument was accepted for use by the Board. Further developments are needed to ensure feedback and acceptability of the instrument by those proposing topics.

External Validation of a Decision Tool To Guide Post-Operative Management of Patients with Secondary Peritonitis.

PubMed

Atema, Jasper J; Ram, Kim; Schultz, Marcus J; Boermeester, Marja A

Timely identification of patients in need of an intervention for abdominal sepsis after initial surgical management of secondary peritonitis is vital but complex. The aim of this study was to validate a decision tool for this purpose and to evaluate its potential to guide post-operative management. A prospective cohort study was conducted on consecutive adult patients undergoing surgery for secondary peritonitis in a single hospital. Assessments using the decision tool, based on one intra-operative and five post-operative variables, were performed on the second and third post-operative days and when the patients' clinical status deteriorated. Scores were compared with the clinical reference standard of persistent sepsis based on the clinical course or findings at imaging or surgery. Additionally, the potential of the decision tool to guide management in terms of diagnostic imaging in three previously defined score categories (low, intermediate, and high) was evaluated. A total of 161 assessments were performed in 69 patients. The majority of cases of secondary peritonitis (68%) were caused by perforation of the gastrointestinal tract. Post-operative persistent sepsis occurred in 28 patients. The discriminative capacity of the decision tool score was fair (area under the curve of the receiver operating characteristic = 0.79). The incidence rate differed significantly between the three score categories (p < 0.001). The negative predictive value of a decision tool score categorized as low probability was 89% (95% confidence interval [CI] 82-94) and 65% (95% CI 47-79) for an intermediate score. Diagnostic imaging was performed more frequently when there was an intermediate score than when the score was categorized as low (46% vs. 24%; p < 0.001). In patients operated on for secondary peritonitis, the decision tool score predicts with fair accuracy whether persistent sepsis is present.

General Tool for Evaluating High-Contrast Coronagraphic Telescope Performance Error Budgets

NASA Technical Reports Server (NTRS)

Marchen, Luis F.

2011-01-01

The Coronagraph Performance Error Budget (CPEB) tool automates many of the key steps required to evaluate the scattered starlight contrast in the dark hole of a space-based coronagraph. The tool uses a Code V prescription of the optical train, and uses MATLAB programs to call ray-trace code that generates linear beam-walk and aberration sensitivity matrices for motions of the optical elements and line-of-sight pointing, with and without controlled fine-steering mirrors (FSMs). The sensitivity matrices are imported by macros into Excel 2007, where the error budget is evaluated. The user specifies the particular optics of interest, and chooses the quality of each optic from a predefined set of PSDs. The spreadsheet creates a nominal set of thermal and jitter motions, and combines that with the sensitivity matrices to generate an error budget for the system. CPEB also contains a combination of form and ActiveX controls with Visual Basic for Applications code to allow for user interaction in which the user can perform trade studies such as changing engineering requirements, and identifying and isolating stringent requirements. It contains summary tables and graphics that can be instantly used for reporting results in view graphs. The entire process to obtain a coronagraphic telescope performance error budget has been automated into three stages: conversion of optical prescription from Zemax or Code V to MACOS (in-house optical modeling and analysis tool), a linear models process, and an error budget tool process. The first process was improved by developing a MATLAB package based on the Class Constructor Method with a number of user-defined functions that allow the user to modify the MACOS optical prescription. The second process was modified by creating a MATLAB package that contains user-defined functions that automate the process. The user interfaces with the process by utilizing an initialization file where the user defines the parameters of the linear model computations. Other than this, the process is fully automated. The third process was developed based on the Terrestrial Planet Finder coronagraph Error Budget Tool, but was fully automated by using VBA code, form, and ActiveX controls.

Transforming data into usable knowledge: the CIRC experience

NASA Astrophysics Data System (ADS)

Mote, P.; Lach, D.; Hartmann, H.; Abatzoglou, J. T.; Stevenson, J.

2017-12-01

NOAA's northwest RISA, the Climate Impacts Research Consortium, emphasizes the transformation of data into usable knowledge. This effort involves physical scientists (e.g., Abatzoglou) building web-based tools with climate and hydrologic data and model output, a team performing data mining to link crop loss claims to droughts, social scientists (eg., Lach, Hartmann) evaluating the effectiveness of such tools at communicating with end users, and two-way engagement with a wide variety of audiences who are interested in using and improving the tools. Unusual in this effort is the seamless integration across timescales past, present, and future; data mining; and the level of effort in evaluating the tools. We provide examples of agriculturally relevant climate variables (e.g. growing degree days, day of first fall freeze) and describe the iterative process of incorporating user feedback.

The Tracking Meteogram, an AWIPS II Tool for Time-Series Analysis

NASA Technical Reports Server (NTRS)

Burks, Jason Eric; Sperow, Ken

2015-01-01

A new tool has been developed for the National Weather Service (NWS) Advanced Weather Interactive Processing System (AWIPS) II through collaboration between NASA's Short-term Prediction Research and Transition (SPoRT) and the NWS Meteorological Development Laboratory (MDL). Referred to as the "Tracking Meteogram", the tool aids NWS forecasters in assessing meteorological parameters associated with moving phenomena. The tool aids forecasters in severe weather situations by providing valuable satellite and radar derived trends such as cloud top cooling rates, radial velocity couplets, reflectivity, and information from ground-based lightning networks. The Tracking Meteogram tool also aids in synoptic and mesoscale analysis by tracking parameters such as the deepening of surface low pressure systems, changes in surface or upper air temperature, and other properties. The tool provides a valuable new functionality and demonstrates the flexibility and extensibility of the NWS AWIPS II architecture. In 2014, the operational impact of the tool was formally evaluated through participation in the NOAA/NWS Operations Proving Ground (OPG), a risk reduction activity to assess performance and operational impact of new forecasting concepts, tools, and applications. Performance of the Tracking Meteogram Tool during the OPG assessment confirmed that it will be a valuable asset to the operational forecasters. This presentation reviews development of the Tracking Meteogram tool, performance and feedback acquired during the OPG activity, and future goals for continued support and extension to other application areas.

Development of Response Surface Models for Rapid Analysis & Multidisciplinary Optimization of Launch Vehicle Design Concepts

NASA Technical Reports Server (NTRS)

Unal, Resit

1999-01-01

Multdisciplinary design optimization (MDO) is an important step in the design and evaluation of launch vehicles, since it has a significant impact on performance and lifecycle cost. The objective in MDO is to search the design space to determine the values of design parameters that optimize the performance characteristics subject to system constraints. Vehicle Analysis Branch (VAB) at NASA Langley Research Center has computerized analysis tools in many of the disciplines required for the design and analysis of launch vehicles. Vehicle performance characteristics can be determined by the use of these computerized analysis tools. The next step is to optimize the system performance characteristics subject to multidisciplinary constraints. However, most of the complex sizing and performance evaluation codes used for launch vehicle design are stand-alone tools, operated by disciplinary experts. They are, in general, difficult to integrate and use directly for MDO. An alternative has been to utilize response surface methodology (RSM) to obtain polynomial models that approximate the functional relationships between performance characteristics and design variables. These approximation models, called response surface models, are then used to integrate the disciplines using mathematical programming methods for efficient system level design analysis, MDO and fast sensitivity simulations. A second-order response surface model of the form given has been commonly used in RSM since in many cases it can provide an adequate approximation especially if the region of interest is sufficiently limited.

Can Teacher Evaluation Improve Teaching?

ERIC Educational Resources Information Center

Principal Leadership, 2013

2013-01-01

The answer to the question, Can evaluation improve teaching? is a qualified yes. Teacher evaluation has changed and the role of the principal has changed as well; the focus now is on evidence, not merely good judgment. With the right tools, systems, and support, it should be possible to help improve teaching performance and student learning…

Evaluation Checklist for Student Writing in Grades K-3, Ottawa County.

ERIC Educational Resources Information Center

Ottawa County Office of Education, OH.

Developed to assist teachers in Ottawa County, Ohio, in monitoring students' pupil performance objectives (PPOs) in grades K-3, this writing evaluation form is the primary record keeping tool in the Competency Based Education (CBE) Program. The form consists of: (1) the evaluation checklist; (2) the intervention code; and (3) record keeping…

The Ling 6(HL) test: typical pediatric performance data and clinical use evaluation.

PubMed

Glista, Danielle; Scollie, Susan; Moodie, Sheila; Easwar, Vijayalakshmi

2014-01-01

The Ling 6(HL) test offers a calibrated version of naturally produced speech sounds in dB HL for evaluation of detection thresholds. Aided performance has been previously characterized in adults. The purpose of this work was to evaluate and refine the Ling 6(HL) test for use in pediatric hearing aid outcome measurement. This work is presented across two studies incorporating an integrated knowledge translation approach in the characterization of normative and typical performance, and in the evaluation of clinical feasibility, utility, acceptability, and implementation. A total of 57 children, 28 normally hearing and 29 with binaural sensorineural hearing loss, were included in Study 1. Children wore their own hearing aids fitted using Desired Sensation Level v5.0. Nine clinicians from The Network of Pediatric Audiologists participated in Study 2. A CD-based test format was used in the collection of unaided and aided detection thresholds in laboratory and clinical settings; thresholds were measured clinically as part of routine clinical care. Confidence intervals were derived to characterize normal performance and typical aided performance according to hearing loss severity. Unaided-aided performance was analyzed using a repeated-measures analysis of variance. The audiologists completed an online questionnaire evaluating the quality, feasibility/executability, utility/comparative value/relative advantage, acceptability/applicability, and interpretability, in addition to recommendation and general comments sections. Ling 6(HL) thresholds were reliably measured with children 3-18 yr old. Normative and typical performance ranges were translated into a scoring tool for use in pediatric outcome measurement. In general, questionnaire respondents generally agreed that the Ling 6(HL) test was a high-quality outcome evaluation tool that can be implemented successfully in clinical settings. By actively collaborating with pediatric audiologists and using an integrated knowledge translation framework, this work supported the creation of an evidence-based clinical tool that has the potential to be implemented in, and useful to, clinical practice. More research is needed to characterize performance in alternative listening conditions to facilitate use with infants, for example. Future efforts focused on monitoring the use of the Ling 6(HL) test in daily clinical practice may help describe whether clinical use has been maintained across time and if any additional adaptations are necessary to facilitate clinical uptake. American Academy of Audiology.

Tranzit XPress : hazardous material fleet management and monitoring system : evaluation report

DOT National Transportation Integrated Search

1997-07-01

In this report the evaluation performed on the first phase of the Tranzit XPress system is presented. The system comprises of a traffic/safety control center, motor vehicle instrumentation, and a variety of off vehicle tools that communicate with eac...

Indicators and Measurement Tools for Health Systems Integration: A Knowledge Synthesis.

PubMed

Suter, Esther; Oelke, Nelly D; da Silva Lima, Maria Alice Dias; Stiphout, Michelle; Janke, Robert; Witt, Regina Rigatto; Van Vliet-Brown, Cheryl; Schill, Kaela; Rostami, Mahnoush; Hepp, Shelanne; Birney, Arden; Al-Roubaiai, Fatima; Marques, Giselda Quintana

2017-11-13

Despite far reaching support for integrated care, conceptualizing and measuring integrated care remains challenging. This knowledge synthesis aimed to identify indicator domains and tools to measure progress towards integrated care. We used an established framework and a Delphi survey with integration experts to identify relevant measurement domains. For each domain, we searched and reviewed the literature for relevant tools. From 7,133 abstracts, we retrieved 114 unique tools. We found many quality tools to measure care coordination, patient engagement and team effectiveness/performance. In contrast, there were few tools in the domains of performance measurement and information systems, alignment of organizational goals and resource allocation. The search yielded 12 tools that measure overall integration or three or more indicator domains. Our findings highlight a continued gap in tools to measure foundational components that support integrated care. In the absence of such targeted tools, "overall integration" tools may be useful for a broad assessment of the overall state of a system. Continued progress towards integrated care depends on our ability to evaluate the success of strategies across different levels and context. This study has identified 114 tools that measure integrated care across 16 domains, supporting efforts towards a unified measurement framework.

Comparative assessment of methods for the fusion transcripts detection from RNA-Seq data

PubMed Central

Kumar, Shailesh; Vo, Angie Duy; Qin, Fujun; Li, Hui

2016-01-01

RNA-Seq made possible the global identification of fusion transcripts, i.e. “chimeric RNAs”. Even though various software packages have been developed to serve this purpose, they behave differently in different datasets provided by different developers. It is important for both users, and developers to have an unbiased assessment of the performance of existing fusion detection tools. Toward this goal, we compared the performance of 12 well-known fusion detection software packages. We evaluated the sensitivity, false discovery rate, computing time, and memory usage of these tools in four different datasets (positive, negative, mixed, and test). We conclude that some tools are better than others in terms of sensitivity, positive prediction value, time consumption and memory usage. We also observed small overlaps of the fusions detected by different tools in the real dataset (test dataset). This could be due to false discoveries by various tools, but could also be due to the reason that none of the tools are inclusive. We have found that the performance of the tools depends on the quality, read length, and number of reads of the RNA-Seq data. We recommend that users choose the proper tools for their purpose based on the properties of their RNA-Seq data. PMID:26862001

Interactive Tools for Measuring Visual Scanning Performance and Reaction Time.

PubMed

Brooks, Johnell; Seeanner, Julia; Hennessy, Sarah; Manganelli, Joseph; Crisler, Matthew; Rosopa, Patrick; Jenkins, Casey; Anderson, Michael; Drouin, Nathalie; Belle, Leah; Truesdail, Constance; Tanner, Stephanie

Occupational therapists are constantly searching for engaging, high-technology interactive tasks that provide immediate feedback to evaluate and train clients with visual scanning deficits. This study examined the relationship between two tools: the VISION COACH™ interactive light board and the Functional Object Detection © (FOD) Advanced driving simulator scenario. Fifty-four healthy drivers, ages 21-66 yr, were divided into three age groups. Participants performed braking response and visual target (E) detection tasks of the FOD Advanced driving scenario, followed by two sets of three trials using the VISION COACH Full Field 60 task. Results showed no significant effect of age on FOD Advanced performance but a significant effect of age on VISION COACH performance. Correlations showed that participants' performance on both braking and E detection tasks were significantly positively correlated with performance on the VISION COACH (.37 < r < .40, p < .01). These tools provide new options for therapists. Copyright © 2017 by the American Occupational Therapy Association, Inc.

An experimental method for the assessment of color simulation tools.

PubMed

Lillo, Julio; Alvaro, Leticia; Moreira, Humberto

2014-07-22

The Simulcheck method for evaluating the accuracy of color simulation tools in relation to dichromats is described and used to test three color simulation tools: Variantor, Coblis, and Vischeck. A total of 10 dichromats (five protanopes, five deuteranopes) and 10 normal trichromats participated in the current study. Simulcheck includes two psychophysical tasks: the Pseudoachromatic Stimuli Identification task and the Minimum Achromatic Contrast task. The Pseudoachromatic Stimuli Identification task allows determination of the two chromatic angles (h(uv) values) that generate a minimum response in the yellow–blue opponent mechanism and, consequently, pseudoachromatic stimuli (greens or reds). The Minimum Achromatic Contrast task requires the selection of the gray background that produces minimum contrast (near zero change in the achromatic mechanism) for each pseudoachromatic stimulus selected in the previous task (L(R) values). Results showed important differences in the colorimetric transformations performed by the three evaluated simulation tools and their accuracy levels. Vischeck simulation accurately implemented the algorithm of Brettel, Viénot, and Mollon (1997). Only Vischeck appeared accurate (similarity in huv and L(R) values between real and simulated dichromats) and, consequently, could render reliable color selections. It is concluded that Simulcheck is a consistent method because it provided an equivalent pattern of results for huv and L(R) values irrespective of the stimulus set used to evaluate a simulation tool. Simulcheck was also considered valid because real dichromats provided expected huv and LR values when performing the two psychophysical tasks included in this method. © 2014 ARVO.

Figure Text Extraction in Biomedical Literature

PubMed Central

Kim, Daehyun; Yu, Hong

2011-01-01

Background Figures are ubiquitous in biomedical full-text articles, and they represent important biomedical knowledge. However, the sheer volume of biomedical publications has made it necessary to develop computational approaches for accessing figures. Therefore, we are developing the Biomedical Figure Search engine (http://figuresearch.askHERMES.org) to allow bioscientists to access figures efficiently. Since text frequently appears in figures, automatically extracting such text may assist the task of mining information from figures. Little research, however, has been conducted exploring text extraction from biomedical figures. Methodology We first evaluated an off-the-shelf Optical Character Recognition (OCR) tool on its ability to extract text from figures appearing in biomedical full-text articles. We then developed a Figure Text Extraction Tool (FigTExT) to improve the performance of the OCR tool for figure text extraction through the use of three innovative components: image preprocessing, character recognition, and text correction. We first developed image preprocessing to enhance image quality and to improve text localization. Then we adapted the off-the-shelf OCR tool on the improved text localization for character recognition. Finally, we developed and evaluated a novel text correction framework by taking advantage of figure-specific lexicons. Results/Conclusions The evaluation on 382 figures (9,643 figure texts in total) randomly selected from PubMed Central full-text articles shows that FigTExT performed with 84% precision, 98% recall, and 90% F1-score for text localization and with 62.5% precision, 51.0% recall and 56.2% F1-score for figure text extraction. When limiting figure texts to those judged by domain experts to be important content, FigTExT performed with 87.3% precision, 68.8% recall, and 77% F1-score. FigTExT significantly improved the performance of the off-the-shelf OCR tool we used, which on its own performed with 36.6% precision, 19.3% recall, and 25.3% F1-score for text extraction. In addition, our results show that FigTExT can extract texts that do not appear in figure captions or other associated text, further suggesting the potential utility of FigTExT for improving figure search. PMID:21249186

Figure text extraction in biomedical literature.

PubMed

Kim, Daehyun; Yu, Hong

2011-01-13

Figures are ubiquitous in biomedical full-text articles, and they represent important biomedical knowledge. However, the sheer volume of biomedical publications has made it necessary to develop computational approaches for accessing figures. Therefore, we are developing the Biomedical Figure Search engine (http://figuresearch.askHERMES.org) to allow bioscientists to access figures efficiently. Since text frequently appears in figures, automatically extracting such text may assist the task of mining information from figures. Little research, however, has been conducted exploring text extraction from biomedical figures. We first evaluated an off-the-shelf Optical Character Recognition (OCR) tool on its ability to extract text from figures appearing in biomedical full-text articles. We then developed a Figure Text Extraction Tool (FigTExT) to improve the performance of the OCR tool for figure text extraction through the use of three innovative components: image preprocessing, character recognition, and text correction. We first developed image preprocessing to enhance image quality and to improve text localization. Then we adapted the off-the-shelf OCR tool on the improved text localization for character recognition. Finally, we developed and evaluated a novel text correction framework by taking advantage of figure-specific lexicons. The evaluation on 382 figures (9,643 figure texts in total) randomly selected from PubMed Central full-text articles shows that FigTExT performed with 84% precision, 98% recall, and 90% F1-score for text localization and with 62.5% precision, 51.0% recall and 56.2% F1-score for figure text extraction. When limiting figure texts to those judged by domain experts to be important content, FigTExT performed with 87.3% precision, 68.8% recall, and 77% F1-score. FigTExT significantly improved the performance of the off-the-shelf OCR tool we used, which on its own performed with 36.6% precision, 19.3% recall, and 25.3% F1-score for text extraction. In addition, our results show that FigTExT can extract texts that do not appear in figure captions or other associated text, further suggesting the potential utility of FigTExT for improving figure search.

High-density fuel effects. Final report, September 1985-April 1988

DOE Office of Scientific and Technical Information (OSTI.GOV)

Rizk, N.K.; Oechsie, V.L.; Ross, P.T.

1988-08-18

The purpose of this program was to determine, by combustor rig tests and data evaluation, the effects of the high-density fuel properties on the performance and durability of the Allison T56-A-15 combustion system. Four high-density fuels in addition to baseline JP4 were evaluated in the effort. The rig-test program included: nozzle-flow bench testing, aerothermal performance and wall temperature, flame stability and ignition, injector coking and plugging, and flow-transient effect. The data-evaluation effort involved the utilization of empirical correlations in addition to analytical multidimensional tools to analyze the performance of the combustor. The modifications required to optimize the performance with high-densitymore » fuels were suggested and the expected improvement in performance was evaluated.« less

Building capacity for water, sanitation, and hygiene programming: Training evaluation theory applied to CLTS management training in Kenya.

PubMed

Crocker, Jonny; Shields, Katherine F; Venkataramanan, Vidya; Saywell, Darren; Bartram, Jamie

2016-10-01

Training and capacity building are long established critical components of global water, sanitation, and hygiene (WaSH) policies, strategies, and programs. Expanding capacity building support for WaSH in developing countries is one of the targets of the Sustainable Development Goals. There are many training evaluation methods and tools available. However, training evaluations in WaSH have been infrequent, have often not utilized these methods and tools, and have lacked rigor. We developed a conceptual framework for evaluating training in WaSH by reviewing and adapting concepts from literature. Our framework includes three target outcomes: learning, individual performance, and improved programming; and two sets of influences: trainee and context factors. We applied the framework to evaluate a seven-month community-led total sanitation (CLTS) management training program delivered to 42 government officials in Kenya from September 2013 to May 2014. Trainees were given a pre-training questionnaire and were interviewed at two weeks and seven months after initial training. We qualitatively analyzed the data using our conceptual framework. The training program resulted in trainees learning the CLTS process and new skills, and improving their individual performance through application of advocacy, partnership, and supervision soft skills. The link from trainees' performance to improved programming was constrained by resource limitations and pre-existing rigidity of trainees' organizations. Training-over-time enhanced outcomes and enabled trainees to overcome constraints in their work. Training in soft skills is relevant to managing public health programs beyond WaSH. We make recommendations on how training programs can be targeted and adapted to improve outcomes. Our conceptual framework can be used as a tool both for planning and evaluating training programs in WaSH. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

A Tool for Measuring NASA's Aeronautics Research Progress Toward Planned Strategic Community Outcomes

NASA Technical Reports Server (NTRS)

Tahmasebi, Farhad; Pearce, Robert

2016-01-01

Description of a tool for portfolio analysis of NASA's Aeronautics research progress toward planned community strategic Outcomes is presented. For efficiency and speed, the tool takes advantage of a function developed in Excels Visual Basic for Applications. The strategic planning process for determining the community Outcomes is also briefly discussed. Stakeholder buy-in, partnership performance, progress of supporting Technical Challenges, and enablement forecast are used as the criteria for evaluating progress toward Outcomes. A few illustrative examples of using the tool are also presented.

A Perspective on Computational Human Performance Models as Design Tools

NASA Technical Reports Server (NTRS)

Jones, Patricia M.

2010-01-01

The design of interactive systems, including levels of automation, displays, and controls, is usually based on design guidelines and iterative empirical prototyping. A complementary approach is to use computational human performance models to evaluate designs. An integrated strategy of model-based and empirical test and evaluation activities is particularly attractive as a methodology for verification and validation of human-rated systems for commercial space. This talk will review several computational human performance modeling approaches and their applicability to design of display and control requirements.

Evaluating Administrators with Portfolios: Principals Report Mostly Positive Experiences when Used as Part of a Performance Review

ERIC Educational Resources Information Center

Russo, Alexander

2004-01-01

Simple checklists, one-shot interviews, brief site visits and narrative evaluations remain widespread as the tools of assessment. In many school districts, the evaluation includes little or no face-to-face contact, and the principal simply gets his or her evaluation in the mail, leading one researcher to describe them as "infrequent, late,…

Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis.

PubMed

Li, Bo; Tang, Jing; Yang, Qingxia; Cui, Xuejiao; Li, Shuang; Chen, Sijie; Cao, Quanxing; Xue, Weiwei; Chen, Na; Zhu, Feng

2016-12-13

In untargeted metabolomics analysis, several factors (e.g., unwanted experimental &biological variations and technical errors) may hamper the identification of differential metabolic features, which requires the data-driven normalization approaches before feature selection. So far, ≥16 normalization methods have been widely applied for processing the LC/MS based metabolomics data. However, the performance and the sample size dependence of those methods have not yet been exhaustively compared and no online tool for comparatively and comprehensively evaluating the performance of all 16 normalization methods has been provided. In this study, a comprehensive comparison on these methods was conducted. As a result, 16 methods were categorized into three groups based on their normalization performances across various sample sizes. The VSN, the Log Transformation and the PQN were identified as methods of the best normalization performance, while the Contrast consistently underperformed across all sub-datasets of different benchmark data. Moreover, an interactive web tool comprehensively evaluating the performance of 16 methods specifically for normalizing LC/MS based metabolomics data was constructed and hosted at http://server.idrb.cqu.edu.cn/MetaPre/. In summary, this study could serve as a useful guidance to the selection of suitable normalization methods in analyzing the LC/MS based metabolomics data.

Performance Evaluation and Online Realization of Data-driven Normalization Methods Used in LC/MS based Untargeted Metabolomics Analysis

PubMed Central

Li, Bo; Tang, Jing; Yang, Qingxia; Cui, Xuejiao; Li, Shuang; Chen, Sijie; Cao, Quanxing; Xue, Weiwei; Chen, Na; Zhu, Feng

2016-01-01

In untargeted metabolomics analysis, several factors (e.g., unwanted experimental & biological variations and technical errors) may hamper the identification of differential metabolic features, which requires the data-driven normalization approaches before feature selection. So far, ≥16 normalization methods have been widely applied for processing the LC/MS based metabolomics data. However, the performance and the sample size dependence of those methods have not yet been exhaustively compared and no online tool for comparatively and comprehensively evaluating the performance of all 16 normalization methods has been provided. In this study, a comprehensive comparison on these methods was conducted. As a result, 16 methods were categorized into three groups based on their normalization performances across various sample sizes. The VSN, the Log Transformation and the PQN were identified as methods of the best normalization performance, while the Contrast consistently underperformed across all sub-datasets of different benchmark data. Moreover, an interactive web tool comprehensively evaluating the performance of 16 methods specifically for normalizing LC/MS based metabolomics data was constructed and hosted at http://server.idrb.cqu.edu.cn/MetaPre/. In summary, this study could serve as a useful guidance to the selection of suitable normalization methods in analyzing the LC/MS based metabolomics data. PMID:27958387

High-volume image quality assessment systems: tuning performance with an interactive data visualization tool

NASA Astrophysics Data System (ADS)

Bresnahan, Patricia A.; Pukinskis, Madeleine; Wiggins, Michael

1999-03-01

Image quality assessment systems differ greatly with respect to the number and types of mags they need to evaluate, and their overall architectures. Managers of these systems, however, all need to be able to tune and evaluate system performance, requirements often overlooked or under-designed during project planning. Performance tuning tools allow users to define acceptable quality standards for image features and attributes by adjusting parameter settings. Performance analysis tools allow users to evaluate and/or predict how well a system performs in a given parameter state. While image assessment algorithms are becoming quite sophisticated, duplicating or surpassing the human decision making process in their speed and reliability, they often require a greater investment in 'training' or fine tuning of parameters in order to achieve optimum performance. This process may involve the analysis of hundreds or thousands of images, generating a large database of files and statistics that can be difficult to sort through and interpret. Compounding the difficulty is the fact that personnel charged with tuning and maintaining the production system may not have the statistical or analytical background required for the task. Meanwhile, hardware innovations have greatly increased the volume of images that can be handled in a given time frame, magnifying the consequences of running a production site with an inadequately tuned system. In this paper, some general requirements for a performance evaluation and tuning data visualization system are discussed. A custom engineered solution to the tuning and evaluation problem is then presented, developed within the context of a high volume image quality assessment, data entry, OCR, and image archival system. A key factor influencing the design of the system was the context-dependent definition of image quality, as perceived by a human interpreter. This led to the development of a five-level, hierarchical approach to image quality evaluation. Lower-level pass-fail conditions and decision rules were coded into the system. Higher-level image quality states were defined by allowing the users to interactively adjust the system's sensitivity to various image attributes by manipulating graphical controls. Results were presented in easily interpreted bar graphs. These graphs were mouse- sensitive, allowing the user to more fully explore the subsets of data indicated by various color blocks. In order to simplify the performance evaluation and tuning process, users could choose to view the results of (1) the existing system parameter state, (2) the results of any arbitrary parameter values they chose, or (3) the results of a quasi-optimum parameter state, derived by applying a decision rule to a large set of possible parameter states. Giving managers easy- to-use tools for defining the more subjective aspects of quality resulted in a system that responded to contextual cues that are difficult to hard-code. It had the additional advantage of allowing the definition of quality to evolve over time, as users became more knowledgeable as to the strengths and limitations of an automated quality inspection system.

GROUND-WATER MODEL TESTING: SYSTEMATIC EVALUATION AND TESTING OF CODE FUNCTIONALITY AND PERFORMANCE

EPA Science Inventory

Effective use of ground-water simulation codes as management decision tools requires the establishment of their functionality, performance characteristics, and applicability to the problem at hand. This is accomplished through application of a systematic code-testing protocol and...

Interchangeable end effector tools utilized on the protoflight manipulator arm

NASA Technical Reports Server (NTRS)

1987-01-01

A subset of teleoperator and effector tools was designed, fabricated, delivered and successfully demonstrated on the Marshall Space Flight Center (MSFC) protoflight manipulator arm (PFMA). The tools delivered included a rotary power tool with interchangeable collets and two fluid coupling mate/demate tools; one for a Fairchild coupling and the other for a Purolator coupling. An electrical interface connector was also provided for the rotary power tool. A tool set, from which the subset was selected, for performing on-orbit satellite maintenance was identified and conceptionally designed. Maintenance requirements were synthesized, evaluated and prioritized to develop design requirements for a set of end effector tools representative of those needed to provide on-orbit maintenance of satellites to be flown in the 1986 to 2000 timeframe.

MetaCRAST: reference-guided extraction of CRISPR spacers from unassembled metagenomes.

PubMed

Moller, Abraham G; Liang, Chun

2017-01-01

Clustered regularly interspaced short palindromic repeat (CRISPR) systems are the adaptive immune systems of bacteria and archaea against viral infection. While CRISPRs have been exploited as a tool for genetic engineering, their spacer sequences can also provide valuable insights into microbial ecology by linking environmental viruses to their microbial hosts. Despite this importance, metagenomic CRISPR detection remains a major challenge. Here we present a reference-guided CRISPR spacer detection tool ( Meta genomic C RISPR R eference- A ided S earch T ool-MetaCRAST) that constrains searches based on user-specified direct repeats (DRs). These DRs could be expected from assembly or taxonomic profiles of metagenomes. We compared the performance of MetaCRAST to those of two existing metagenomic CRISPR detection tools-Crass and MinCED-using both real and simulated acid mine drainage (AMD) and enhanced biological phosphorus removal (EBPR) metagenomes. Our evaluation shows MetaCRAST improves CRISPR spacer detection in real metagenomes compared to the de novo CRISPR detection methods Crass and MinCED. Evaluation on simulated metagenomes show it performs better than de novo tools for Illumina metagenomes and comparably for 454 metagenomes. It also has comparable performance dependence on read length and community composition, run time, and accuracy to these tools. MetaCRAST is implemented in Perl, parallelizable through the Many Core Engine (MCE), and takes metagenomic sequence reads and direct repeat queries (FASTA or FASTQ) as input. It is freely available for download at https://github.com/molleraj/MetaCRAST.

Evaluation as a critical factor of success in local public health accreditation programs.

PubMed

Tremain, Beverly; Davis, Mary; Joly, Brenda; Edgar, Mark; Kushion, Mary L; Schmidt, Rita

2007-01-01

This article presents the variety of approaches used to conduct evaluations of performance improvement or accreditation systems, while illustrating the complexity of conducting evaluations to inform local public health practice. We, in addition, hope to inform the Exploring Accreditation Program about relevant experiences involving accreditation and performance assessment processes, specifically evaluation, as it debates and discusses a national voluntary model. A background of each state is given. To further explore these issues, interviews were conducted with each state's evaluator to gain more in-depth information on the many different evaluation strategies and approaches used. On the basis of the interviews, the authors provide several overall themes, which suggest that evaluation is a critical tool and success factor for performance assessment or accreditation programs.

[Performance assessment employing slide sets: A tool for the classification of senior microscopists from Colombia´s Malaria Control Program].

PubMed

Mendoza, Nohora Marcela; González, Nohora Elizabeth

2015-01-01

One of the most important activities for quality assurance of malaria diagnosis is performance assessment. In Colombia, performance assessment of malaria microscopists has been done through the external performance assessment and indirect external performance assessment programs. To assess the performance of malaria microscopists of public reference laboratories using slide sets, and to describe the methodology used for this purpose. This was a retrospective study to evaluate the concordance of senior microscopists regarding parasite detection, species identification and parasite count based on the results of the assessment of competences using two sets, one comprising 40 slides, and another one with 17 slides. The concordance for parasite detection was 96.9% (95% CI: 96.0-97.5) and 88.7% (95% CI: 86.6-90.5) for species identification. The average percentage of concordant slides in the group evaluated was 89.7% (95% CI: 87.5-91.6). Most of the senior microscopists in Colombia were classified in the two top categories in the performance assessment using slide sets. The most common difficulty encountered was the identification of parasite species. The use of this tool to assess individual performance of microscopists in the evaluation of samples with different degrees of difficulty allows for characterizing the members of the malaria diagnosis network and strengthening the abilities of those who require it.

Risk assessment tools to identify women with increased risk of osteoporotic fracture: complexity or simplicity? A systematic review.

PubMed

Rubin, Katrine Hass; Friis-Holmberg, Teresa; Hermann, Anne Pernille; Abrahamsen, Bo; Brixen, Kim

2013-08-01

A huge number of risk assessment tools have been developed. Far from all have been validated in external studies, more of them have absence of methodological and transparent evidence, and few are integrated in national guidelines. Therefore, we performed a systematic review to provide an overview of existing valid and reliable risk assessment tools for prediction of osteoporotic fractures. Additionally, we aimed to determine if the performance of each tool was sufficient for practical use, and last, to examine whether the complexity of the tools influenced their discriminative power. We searched PubMed, Embase, and Cochrane databases for papers and evaluated these with respect to methodological quality using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS) checklist. A total of 48 tools were identified; 20 had been externally validated, however, only six tools had been tested more than once in a population-based setting with acceptable methodological quality. None of the tools performed consistently better than the others and simple tools (i.e., the Osteoporosis Self-assessment Tool [OST], Osteoporosis Risk Assessment Instrument [ORAI], and Garvan Fracture Risk Calculator [Garvan]) often did as well or better than more complex tools (i.e., Simple Calculated Risk Estimation Score [SCORE], WHO Fracture Risk Assessment Tool [FRAX], and Qfracture). No studies determined the effectiveness of tools in selecting patients for therapy and thus improving fracture outcomes. High-quality studies in randomized design with population-based cohorts with different case mixes are needed. Copyright © 2013 American Society for Bone and Mineral Research.

Decision support tools for proton therapy ePR: intelligent treatment planning navigator and radiation toxicity tool for evaluating of prostate cancer treatment

NASA Astrophysics Data System (ADS)

Le, Anh H.; Deshpande, Ruchi; Liu, Brent J.

2010-03-01

The electronic patient record (ePR) has been developed for prostate cancer patients treated with proton therapy. The ePR has functionality to accept digital input from patient data, perform outcome analysis and patient and physician profiling, provide clinical decision support and suggest courses of treatment, and distribute information across different platforms and health information systems. In previous years, we have presented the infrastructure of a medical imaging informatics based ePR for PT with functionality to accept digital patient information and distribute this information across geographical location using Internet protocol. In this paper, we present the ePR decision support tools which utilize the imaging processing tools and data collected in the ePR. The two decision support tools including the treatment plan navigator and radiation toxicity tool are presented to evaluate prostate cancer treatment to improve proton therapy operation and improve treatment outcomes analysis.

The Impact of Providing Performance Feedback to Teachers and Principals. NCEE 2018-4001

ERIC Educational Resources Information Center

Garet, Michael S.; Wayne, Andrew J.; Brown, Seth; Rickles, Jordan; Song, Mengli; Manzeske, David

2017-01-01

Educator performance evaluation systems are a potential tool for improving student achievement by increasing the effectiveness of the educator workforce. For example, recent research suggests that giving more frequent, specific feedback on classroom practice may lead to improvements in teacher performance and student achievement. This report is…

The Impact of Providing Performance Feedback to Teachers and Principals. Executive Summary. NCEE 2018-4000

ERIC Educational Resources Information Center

Garet, Michael S.; Wayne, Andrew J.; Brown, Seth; Rickles, Jordan; Song, Mengli; Manzeske, David

2017-01-01

Educator performance evaluation systems are a potential tool for improving student achievement by increasing the effectiveness of the educator workforce. For example, recent research suggests that giving more frequent, specific feedback on classroom practice may lead to improvements in teacher performance and student achievement. This report is…

Exploring the Usage of a Video Application Tool: Experiences in Film Studies

ERIC Educational Resources Information Center

Ali, Nazlena Mohamad; Smeaton, Alan F.

2011-01-01

This paper explores our experiences in deploying a video application tool in film studies, and its evaluation in terms of realistic contextual end-users who have real tasks to perform in a real environment. We demonstrate our experiences and core lesson learnt in deploying our novel movie browser application with undergraduate and graduate…

Machine Tool Series. Duty Task List.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater. Curriculum and Instructional Materials Center.

This task list is intended for use in planning and/or evaluating a competency-based course to prepare machine tool, drill press, grinding machine, lathe, mill, and/or power saw operators. The listing is divided into six sections, with each one outlining the tasks required to perform the duties that have been identified for the given occupation.…

The value of SPaCE in delivering patient feedback.

PubMed

Clapham, Laura; Allan, Laura; Stirling, Kevin

2016-02-01

The use of simulated patients (SPs) within undergraduate medical curricula is an established and valued learning opportunity. Within the context of simulation, it is imperative to capture feedback from all participants within the simulation activity. The Simulated Patient Candidate Evaluation (SPaCE) tool was developed to deliver SP feedback following a simulation activity. SpaCE is a closed feedback tool that allows SPs to rate a student's performance, using a five-point Likert scale, in three domains: attitude; interaction skills; and management. This research study examined the value of the SPaCE tool and how it contributes to the overall feedback that a student receives. Classical test theory was used to determine the reliability of the SPaCE tool. An evaluation of all SP responses was conducted to observe trends in scoring patterns for each question. Qualitative data were collected via a free-text questionnaire and subsequent focus group discussion. It is imperative to capture feedback from all participants within the simulation activity Classical test theory determined that the SPaCE tool had a reliability co-efficient of 0.89. A total of 13 SPs replied to the questionnaire. A thematic analysis of all questionnaire data identified that the SPaCE tool provides a structure that allows patient feedback to be given effectively following a simulation activity. These themes were discussed further with six SPs who attended the subsequent focus group session. The SPaCE tool has been shown to be a reliable closed feedback tool that allows SPs to discriminate between students, based on their performance. The next stage in the development of the SPaCE tool is to test the wider applicability of this feedback tool. © 2015 John Wiley & Sons Ltd.

Application of structured analysis to a telerobotic system

NASA Technical Reports Server (NTRS)

Dashman, Eric; Mclin, David; Harrison, F. W.; Soloway, Donald; Young, Steven

1990-01-01

The analysis and evaluation of a multiple arm telerobotic research and demonstration system developed by the NASA Intelligent Systems Research Laboratory (ISRL) is described. Structured analysis techniques were used to develop a detailed requirements model of an existing telerobotic testbed. Performance models generated during this process were used to further evaluate the total system. A commercial CASE tool called Teamwork was used to carry out the structured analysis and development of the functional requirements model. A structured analysis and design process using the ISRL telerobotic system as a model is described. Evaluation of this system focused on the identification of bottlenecks in this implementation. The results demonstrate that the use of structured methods and analysis tools can give useful performance information early in a design cycle. This information can be used to ensure that the proposed system meets its design requirements before it is built.

WRF/CMAQ AQMEII3 Simulations of US Regional-Scale ...

EPA Pesticide Factsheets

Chemical boundary conditions are a key input to regional-scale photochemical models. In this study, performed during the third phase of the Air Quality Model Evaluation International Initiative (AQMEII3), we perform annual simulations over North America with chemical boundary conditions prepared from four different global models. Results indicate that the impacts of different boundary conditions are significant for ozone throughout the year and most pronounced outside the summer season. The National Exposure Research Laboratory (NERL) Computational Exposure Division (CED) develops and evaluates data, decision-support tools, and models to be applied to media-specific or receptor-specific problem areas. CED uses modeling-based approaches to characterize exposures, evaluate fate and transport, and support environmental diagnostics/forensics with input from multiple data sources. It also develops media- and receptor-specific models, process models, and decision support tools for use both within and outside of EPA.

Evaluating biomarkers for prognostic enrichment of clinical trials.

PubMed

Kerr, Kathleen F; Roth, Jeremy; Zhu, Kehao; Thiessen-Philbrook, Heather; Meisner, Allison; Wilson, Francis Perry; Coca, Steven; Parikh, Chirag R

2017-12-01

A potential use of biomarkers is to assist in prognostic enrichment of clinical trials, where only patients at relatively higher risk for an outcome of interest are eligible for the trial. We investigated methods for evaluating biomarkers for prognostic enrichment. We identified five key considerations when considering a biomarker and a screening threshold for prognostic enrichment: (1) clinical trial sample size, (2) calendar time to enroll the trial, (3) total patient screening costs and the total per-patient trial costs, (4) generalizability of trial results, and (5) ethical evaluation of trial eligibility criteria. Items (1)-(3) are amenable to quantitative analysis. We developed the Biomarker Prognostic Enrichment Tool for evaluating biomarkers for prognostic enrichment at varying levels of screening stringency. We demonstrate that both modestly prognostic and strongly prognostic biomarkers can improve trial metrics using Biomarker Prognostic Enrichment Tool. Biomarker Prognostic Enrichment Tool is available as a webtool at http://prognosticenrichment.com and as a package for the R statistical computing platform. In some clinical settings, even biomarkers with modest prognostic performance can be useful for prognostic enrichment. In addition to the quantitative analysis provided by Biomarker Prognostic Enrichment Tool, investigators must consider the generalizability of trial results and evaluate the ethics of trial eligibility criteria.

Assessment of tools for modeling aircraft noise in national parks

DOT National Transportation Integrated Search

2005-03-18

The first objective of this study was to evaluate the series of model enhancements that were included in : INM as a result of the recommendations from the GCNP MVS. Specifically, there was a desire to : evaluate the performance of the latest versions...

COMPUTERIZED NEEDS-ORIENTED QUALITY MEASUREMENT EVALUATION SYSTEM (CONQUEST)

EPA Science Inventory

CONQUEST is an easy-to-use quality improvement software tool that uses a common structure and language to help users identity, understand, compare, evaluate, and select among 1,200 clinical performance measures that can be used to assess and improve quality of care. CONQUEST's in...

Surgeon-tool force/torque signatures--evaluation of surgical skills in minimally invasive surgery.

PubMed

Rosen, J; MacFarlane, M; Richards, C; Hannaford, B; Sinanan, M

1999-01-01

The best method of training for laparoscopic surgical skills is controversial. Some advocate observation in the operating room, while others promote animal and simulated models or a combination of surgical related tasks. The mode of proficiency evaluation common to all of these methods has been subjective evaluation by a skilled surgeon. In order to define an objective means of evaluating performance, an instrumented laparoscopic grasper was developed measuring the force/torque at the surgeon hand/tool interface. The measured database demonstrated substantial differences between experienced and novice surgeon groups. Analyzing forces and torques combined with the state transition during surgical procedures allows an objective measurement of skill in MIS. Teaching the novice surgeon to limit excessive loads and improve movement efficiency during surgical procedures can potentially result in less injury to soft tissues and less wasted time during laparoscopic surgery. Moreover the force/torque database measured in this study may be used for developing realistic virtual reality simulators and optimization of medical robots performance.

An integrated evaluation for the performance of clinical engineering department.

PubMed

Yousry, Ahmed M; Ouda, Bassem K; Eldeib, Ayman M

2014-01-01

Performance benchmarking have become a very important component in all successful organizations nowadays that must be used by Clinical Engineering Department (CED) in hospitals. Many researchers identified essential mainstream performance indicators needed to improve the CED's performance. These studies revealed mainstream performance indicators that use the database of a CED to evaluate its performance. In this work, we believe that those indicators are insufficient for hospitals. Additional important indicators should be included to improve the evaluation accuracy. Therefore, we added new indicators: technical/maintenance indicators, economic indicators, intrinsic criticality indicators, basic hospital indicators, equipment acquisition, and safety indicators. Data is collected from 10 hospitals that cover different types of healthcare organizations. We developed a software tool that analyses collected data to provide a score for each CED under evaluation. Our results indicate that there is an average gap of 67% between the CEDs' performance and the ideal target. The reasons for the noncompliance are discussed in order to improve performance of CEDs under evaluation.

Summary

DOE Office of Scientific and Technical Information (OSTI.GOV)

Powell, Danny H; Elwood Jr, Robert H

2011-01-01

An effective risk assessment system is needed to address the threat posed by an active or passive insider who, acting alone or in collusion, could attempt diversion or theft of nuclear material. The material control and accountability (MC&A) system effectiveness tool (MSET) is a self-assessment or inspection tool utilizing probabilistic risk assessment (PRA) methodology to calculate the system effectiveness of a nuclear facility's material protection, control, and accountability (MPC&A) system. The MSET process is divided into four distinct and separate parts: (1) Completion of the questionnaire that assembles information about the operations of every aspect of the MPC&A system; (2)more » Conversion of questionnaire data into numeric values associated with risk; (3) Analysis of the numeric data utilizing the MPC&A fault tree and the SAPHIRE computer software; and (4) Self-assessment using the MSET reports to perform the effectiveness evaluation of the facility's MPC&A system. The process should lead to confirmation that mitigating features of the system effectively minimize the threat, or it could lead to the conclusion that system improvements or upgrades are necessary to achieve acceptable protection against the threat. If the need for system improvements or upgrades is indicated when the system is analyzed, MSET provides the capability to evaluate potential or actual system improvements or upgrades. A facility's MC&A system can be evaluated at a point in time. The system can be reevaluated after upgrades are implemented or after other system changes occur. The total system or specific subareas within the system can be evaluated. Areas of potential system improvement can be assessed to determine where the most beneficial and cost-effective improvements should be made. Analyses of risk importance factors show that sustainability is essential for optimal performance and reveals where performance degradation has the greatest impact on total system risk. The risk importance factors show the amount of risk reduction achievable with potential upgrades and the amount of risk reduction achieved after upgrades are completed. Applying the risk assessment tool gives support to budget prioritization by showing where budget support levels must be sustained for MC&A functions most important to risk. Results of the risk assessment are also useful in supporting funding justifications for system improvements that significantly reduce system risk. The functional model, the system risk assessment tool, and the facility evaluation questionnaire are valuable educational tools for MPC&A personnel. These educational tools provide a framework for ongoing dialogue between organizations regarding the design, development, implementation, operation, assessment, and sustainability of MPC&A systems. An organization considering the use of MSET as an analytical tool for evaluating the effectiveness of its MPC&A system will benefit from conducting a complete MSET exercise at an existing nuclear facility.« less

Verification and Validation Methodology of Real-Time Adaptive Neural Networks for Aerospace Applications

NASA Technical Reports Server (NTRS)

Gupta, Pramod; Loparo, Kenneth; Mackall, Dale; Schumann, Johann; Soares, Fola

2004-01-01

Recent research has shown that adaptive neural based control systems are very effective in restoring stability and control of an aircraft in the presence of damage or failures. The application of an adaptive neural network with a flight critical control system requires a thorough and proven process to ensure safe and proper flight operation. Unique testing tools have been developed as part of a process to perform verification and validation (V&V) of real time adaptive neural networks used in recent adaptive flight control system, to evaluate the performance of the on line trained neural networks. The tools will help in certification from FAA and will help in the successful deployment of neural network based adaptive controllers in safety-critical applications. The process to perform verification and validation is evaluated against a typical neural adaptive controller and the results are discussed.

Playbook Data Analysis Tool: Collecting Interaction Data from Extremely Remote Users

NASA Technical Reports Server (NTRS)

Kanefsky, Bob; Zheng, Jimin; Deliz, Ivonne; Marquez, Jessica J.; Hillenius, Steven

2017-01-01

Typically, user tests for software tools are conducted in person. At NASA, the users may be located at the bottom of the ocean in a pressurized habitat, above the atmosphere in the International Space Station, or in an isolated capsule on a simulated asteroid mission. The Playbook Data Analysis Tool (P-DAT) is a human-computer interaction (HCI) evaluation tool that the NASA Ames HCI Group has developed to record user interactions with Playbook, the group's existing planning-and-execution software application. Once the remotely collected user interaction data makes its way back to Earth, researchers can use P-DAT for in-depth analysis. Since a critical component of the Playbook project is to understand how to develop more intuitive software tools for astronauts to plan in space, P-DAT helps guide us in the development of additional easy-to-use features for Playbook, informing the design of future crew autonomy tools.P-DAT has demonstrated the capability of discreetly capturing usability data in amanner that is transparent to Playbook’s end-users. In our experience, P-DAT data hasalready shown its utility, revealing potential usability patterns, helping diagnose softwarebugs, and identifying metrics and events that are pertinent to Playbook usage aswell as spaceflight operations. As we continue to develop this analysis tool, P-DATmay yet provide a method for long-duration, unobtrusive human performance collectionand evaluation for mission controllers back on Earth and researchers investigatingthe effects and mitigations related to future human spaceflight performance.

Relating MBSE to Spacecraft Development: A NASA Pathfinder

NASA Technical Reports Server (NTRS)

Othon, Bill

2016-01-01

The NASA Engineering and Safety Center (NESC) has sponsored a Pathfinder Study to investigate how Model Based Systems Engineering (MBSE) and Model Based Engineering (MBE) techniques can be applied by NASA spacecraft development projects. The objectives of this Pathfinder Study included analyzing both the products of the modeling activity, as well as the process and tool chain through which the spacecraft design activities are executed. Several aspects of MBSE methodology and process were explored. Adoption and consistent use of the MBSE methodology within an existing development environment can be difficult. The Pathfinder Team evaluated the possibility that an "MBSE Template" could be developed as both a teaching tool as well as a baseline from which future NASA projects could leverage. Elements of this template include spacecraft system component libraries, data dictionaries and ontology specifications, as well as software services that do work on the models themselves. The Pathfinder Study also evaluated the tool chain aspects of development. Two chains were considered: 1. The Development tool chain, through which SysML model development was performed and controlled, and 2. The Analysis tool chain, through which both static and dynamic system analysis is performed. Of particular interest was the ability to exchange data between SysML and other engineering tools such as CAD and Dynamic Simulation tools. For this study, the team selected a Mars Lander vehicle as the element to be designed. The paper will discuss what system models were developed, how data was captured and exchanged, and what analyses were conducted.

Evaluating the impact of a pre-rotation workshop on student preparation for clinical advanced pharmacy practice experiences.

PubMed

Medina, Melissa S; Stark, Jennifer E; Vesta, Kimi S; Lockhart, Staci M

2008-10-01

This pilot study was designed to evaluate the impact of a pre-rotation workshop (PRW) on pharmacy students' clinical skills and preparation for clinical Advanced Pharmacy Practice Experiences (APPE) involving direct patient care. Randomized controlled trial of an educational intervention with Institutional Review Board approval. PRW activities designed to simulate rotation activities around five competencies, patient charts, medication histories, SOAP notes, patient presentations, and professionalism. Endpoints were evaluated using clinical rotation preceptors' evaluation of performance and students' performance on objective structured clinical exams (OSCE). Eight fourth-year students and eight GPA matched controls (20% of the total class) were selected to voluntarily participate. The PRW demonstrated a positive impact on students' clinical skills and preparation for rotations by improving OSCE performance. However, no significant differences were found between groups when comparing preceptor evaluations of skills on rotations. These results are limited by the small sample size, potential OSCE "test-wiseness" effects, lack of OSCE evaluator blinding to study groups, potential case specificity effects due to the limited number of cases used on the OSCE and possible lack of sensitivity of the rotation evaluation tool to capture true differences among the experimental and control group participants. The PRW was successful at advancing students' clinical skills and preparation for rotations and may be considered as a tool to help bridge didactic to clinical experiences in the Pharm.D. curriculum.

Evaluation of the XSENS Force Shoe on ISS

NASA Technical Reports Server (NTRS)

Hanson, A. M.; Peters, B. T.; Newby, N.; Ploutz-Snyder, L

2014-01-01

The Advanced Resistive Exercise Device (ARED) offers crewmembers a wide range of resistance exercises but does not provide any type of load monitoring; any load data received are based on crew self-report of dialed in load. This lack of real-time ARED load monitoring severely limits research analysis. To address this issue, portable load monitoring technologies are being evaluated to act as a surrogate to ARED's failed instrumentation. The XSENS ForceShoe"TM" is a commercial portable load monitoring tool, and performed well in ground tests. The ForceShoe "TM" was recently deployed on the International Space Station (ISS), and is being evaluated as a tool to monitor ARED loads.

An Evaluation of the Performance Diagnostic Checklist-Human Services to Assess an Employee Performance Problem in a Center-Based Autism Treatment Facility

ERIC Educational Resources Information Center

Ditzian, Kyle; Wilder, David A.; King, Allison; Tanz, Jeanine

2015-01-01

The Performance Diagnostic Checklist-Human Services (PDC-HS) is an informant-based tool designed to assess the environmental variables that contribute to poor employee performance in human services settings. We administered the PDC-HS to 3 supervisors to assess the variables that contributed to poor performance by 4 staff members when securing…

Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians' Professional Performance.

PubMed

van der Meulen, Mirja W; Boerebach, Benjamin C M; Smirnova, Alina; Heeneman, Sylvia; Oude Egbrink, Mirjam G A; van der Vleuten, Cees P M; Arah, Onyebuchi A; Lombarts, Kiki M J M H

2017-01-01

Multisource feedback (MSF) instruments are used to and must feasibly provide reliable and valid data on physicians' performance from multiple perspectives. The "INviting Co-workers to Evaluate Physicians Tool" (INCEPT) is a multisource feedback instrument used to evaluate physicians' professional performance as perceived by peers, residents, and coworkers. In this study, we report on the validity, reliability, and feasibility of the INCEPT. The performance of 218 physicians was assessed by 597 peers, 344 residents, and 822 coworkers. Using explorative and confirmatory factor analyses, multilevel regression analyses between narrative and numerical feedback, item-total correlations, interscale correlations, Cronbach's α and generalizability analyses, the psychometric qualities, and feasibility of the INCEPT were investigated. For all respondent groups, three factors were identified, although constructed slightly different: "professional attitude," "patient-centeredness," and "organization and (self)-management." Internal consistency was high for all constructs (Cronbach's α ≥ 0.84 and item-total correlations ≥ 0.52). Confirmatory factor analyses indicated acceptable to good fit. Further validity evidence was given by the associations between narrative and numerical feedback. For reliable total INCEPT scores, three peer, two resident and three coworker evaluations were needed; for subscale scores, evaluations of three peers, three residents and three to four coworkers were sufficient. The INCEPT instrument provides physicians performance feedback in a valid and reliable way. The number of evaluations to establish reliable scores is achievable in a regular clinical department. When interpreting feedback, physicians should consider that respondent groups' perceptions differ as indicated by the different item clustering per performance factor.

Investigation, quantification, and recommendations : performance of alternatively fueled buses.

DOT National Transportation Integrated Search

2014-08-01

The goal of this project was to continue consistent collection and reporting of data on the performance and costs of alternatively fueled public transit vehicles in the U.S. transit fleet in order to keep the Bus Fuels Fleet Evaluation Tool (BuFFeT; ...

Development of field performance evaluation tools and program for pavement marking materials : technical report

DOT National Transportation Integrated Search

2011-03-01

Historically the prequalification or selection of pavement marking materials (PMMs) is mainly based on : product specifications and lab testing, which do not correlate well with the field performance of the products. : On the other hand, there is no ...

ESH assessment of advanced lithography materials and processes

NASA Astrophysics Data System (ADS)

Worth, Walter F.; Mallela, Ram

2004-05-01

The ESH Technology group at International SEMATECH is conducting environment, safety, and health (ESH) assessments in collaboration with the lithography technologists evaluating the performance of an increasing number of new materials and technologies being considered for advanced lithography such as 157nm photresist and extreme ultraviolet (EUV). By performing data searches for 75 critical data types, emissions characterizations, and industrial hygiene (IH) monitoring during the use of the resist candidates, it has been shown that the best performing resist formulations, so far, appear to be free of potential ESH concerns. The ESH assessment of the EUV lithography tool that is being developed for SEMATECH has identified several features of the tool that are of ESH concern: high energy consumption, poor energy conversion efficiency, tool complexity, potential ergonomic and safety interlock issues, use of high powered laser(s), generation of ionizing radiation (soft X-rays), need for adequate shielding, and characterization of the debris formed by the extreme temperature of the plasma. By bringing these ESH challenges to the attention of the technologists and tool designers, it is hoped that the processes and tools can be made more ESH friendly.

A tool for hearing aid and cochlear implant users to judge the usability of cellular telephones in field conditions

NASA Astrophysics Data System (ADS)

Deer, Maria Soledad

The auditory experience of using a hearing aid or a cochlear implant simultaneously with a cell phone is driven by a number of factors. These factors are: radiofrequency and baseband interference, speech intelligibility, sound quality, handset design, volume control and signal strength. The purpose of this study was to develop a tool to be used by hearing aid and cochlear implant users in retail stores as they try cell phones before buying them. This tool is meant to be an efficient, practical and systematic consumer selection tool that will capture and document information on all the domains that play a role in the auditory experience of using a cell phone with a hearing aid or cochlear implant. The development of this consumer tool involved three steps as follows: preparation, verification and measurement of success according to a predefined criterion. First, the consumer tool, consisting of a comparison chart and speech material, was prepared. Second, the consumer tool was evaluated by groups of subjects in a two-step verification process. Phase I was conducted in a controlled setting and it was followed by Phase II which took place in real world (field) conditions. In order to perform a systematic evaluation of the consumer tool two questionnaires were developed: one questionnaire for each phase. Both questionnaires involved five quantitative variables scored with the use of ratings scales. These ratings were averaged yielding an Overall Consumer Performance Score. A qualitative performance category corresponding to the Mean Opinion Score (MOS) was allocated to each final score within a scale ranging from 1 to 5 (where 5 = excellent and 1 = bad). Finally, the consumer tool development was determined to be successful if at least 80% of the participants in verification Phase II rated the comparison chart as excellent or good according to the qualitative MOS score. The results for verification Phase II (field conditions) indicated that the Overall Consumer Performance score for 92% of the subjects (11/12) was 3.7 and above corresponding to Good and Excellent MOS qualitative categories. It was concluded that this is a practical and efficient tool for hearing aid/cochlear implant users as they approach a cell phone selection process.

The use of high-fidelity human patient simulation as an evaluative tool in the development of clinical research protocols and procedures.

PubMed

Wright, Melanie C; Taekman, Jeffrey M; Barber, Linda; Hobbs, Gene; Newman, Mark F; Stafford-Smith, Mark

2005-12-01

Errors in clinical research can be costly, in terms of patient safety, data integrity, and data collection. Data inaccuracy in early subjects of a clinical study may be associated with problems in the design of the protocol, procedures, and data collection tools. High-fidelity patient simulation centers provide an ideal environment to apply human-centered design to clinical trial development. A draft of a complex clinical protocol was designed, evaluated and modified using a high-fidelity human patient simulator in the Duke University Human Simulation and Patient Safety Center. The process included walk-throughs, detailed modifications of the protocol and development of procedural aids. Training of monitors and coordinators provided an opportunity for observation of performance that was used to identify further improvements to the protocol. Evaluative steps were used to design the research protocol and procedures. Iterative modifications were made to the protocol and data collection tools. The success in use of human simulation in the preparation of a complex clinical drug trial suggests the benefits of human patient simulation extend beyond training and medical equipment evaluation. Human patient simulation can provide a context for informal expert evaluation of clinical protocol design and for formal "rehearsal" to evaluate the efficacy of procedures and support tools.

Assessing resident's knowledge and communication skills using four different evaluation tools.

PubMed

Nuovo, Jim; Bertakis, Klea D; Azari, Rahman

2006-07-01

This study assesses the relationship between 4 Accreditation Council for Graduate Medical Education (ACGME) outcome project measures for interpersonal and communication skills and medical knowledge; specifically, monthly performance evaluations, objective structured clinical examinations (OSCEs), the American Board of Family Practice in-training examination (ABFP-ITE) and the Davis observation code (DOC) practice style profiles. Based on previous work, we have DOC scoring for 29 residents from the University of California, Davis Department of Family and Community Medicine. For all these residents we also had the results of monthly performance evaluations, 2 required OSCE exercises, and the results of 3 American Board of Family Medicine (ABFM) ITEs. Data for each of these measures were abstracted for each resident. The Pearson correlation coefficient was used to assess the presence or lack of correlation between each of these evaluation methods. There is little correlation between various evaluation methods used to assess medical knowledge, and there is also little correlation between various evaluation methods used to assess communication skills. The outcome project remains a 'work in progress', with the need for larger studies to assess the value of different assessment measures of resident competence. It is unlikely that DOC will become a useful evaluation tool.

Review of nutritional screening and assessment tools and clinical outcomes in heart failure.

PubMed

Lin, Hong; Zhang, Haifeng; Lin, Zheng; Li, Xinli; Kong, Xiangqin; Sun, Gouzhen

2016-09-01

Recent studies have suggested that undernutrition as defined using multidimensional nutritional evaluation tools may affect clinical outcomes in heart failure (HF). The evidence supporting this correlation is unclear. Therefore, we conducted this systematic review to critically appraise the use of multidimensional evaluation tools in the prediction of clinical outcomes in HF. We performed descriptive analyses of all identified articles involving qualitative analyses. We used STATA to conduct meta-analyses when at least three studies that tested the same type of nutritional assessment or screening tools and used the same outcome were identified. Sensitivity analyses were conducted to validate our positive results. We identified 17 articles with qualitative analyses and 11 with quantitative analysis after comprehensive literature searching and screening. We determined that the prevalence of malnutrition is high in HF (range 16-90 %), particularly in advanced and acute decompensated HF (approximate range 75-90 %). Undernutrition as identified by multidimensional evaluation tools may be significantly associated with hospitalization, length of stay and complications and is particularly strongly associated with high mortality. The meta-analysis revealed that compared with other tools, Mini Nutritional Assessment (MNA) scores were the strongest predictors of mortality in HF [HR (4.32, 95 % CI 2.30-8.11)]. Our results remained reliable after conducting sensitivity analyses. The prevalence of malnutrition is high in HF, particularly in advanced and acute decompensated HF. Moreover, undernutrition as identified by multidimensional evaluation tools is significantly associated with unfavourable prognoses and high mortality in HF.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Salama, A.; Mikhail, M.

Comprehensive software packages have been developed at the Western Research Centre as tools to help coal preparation engineers analyze, evaluate, and control coal cleaning processes. The COal Preparation Software package (COPS) performs three functions: (1) data handling and manipulation, (2) data analysis, including the generation of washability data, performance evaluation and prediction, density and size modeling, evaluation of density and size partition characteristics and attrition curves, and (3) generation of graphics output. The Separation ChARacteristics Estimation software packages (SCARE) are developed to balance raw density or size separation data. The cases of density and size separation data are considered. Themore » generated balanced data can take the balanced or normalized forms. The scaled form is desirable for direct determination of the partition functions (curves). The raw and generated separation data are displayed in tabular and/or graphical forms. The computer softwares described in this paper are valuable tools for coal preparation plant engineers and operators for evaluating process performance, adjusting plant parameters, and balancing raw density or size separation data. These packages have been applied very successfully in many projects carried out by WRC for the Canadian coal preparation industry. The software packages are designed to run on a personal computer (PC).« less

Analysis of key technologies for virtual instruments metrology

NASA Astrophysics Data System (ADS)

Liu, Guixiong; Xu, Qingui; Gao, Furong; Guan, Qiuju; Fang, Qiang

2008-12-01

Virtual instruments (VIs) require metrological verification when applied as measuring instruments. Owing to the software-centered architecture, metrological evaluation of VIs includes two aspects: measurement functions and software characteristics. Complexity of software imposes difficulties on metrological testing of VIs. Key approaches and technologies for metrology evaluation of virtual instruments are investigated and analyzed in this paper. The principal issue is evaluation of measurement uncertainty. The nature and regularity of measurement uncertainty caused by software and algorithms can be evaluated by modeling, simulation, analysis, testing and statistics with support of powerful computing capability of PC. Another concern is evaluation of software features like correctness, reliability, stability, security and real-time of VIs. Technologies from software engineering, software testing and computer security domain can be used for these purposes. For example, a variety of black-box testing, white-box testing and modeling approaches can be used to evaluate the reliability of modules, components, applications and the whole VI software. The security of a VI can be assessed by methods like vulnerability scanning and penetration analysis. In order to facilitate metrology institutions to perform metrological verification of VIs efficiently, an automatic metrological tool for the above validation is essential. Based on technologies of numerical simulation, software testing and system benchmarking, a framework for the automatic tool is proposed in this paper. Investigation on implementation of existing automatic tools that perform calculation of measurement uncertainty, software testing and security assessment demonstrates the feasibility of the automatic framework advanced.

Designing workload analysis questionnaire to evaluate needs of employees

NASA Astrophysics Data System (ADS)

Astuti, Rahmaniyah Dwi; Navi, Muhammad Abdu Haq

2018-02-01

Incompatibility between workload with work capacity is one of main problem to make optimal result. In case at the office, there are constraints to determine workload because of non-repetitive works. Employees do work based on the targets set in a working period. At the end of the period is usually performed an evaluation of employees performance to evaluate needs of employees. The aims of this study to design a workload questionnaire tools to evaluate the efficiency level of position as indicator to determine needs of employees based on the Indonesian State Employment Agency Regulation on workload analysis. This research is applied to State-Owned Enterprise PT. X by determining 3 positions as a pilot project. Position A is held by 2 employees, position B is held by 7 employees, and position C is held by 6 employees. From the calculation result, position A has an efficiency level of 1,33 or "very good", position B has an efficiency level of 1.71 or "enough", and position C has an efficiency level of 1.03 or "very good". The application of this tools giving suggestion the needs of employees of position A is 3 people, position B is 5 people, and position C is 6 people. The difference between the number of employees and the calculation result is then analyzed by interviewing the employees to get more data about personal perception. It can be concluded that this workload evaluation tools can be used as an alternative solution to evaluate needs of employees in office.

Web-based tool for visualization of electric field distribution in deep-seated body structures and planning of electroporation-based treatments.

PubMed

Marčan, Marija; Pavliha, Denis; Kos, Bor; Forjanič, Tadeja; Miklavčič, Damijan

2015-01-01

Treatments based on electroporation are a new and promising approach to treating tumors, especially non-resectable ones. The success of the treatment is, however, heavily dependent on coverage of the entire tumor volume with a sufficiently high electric field. Ensuring complete coverage in the case of deep-seated tumors is not trivial and can in best way be ensured by patient-specific treatment planning. The basis of the treatment planning process consists of two complex tasks: medical image segmentation, and numerical modeling and optimization. In addition to previously developed segmentation algorithms for several tissues (human liver, hepatic vessels, bone tissue and canine brain) and the algorithms for numerical modeling and optimization of treatment parameters, we developed a web-based tool to facilitate the translation of the algorithms and their application in the clinic. The developed web-based tool automatically builds a 3D model of the target tissue from the medical images uploaded by the user and then uses this 3D model to optimize treatment parameters. The tool enables the user to validate the results of the automatic segmentation and make corrections if necessary before delivering the final treatment plan. Evaluation of the tool was performed by five independent experts from four different institutions. During the evaluation, we gathered data concerning user experience and measured performance times for different components of the tool. Both user reports and performance times show significant reduction in treatment-planning complexity and time-consumption from 1-2 days to a few hours. The presented web-based tool is intended to facilitate the treatment planning process and reduce the time needed for it. It is crucial for facilitating expansion of electroporation-based treatments in the clinic and ensuring reliable treatment for the patients. The additional value of the tool is the possibility of easy upgrade and integration of modules with new functionalities as they are developed.

Web-based tool for visualization of electric field distribution in deep-seated body structures and planning of electroporation-based treatments

PubMed Central

2015-01-01

Background Treatments based on electroporation are a new and promising approach to treating tumors, especially non-resectable ones. The success of the treatment is, however, heavily dependent on coverage of the entire tumor volume with a sufficiently high electric field. Ensuring complete coverage in the case of deep-seated tumors is not trivial and can in best way be ensured by patient-specific treatment planning. The basis of the treatment planning process consists of two complex tasks: medical image segmentation, and numerical modeling and optimization. Methods In addition to previously developed segmentation algorithms for several tissues (human liver, hepatic vessels, bone tissue and canine brain) and the algorithms for numerical modeling and optimization of treatment parameters, we developed a web-based tool to facilitate the translation of the algorithms and their application in the clinic. The developed web-based tool automatically builds a 3D model of the target tissue from the medical images uploaded by the user and then uses this 3D model to optimize treatment parameters. The tool enables the user to validate the results of the automatic segmentation and make corrections if necessary before delivering the final treatment plan. Results Evaluation of the tool was performed by five independent experts from four different institutions. During the evaluation, we gathered data concerning user experience and measured performance times for different components of the tool. Both user reports and performance times show significant reduction in treatment-planning complexity and time-consumption from 1-2 days to a few hours. Conclusions The presented web-based tool is intended to facilitate the treatment planning process and reduce the time needed for it. It is crucial for facilitating expansion of electroporation-based treatments in the clinic and ensuring reliable treatment for the patients. The additional value of the tool is the possibility of easy upgrade and integration of modules with new functionalities as they are developed. PMID:26356007

High Thermal Conductivity Polymer Composites for Low Cost Heat Exchangers

DOE Office of Scientific and Technical Information (OSTI.GOV)

None

2017-08-01

This factsheet describes a project that identified and evaluated commercially available and state-of-the-art polymer-based material options for manufacturing industrial and commercial non-metallic heat exchangers. A heat exchanger concept was also developed and its performance evaluated with heat transfer modeling tools.

The development and assessment of an evaluation tool for pediatric resident competence in leading simulated pediatric resuscitations.

PubMed

Grant, Estée C; Grant, Vincent J; Bhanji, Farhan; Duff, Jonathan P; Cheng, Adam; Lockyer, Jocelyn M

2012-07-01

It is critical that competency in pediatric resuscitation is achieved and assessed during residency or post graduate medical training. The purpose of this study was to create and evaluate a tool to measure all elements of pediatric resuscitation team leadership competence. An initial set of items, derived from a literature review and a brainstorming session, were refined to a 26 item assessment tool through the use of Delphi methodology. The tool was tested using videos of standardized resuscitations. A psychometric assessment of the evidence for instrument validity and reliability was undertaken. The performance of 30 residents on two videotaped scenarios was assessed by 4 pediatricians using the tool, with 12 items assessing 'leadership and communication skills' (LCS) and 14 items assessing 'knowledge and clinical skills' (KCS). The instrument showed evidence of reliability; the Cronbach's alpha and generalizability co-efficients for the overall instrument were α=0.818 and Ep(2)=0.76, for LCS were α=0.827 and Ep(2)=0.844, and for KCS were α=0.673 and Ep(2)=0.482. While validity was initially established through literature review and brainstorming by the panel of experts, it was further built through the high strength of correlation between global scores and scores for overall performance (r=0.733), LCS (r=0.718) and KCS (r=0.662) as well as the factor analysis which accounted for 40.2% of the variance. The results of the study demonstrate that the instrument is a valid and reliable tool to evaluate pediatric resuscitation team leader competence. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

Comparing genome versus proteome-based identification of clinical bacterial isolates.

PubMed

Galata, Valentina; Backes, Christina; Laczny, Cédric Christian; Hemmrich-Stanisak, Georg; Li, Howard; Smoot, Laura; Posch, Andreas Emanuel; Schmolke, Susanne; Bischoff, Markus; von Müller, Lutz; Plum, Achim; Franke, Andre; Keller, Andreas

2018-05-01

Whole-genome sequencing (WGS) is gaining importance in the analysis of bacterial cultures derived from patients with infectious diseases. Existing computational tools for WGS-based identification have, however, been evaluated on previously defined data relying thereby unwarily on the available taxonomic information.Here, we newly sequenced 846 clinical gram-negative bacterial isolates representing multiple distinct genera and compared the performance of five tools (CLARK, Kaiju, Kraken, DIAMOND/MEGAN and TUIT). To establish a faithful 'gold standard', the expert-driven taxonomy was compared with identifications based on matrix-assisted laser desorption/ionization time-of-flight (MALDI-TOF) mass spectrometry (MS) analysis. Additionally, the tools were also evaluated using a data set of 200 Staphylococcus aureus isolates.CLARK and Kraken (with k =31) performed best with 626 (100%) and 193 (99.5%) correct species classifications for the gram-negative and S. aureus isolates, respectively. Moreover, CLARK and Kraken demonstrated highest mean F-measure values (85.5/87.9% and 94.4/94.7% for the two data sets, respectively) in comparison with DIAMOND/MEGAN (71 and 85.3%), Kaiju (41.8 and 18.9%) and TUIT (34.5 and 86.5%). Finally, CLARK, Kaiju and Kraken outperformed the other tools by a factor of 30 to 170 fold in terms of runtime.We conclude that the application of nucleotide-based tools using k-mers-e.g. CLARK or Kraken-allows for accurate and fast taxonomic characterization of bacterial isolates from WGS data. Hence, our results suggest WGS-based genotyping to be a promising alternative to the MS-based biotyping in clinical settings. Moreover, we suggest that complementary information should be used for the evaluation of taxonomic classification tools, as public databases may suffer from suboptimal annotations.

Development and Validation of a Family Meeting Assessment Tool (FMAT).

PubMed

Hagiwara, Yuya; Healy, Jennifer; Lee, Shuko; Ross, Jeanette; Fischer, Dixie; Sanchez-Reilly, Sandra

2018-01-01

A cornerstone procedure in Palliative Medicine is to perform family meetings. Learning how to lead a family meeting is an important skill for physicians and others who care for patients with serious illnesses and their families. There is limited evidence on how to assess best practice behaviors during end-of-life family meetings. Our aim was to develop and validate an observational tool to assess trainees' ability to lead a simulated end-of-life family meeting. Building on evidence from published studies and accrediting agency guidelines, an expert panel at our institution developed the Family Meeting Assessment Tool. All fourth-year medical students (MS4) and eight geriatric and palliative medicine fellows (GPFs) were invited to participate in a Family Meeting Objective Structured Clinical Examination, where each trainee assumed the physician role leading a complex family meeting. Two evaluators observed and rated randomly chosen students' performances using the Family Meeting Assessment Tool during the examination. Inter-rater reliability was measured using percent agreement. Internal consistency was measured using Cronbach α. A total of 141 trainees (MS4 = 133 and GPF = 8) and 26 interdisciplinary evaluators participated in the study. Internal reliability (Cronbach α) of the tool was 0.85. Number of trainees rated by two evaluators was 210 (MS4 = 202 and GPF = 8). Rater agreement was 84%. Composite scores, on average, were significantly higher for fellows than for medical students (P < 0.001). Expert-based content, high inter-rater reliability, good internal consistency, and ability to predict educational level provided initial evidence for construct validity for this novel assessment tool. Copyright © 2017 American Academy of Hospice and Palliative Medicine. All rights reserved.

Error management training and simulation education.

PubMed

Gardner, Aimee; Rich, Michelle

2014-12-01

The integration of simulation into the training of health care professionals provides context for decision making and procedural skills in a high-fidelity environment, without risk to actual patients. It was hypothesised that a novel approach to simulation-based education - error management training - would produce higher performance ratings compared with traditional step-by-step instruction. Radiology technology students were randomly assigned to participate in traditional procedural-based instruction (n = 11) or vicarious error management training (n = 11). All watched an instructional video and discussed how well each incident was handled (traditional instruction group) or identified where the errors were made (vicarious error management training). Students then participated in a 30-minute case-based simulation. Simulations were videotaped for performance analysis. Blinded experts evaluated performance using a predefined evaluation tool created specifically for the scenario. Blinded experts evaluated performance using a predefined evaluation tool created specifically for the scenario The vicarious error management group scored higher on observer-rated performance (Mean = 9.49) than students in the traditional instruction group (Mean = 9.02; p < 0.01). These findings suggest that incorporating the discussion of errors and how to handle errors during the learning session will better equip students when performing hands-on procedures and skills. This pilot study provides preliminary evidence for integrating error management skills into medical curricula and for the design of learning goals in simulation-based education. © 2014 John Wiley & Sons Ltd.

Performance Evaluation of a Mobile Touchscreen Interface for Assistive Robotic Manipulators: A Pilot Study.

PubMed

Chung, Cheng-Shiu; Ka, Hyun W; Wang, Hongu; Ding, Dan; Kelleher, Annmarie; Cooper, Rory A

2017-01-01

Background: Assistive robotic manipulators (ARMs) have been developed to provide enhanced assistance and independence in performance of daily activities among people with spinal cord injury when a caregiver is not on site. However, the current commercial ARM user interfaces (UIs) may be difficult to learn and control. A touchscreen mobile UI was developed to overcome these challenges. Objective: The object of this study was to evaluate the performance between 2 ARM UIs, touchscreen and the original joystick, using an ARM evaluation tool (ARMET). Methods: This is a pilot study of people with upper extremity impairments ( N = 8). Participants were trained on 2 UIs, and then they chose one to use when performing 3 tasks on the ARMET: flipping a toggle switch, pushing down a door handle, and turning a knob. Task completion time, mean velocity, and open interviews were the main outcome measurements. Results: Among 8 novice participants, 7 chose the touchscreen UI and 1 chose the joystick UI. All participants could complete the ARMET tasks independently. Use of the touchscreen UI resulted in enhanced ARMET performance (higher mean moving speed and faster task completion). Conclusions: Mobile ARM UIs demonstrated easier learning experience, less physical effort, and better ARMET performance. The improved performance, the accessibility, and lower physical effort suggested that the touchscreen UI might be an efficient tool for the ARM users.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bunshah, R.F.; Shabaik, A.H.

The process of Activated Reactive Evaporation is used to synthesize superhard materials like carbides, oxides, nitrides, ultrafine grain cermets. The deposits are characterized by hardness, microstructure and lattice parameter measurements. The synthesis and characterization of TiC-Ni cermets, Al/sub 2/O/sub 3/ and VC-TiC alloy carbides is given. Tools of different coating characteristics are tested for machining performance at different speeds and feeds. The machining evaluation and the selection of coatings is based on the rate of deterioration of the costing, tool temperature, and cutting forces. Tool life tests show coated high speed steel tools show a 300% improvement in tool life.more » (Author) (GRA)« less

Comparative assessment of three standardized robotic surgery training methods.

PubMed

Hung, Andrew J; Jayaratna, Isuru S; Teruya, Kara; Desai, Mihir M; Gill, Inderbir S; Goh, Alvin C

2013-10-01

To evaluate three standardized robotic surgery training methods, inanimate, virtual reality and in vivo, for their construct validity. To explore the concept of cross-method validity, where the relative performance of each method is compared. Robotic surgical skills were prospectively assessed in 49 participating surgeons who were classified as follows: 'novice/trainee': urology residents, previous experience <30 cases (n = 38) and 'experts': faculty surgeons, previous experience ≥30 cases (n = 11). Three standardized, validated training methods were used: (i) structured inanimate tasks; (ii) virtual reality exercises on the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA); and (iii) a standardized robotic surgical task in a live porcine model with performance graded by the Global Evaluative Assessment of Robotic Skills (GEARS) tool. A Kruskal-Wallis test was used to evaluate performance differences between novices and experts (construct validity). Spearman's correlation coefficient (ρ) was used to measure the association of performance across inanimate, simulation and in vivo methods (cross-method validity). Novice and expert surgeons had previously performed a median (range) of 0 (0-20) and 300 (30-2000) robotic cases, respectively (P < 0.001). Construct validity: experts consistently outperformed residents with all three methods (P < 0.001). Cross-method validity: overall performance of inanimate tasks significantly correlated with virtual reality robotic performance (ρ = -0.7, P < 0.001) and in vivo robotic performance based on GEARS (ρ = -0.8, P < 0.0001). Virtual reality performance and in vivo tissue performance were also found to be strongly correlated (ρ = 0.6, P < 0.001). We propose the novel concept of cross-method validity, which may provide a method of evaluating the relative value of various forms of skills education and assessment. We externally confirmed the construct validity of each featured training tool. © 2013 BJU International.

Comparison of tool life and surface roughness with MQL, flood cooling, and dry cutting conditions with P20 and D2 steel

NASA Astrophysics Data System (ADS)

Senevirathne, S. W. M. A. I.; Punchihewa, H. K. G.

2017-09-01

Minimum quantity lubrication (MQL) is a cutting fluid (CF) application method that has given promising results in improving machining performances. It has shown that, the performance of cutting systems, depends on the work and tool materials used. AISI P20, and D2 are popular in tool making industry. However, the applicability of MQL in machining these two steels has not been studied previously. This experimental study is focused on evaluating performances of MQL compared to dry cutting, and conventional flood cooling method. Trials were carried out with P20, and D2 steels, using coated carbides as tool material, emulsion cutting oil as the CF. Tool nose wear, and arithmetic average surface roughness (Ra) were taken as response variables. Results were statistically analysed for differences in response variables. Although many past literature has suggested that MQL causes improvements in tool wear, and surface finish, this study has found contradicting results. MQL has caused nearly 200% increase in tool nose wear, and nearly 11-13% increase in surface roughness compared flood cooling method with both P20 and D2. Therefore, this study concludes that MQL affects adversely in machining P20, and D2 steels.

Outcome Analysis Tool for Army Refractive Surgery Program

DTIC Science & Technology

2005-03-01

analysis function produces reports on the following information: " Evaluation of the safety of PRK and LASIK for maintenance of optimal visual...performance and ocular integrity. " Evaluation of the efficacy of PRK and LASIK by assessing the improvement in uncorrected vision for target detection...discrimination and recognition. "* Evaluation of the efficacy of PRK and LASIK by evaluating the stability of the refractive error over time

[Evaluation of budget design and execution, an instrument of performance-based budgeting: some experiences applied to health].

PubMed

Peñaloza-Vassallo, K; Gutiérrez-Aguado, A; Prado-Fernández, M

2017-01-01

Since 2008, the evaluation of budget design and execution (EDEP for its acronym in Spanish) - one of the evaluations tools developed by the Peruvian Ministry of Economy and Finance (MEF) as part of the implementation of Performance Budgeting- seek to provide reliable information about design coherence and progress in the implementation of public interventions, in order to improve their management and make informed budget decisions. The EDEP methodology includes preparing an evaluation report and defining a matrix of commitments to improve performance. Commitments are defined based on the recommendation of the EDEP. The EDEP seeks to correct exiting problems in public programs and optimize their results. MEF tracks the fulfillment of these commitments and links together the analysis of public budget requests and the progress of these commitments. Now, almost 10 years after its implementation, 57 EDEP have been carried out in different sectors and 07 of them are related to health interventions such as: the comprehensive health system, vaccination service, normal births, acute respiratory infections and acute diarrheal diseases, among others. Beyond the discrepancies in the application of this tool, the EDEP and its matrix of commitments have allowed the use of the results of the evaluations and have become a mechanism to generate useful information to improve the public services.

Outcome measures to evaluate new technology for tonsillectomy: preliminary experience with Coblation

NASA Astrophysics Data System (ADS)

Shah, Udayan K.; Puchalski, Robert; Briggs, Marianne; Chiavacci, Rosetta; Galinkin, Jeffrey

2001-05-01

Evaluating the benefits of new surgical technologies does not end with the observation of successful instrument-to- tissue interaction. The impact of new technologies in medicine today is also gauged by improvements in patients' daily activities and performance. We present our outcomes assessment tool for judging the value of applying a novel tonsillectomy technique, plasma- mediated ablation using Coblation technology. Plasma- mediated ablation (PMA) achieves soft tissue resection in the oropharynx by energizing protons to break bonds. Less heat is released, allowing for less thermal injury, and possibly less pain, than with tonsillectomy performed using electrocautery alone. Children undergoing tonsillectomy by PMA, were evaluated using our outcomes-based scale, which asked families to report the degree of interruption of normal activities for the patient and their family during the post-tonsillectomy recovery period. A preliminary review of several outcomes assessments exemplify the benefits and limitations of this tool. The tracking of valuable data is weighed against the limitations of a short time course relative to the duration of disability, and a poor response rate. Future work aims to improve this data collection tool to allow application to other new technologies in otolaryngology.

Development of an interprofessional lean facilitator assessment scale.

PubMed

Bravo-Sanchez, Cindy; Dorazio, Vincent; Denmark, Robert; Heuer, Albert J; Parrott, J Scott

2018-05-01

High reliability is important for optimising quality and safety in healthcare organisations. Reliability efforts include interprofessional collaborative practice (IPCP) and Lean quality/process improvement strategies, which require skilful facilitation. Currently, no validated Lean facilitator assessment tool for interprofessional collaboration exists. This article describes the development and pilot evaluation of such a tool; the Interprofessional Lean Facilitator Assessment Scale (ILFAS), which measures both technical and 'soft' skills, which have not been measured in other instruments. The ILFAS was developed using methodologies and principles from Lean/Shingo, IPCP, metacognition research and Bloom's Taxonomy of Learning Domains. A panel of experts confirmed the initial face validity of the instrument. Researchers independently assessed five facilitators, during six Lean sessions. Analysis included quantitative evaluation of rater agreement. Overall inter-rater agreement of the assessment of facilitator performance was high (92%), and discrepancies in the agreement statistics were analysed. Face and content validity were further established, and usability was evaluated, through primary stakeholder post-pilot feedback, uncovering minor concerns, leading to tool revision. The ILFAS appears comprehensive in the assessment of facilitator knowledge, skills, abilities, and may be useful in the discrimination between facilitators of different skill levels. Further study is needed to explore instrument performance and validity.

Integration of Occupational Safety to Contractors` or Subcontractors` Performance Evaluation in Construction Projects

NASA Astrophysics Data System (ADS)

Kozlovská, Mária; Struková, Zuzana

2013-06-01

Several factors should be considered by the owner and general contractor in the process of contractors` and subcontractors` selection and evaluation. The paper reviews the recent models addressed to guide general contractors in subcontractors' selection process and in evaluation of different contractors during the execution of the project. Moreover the paper suggests the impact of different contractors' performance to the overall level of occupational health and safety culture at the sites. It deals with the factors influencing the safety performance of contractors during construction and analyses the methods for assessing the safety performance of construction contractors. The results of contractors' safety performance evaluation could be a useful tool in motivating contractors to achieve better safety outcomes or could have effect on owners` or general contractors' decision making about contractors suitability for future contracting works.

Human performance cognitive-behavioral modeling: a benefit for occupational safety.

PubMed

Gore, Brian F

2002-01-01

Human Performance Modeling (HPM) is a computer-aided job analysis software methodology used to generate predictions of complex human-automation integration and system flow patterns with the goal of improving operator and system safety. The use of HPM tools has recently been increasing due to reductions in computational cost, augmentations in the tools' fidelity, and usefulness in the generated output. An examination of an Air Man-machine Integration Design and Analysis System (Air MIDAS) model evaluating complex human-automation integration currently underway at NASA Ames Research Center will highlight the importance to occupational safety of considering both cognitive and physical aspects of performance when researching human error.

The importance of performance assessment in local government decisions to fund health and human services nonprofit organizations.

PubMed

Vaughan, Shannon K

2010-01-01

In times of fiscal crisis, demand for health and human services increases while revenues shrink, causing funders to focus more intently on identifying the most successful organizations in which to invest scarce resources. This research grew out of interest in enhancing performance assessment of nonprofit organizations expressed by local government managers. A survey of Alliance for Innovation Members explores two primary research questions: 1) what is a successful nonprofit; and 2) what type(s) of performance assessment tools are the most useful. The results strengthen our understanding of what information city and county managers want and why they prefer certain evaluation tools.

Diverter AI based decision aid, phases 1 and 2

NASA Technical Reports Server (NTRS)

Sexton, George A.; Bayles, Scott J.; Patterson, Robert W.; Schulke, Duane A.; Williams, Deborah C.

1989-01-01

It was determined that a system to incorporate artificial intelligence (AI) into airborne flight management computers is feasible. The AI functions that would be most useful to the pilot are to perform situational assessment, evaluate outside influences on the contemplated rerouting, perform flight planning/replanning, and perform maneuver planning. A study of the software architecture and software tools capable of demonstrating Diverter was also made. A skeletal planner known as the Knowledge Acquisition Development Tool (KADET), which is a combination script-based and rule-based system, was used to implement the system. A prototype system was developed which demonstrates advanced in-flight planning/replanning capabilities.

Human performance cognitive-behavioral modeling: a benefit for occupational safety

NASA Technical Reports Server (NTRS)

Gore, Brian F.

2002-01-01

Human Performance Modeling (HPM) is a computer-aided job analysis software methodology used to generate predictions of complex human-automation integration and system flow patterns with the goal of improving operator and system safety. The use of HPM tools has recently been increasing due to reductions in computational cost, augmentations in the tools' fidelity, and usefulness in the generated output. An examination of an Air Man-machine Integration Design and Analysis System (Air MIDAS) model evaluating complex human-automation integration currently underway at NASA Ames Research Center will highlight the importance to occupational safety of considering both cognitive and physical aspects of performance when researching human error.

The Business Change Initiative: A Novel Approach to Improved Cost and Schedule Management

NASA Technical Reports Server (NTRS)

Shinn, Stephen A.; Bryson, Jonathan; Klein, Gerald; Lunz-Ruark, Val; Majerowicz, Walt; McKeever, J.; Nair, Param

2016-01-01

Goddard Space Flight Center's Flight Projects Directorate employed a Business Change Initiative (BCI) to infuse a series of activities coordinated to drive improved cost and schedule performance across Goddard's missions. This sustaining change framework provides a platform to manage and implement cost and schedule control techniques throughout the project portfolio. The BCI concluded in December 2014, deploying over 100 cost and schedule management changes including best practices, tools, methods, training, and knowledge sharing. The new business approach has driven the portfolio to improved programmatic performance. The last eight launched GSFC missions have optimized cost, schedule, and technical performance on a sustained basis to deliver on time and within budget, returning funds in many cases. While not every future mission will boast such strong performance, improved cost and schedule tools, management practices, and ongoing comprehensive evaluations of program planning and control methods to refine and implement best practices will continue to provide a framework for sustained performance. This paper will describe the tools, techniques, and processes developed during the BCI and the utilization of collaborative content management tools to disseminate project planning and control techniques to ensure continuous collaboration and optimization of cost and schedule management in the future.

Evaluation of a Dispatcher's Route Optimization Decision Aid to Avoid Aviation Weather Hazards

NASA Technical Reports Server (NTRS)

Dorneich, Michael C.; Olofinboba, Olu; Pratt, Steve; Osborne, Dannielle; Feyereisen, Thea; Latorella, Kara

2003-01-01

This document describes the results and analysis of the formal evaluation plan for the Honeywell software tool developed under the NASA AWIN (Aviation Weather Information) 'Weather Avoidance using Route Optimization as a Decision Aid' project. The software tool aims to provide airline dispatchers with a decision aid for selecting optimal routes that avoid weather and other hazards. This evaluation compares and contrasts route selection performance with the AWIN tool to that of subjects using a more traditional dispatcher environment. The evaluation assesses gains in safety, in fuel efficiency of planned routes, and in time efficiency in the pre-flight dispatch process through the use of the AWIN decision aid. In addition, we are interested in how this AWIN tool affects constructs that can be related to performance. The construct of Situation Awareness (SA), workload, trust in an information system, and operator acceptance are assessed using established scales, where these exist, as well as through the evaluation of questionnaire responses and subject comments. The intention of the experiment is to set up a simulated operations area for the dispatchers to work in. They will be given scenarios in which they are presented with stored company routes for a particular city-pair and aircraft type. A diverse set of external weather information sources is represented by a stand-alone display (MOCK), containing the actual historical weather data typically used by dispatchers. There is also the possibility of presenting selected weather data on the route visualization tool. The company routes have not been modified to avoid the weather except in the case of one additional route generated by the Honeywell prototype flight planning system. The dispatcher will be required to choose the most appropriate and efficient flight plan route in the displayed weather conditions. The route may be modified manually or may be chosen from those automatically displayed.

Evaluation of virtual reality bronchoscopy as a learning and assessment tool.

PubMed

Moorthy, K; Smith, S; Brown, T; Bann, S; Darzi, A

2003-01-01

Conventional training in bronchoscopy involves a trainee performing on a real patient under supervision. This method of training is not only expensive, but there is also potential for increased patient discomfort. Simulators permit the acquisition of necessary technical skills required for the procedure. Virtual reality (VR) has been an integral part of training in aviation, and the application of this technology in medical training needs to be evaluated. This study was conducted to evaluate the efficacy of a VR bronchoscopy simulator as a learning and assessment tool. The bronchoscopic simulator (HT Medical Systems, Maryland, USA) is a VR computer programme. The simulator has the ability to assess competence by a set of parameters, which formed the data for the study. Nine novices without previous bronchoscopic experience formed the study group (group 1). Nine experienced bronchoscopists having performed between 200 and 1000 bronchoscopies formed the other group (group 2). We assessed the efficacy of the system as a learning tool by studying whether there was a significant difference between the first and subsequent sessions of the subjects from group 1 and by comparing the performance of the two groups. Statistical analysis was done using the Mann-Whitney U test and the Wilcoxon signed ranks test. There was a significant difference in performance between the first attempt of group 1 and the performance of the experts in terms of percentage of segments visualised and number of wall collisions and the economy of performance. Among the subjects from group 1, there was a significant improvement in percentage of segments visualised by the third attempt (p = 0.04), in the economy of performance by the sixth attempt (0.008) and in the number of wall collisions by the sixth attempt (0.024). When each attempt of the novices was compared with the performance of group 2, the significance in the difference of the percentage of segments studied (p = 0.09) and the economy of performance disappeared by the third attempt (0.06), while the difference in the number of wall collisions disappeared by the fifth attempt (p = 0.06). This study has been able to establish the face, construct and content validity of the simulator and the potential for it to be an effective training tool. Copyright 2003 S. Karger AG, Basel

Building a Community Infrastructure for Scalable On-Line Performance Analysis Tools around Open|Speedshop

DOE Office of Scientific and Technical Information (OSTI.GOV)

Miller, Barton

2014-06-30

Peta-scale computing environments pose significant challenges for both system and application developers and addressing them required more than simply scaling up existing tera-scale solutions. Performance analysis tools play an important role in gaining this understanding, but previous monolithic tools with fixed feature sets have not sufficed. Instead, this project worked on the design, implementation, and evaluation of a general, flexible tool infrastructure supporting the construction of performance tools as “pipelines” of high-quality tool building blocks. These tool building blocks provide common performance tool functionality, and are designed for scalability, lightweight data acquisition and analysis, and interoperability. For this project, wemore » built on Open|SpeedShop, a modular and extensible open source performance analysis tool set. The design and implementation of such a general and reusable infrastructure targeted for petascale systems required us to address several challenging research issues. All components needed to be designed for scale, a task made more difficult by the need to provide general modules. The infrastructure needed to support online data aggregation to cope with the large amounts of performance and debugging data. We needed to be able to map any combination of tool components to each target architecture. And we needed to design interoperable tool APIs and workflows that were concrete enough to support the required functionality, yet provide the necessary flexibility to address a wide range of tools. A major result of this project is the ability to use this scalable infrastructure to quickly create tools that match with a machine architecture and a performance problem that needs to be understood. Another benefit is the ability for application engineers to use the highly scalable, interoperable version of Open|SpeedShop, which are reassembled from the tool building blocks into a flexible, multi-user interface set of tools. This set of tools targeted at Office of Science Leadership Class computer systems and selected Office of Science application codes. We describe the contributions made by the team at the University of Wisconsin. The project built on the efforts in Open|SpeedShop funded by DOE/NNSA and the DOE/NNSA Tri-Lab community, extended Open|Speedshop to the Office of Science Leadership Class Computing Facilities, and addressed new challenges found on these cutting edge systems. Work done under this project at Wisconsin can be divided into two categories, new algorithms and techniques for debugging, and foundation infrastructure work on our Dyninst binary analysis and instrumentation toolkits and MRNet scalability infrastructure.« less

Demystifying Results-Based Performance Measurement.

ERIC Educational Resources Information Center

Jorjani, Hamid

Many evaluators are convinced that Results-based Performance Measurement (RBPM) is an effective tool to improve service delivery and cost effectiveness in both public and private sectors. Successful RBPM requires self-directed and cross-functional work teams and the supporting infrastructure to make it work. There are many misconceptions and…

Hybrid and Electric Advanced Vehicle Systems Simulation

NASA Technical Reports Server (NTRS)

Beach, R. F.; Hammond, R. A.; Mcgehee, R. K.

1985-01-01

Predefined components connected to represent wide variety of propulsion systems. Hybrid and Electric Advanced Vehicle System (HEAVY) computer program is flexible tool for evaluating performance and cost of electric and hybrid vehicle propulsion systems. Allows designer to quickly, conveniently, and economically predict performance of proposed drive train.

Evaluating hydrological model performance using information theory-based metrics

USDA-ARS?s Scientific Manuscript database

The accuracy-based model performance metrics not necessarily reflect the qualitative correspondence between simulated and measured streamflow time series. The objective of this work was to use the information theory-based metrics to see whether they can be used as complementary tool for hydrologic m...

Study of the time and effort signal in cutting operations

NASA Astrophysics Data System (ADS)

Grosset, E.; Maillard, A.; Bouhelier, C.; Gasnier, J.

1990-02-01

Perception and treatment of an effort signal by computer methods is discussed. An automatic control system used to measure the wear of machine tools and carry out quality control throughout the cutting process is described. The testing system is used to evaluate the performance of tools which have been vacuum plated. The system is used as part of the BRITE study, the goal of which is to develop an expert system for measuring the wear of tools used during drilling and perforation operations.

Summary of performance data for technologies to control gaseous, odor, and particulate emissions from livestock operations: Air management practices assessment tool (AMPAT)

PubMed Central

Maurer, Devin L.; Koziel, Jacek A.; Harmon, Jay D.; Hoff, Steven J.; Rieck-Hinz, Angela M.; Andersen, Daniel S.

2016-01-01

The livestock and poultry production industry, regulatory agencies, and researchers lack a current, science-based guide and data base for evaluation of air quality mitigation technologies. Data collected from science-based review of mitigation technologies using practical, stakeholders-oriented evaluation criteria to identify knowledge gaps/needs and focuses for future research efforts on technologies and areas with the greatest impact potential is presented in the Literature Database tab on the air management practices tool (AMPAT). The AMPAT is web-based (available at www.agronext.iastate.edu/ampat) and provides an objective overview of mitigation practices best suited to address odor, gaseous, and particulate matter (PM) emissions at livestock operations. The data was compiled into Excel spreadsheets from a literature review of 265 papers was performed to (1) evaluate mitigation technologies performance for emissions of odor, volatile organic compounds (VOCs), ammonia (NH3), hydrogen sulfide (H2S), particulate matter (PM), and greenhouse gases (GHGs) and to (2) inform future research needs. PMID:27158660

One important way to get the best performance from staff.

PubMed

Goldstein, J M

1999-03-01

Good communication about expectations is essential to encouraging the best performance from staff. An excellent tool for fostering this communication is the performance evaluation. Used correctly, it can open a dialogue that can lead to agreement between the supervisor and staff member on areas that need to be improved and ways to do so.

Motor Skill Assessment of Children: Is There an Association between Performance-Based, Child-Report, and Parent-Report Measures of Children's Motor Skills?

ERIC Educational Resources Information Center

Kennedy, Johanna; Brown, Ted; Chien, Chi-Wen

2012-01-01

Client-centered practice requires therapists to actively seek the perspectives of children and families. Several assessment tools are available to facilitate this process. However, when evaluating motor skill performance, therapists typically concentrate on performance-based assessment. To improve understanding of the information provided by the…

Application of the Tool for Turbine Engine Closed-loop Transient Analysis (TTECTrA) for Dynamic Systems Analysis

NASA Technical Reports Server (NTRS)

Csank, Jeffrey; Zinnecker, Alicia

2014-01-01

Systems analysis involves steady-state simulations of combined components to evaluate the steady-state performance, weight, and cost of a system; dynamic considerations are not included until later in the design process. The Dynamic Systems Analysis task, under NASAs Fixed Wing project, is developing the capability for assessing dynamic issues at earlier stages during systems analysis. To provide this capability the Tool for Turbine Engine Closed-loop Transient Analysis (TTECTrA) has been developed to design a single flight condition controller (defined as altitude and Mach number) and, ultimately, provide an estimate of the closed-loop performance of the engine model. This tool has been integrated with the Commercial Modular Aero-Propulsion System Simulation 40,000(CMAPSS40k) engine model to demonstrate the additional information TTECTrA makes available for dynamic systems analysis. This dynamic data can be used to evaluate the trade-off between performance and safety, which could not be done with steady-state systems analysis data. TTECTrA has been designed to integrate with any turbine engine model that is compatible with the MATLABSimulink (The MathWorks, Inc.) environment.

Application of the Tool for Turbine Engine Closed-loop Transient Analysis (TTECTrA) for Dynamic Systems Analysis

NASA Technical Reports Server (NTRS)

Csank, Jeffrey Thomas; Zinnecker, Alicia Mae

2014-01-01

Systems analysis involves steady-state simulations of combined components to evaluate the steady-state performance, weight, and cost of a system; dynamic considerations are not included until later in the design process. The Dynamic Systems Analysis task, under NASAs Fixed Wing project, is developing the capability for assessing dynamic issues at earlier stages during systems analysis. To provide this capability the Tool for Turbine Engine Closed-loop Transient Analysis (TTECTrA) has been developed to design a single flight condition controller (defined as altitude and Mach number) and, ultimately, provide an estimate of the closed-loop performance of the engine model. This tool has been integrated with the Commercial Modular Aero-Propulsion System Simulation 40,000 (CMAPSS 40k) engine model to demonstrate the additional information TTECTrA makes available for dynamic systems analysis. This dynamic data can be used to evaluate the trade-off between performance and safety, which could not be done with steady-state systems analysis data. TTECTrA has been designed to integrate with any turbine engine model that is compatible with the MATLAB Simulink (The MathWorks, Inc.) environment.

Evaluating supplier quality performance using fuzzy analytical hierarchy process

NASA Astrophysics Data System (ADS)

Ahmad, Nazihah; Kasim, Maznah Mat; Rajoo, Shanmugam Sundram Kalimuthu

2014-12-01

Evaluating supplier quality performance is vital in ensuring continuous supply chain improvement, reducing the operational costs and risks towards meeting customer's expectation. This paper aims to illustrate an application of Fuzzy Analytical Hierarchy Process to prioritize the evaluation criteria in a context of automotive manufacturing in Malaysia. Five main criteria were identified which were quality, cost, delivery, customer serviceand technology support. These criteria had been arranged into hierarchical structure and evaluated by an expert. The relative importance of each criteria was determined by using linguistic variables which were represented as triangular fuzzy numbers. The Center of Gravity defuzzification method was used to convert the fuzzy evaluations into their corresponding crisps values. Such fuzzy evaluation can be used as a systematic tool to overcome the uncertainty evaluation of suppliers' performance which usually associated with human being subjective judgments.

Algorithmic Classification of Five Characteristic Types of Paraphasias.

PubMed

Fergadiotis, Gerasimos; Gorman, Kyle; Bedrick, Steven

2016-12-01

This study was intended to evaluate a series of algorithms developed to perform automatic classification of paraphasic errors (formal, semantic, mixed, neologistic, and unrelated errors). We analyzed 7,111 paraphasias from the Moss Aphasia Psycholinguistics Project Database (Mirman et al., 2010) and evaluated the classification accuracy of 3 automated tools. First, we used frequency norms from the SUBTLEXus database (Brysbaert & New, 2009) to differentiate nonword errors and real-word productions. Then we implemented a phonological-similarity algorithm to identify phonologically related real-word errors. Last, we assessed the performance of a semantic-similarity criterion that was based on word2vec (Mikolov, Yih, & Zweig, 2013). Overall, the algorithmic classification replicated human scoring for the major categories of paraphasias studied with high accuracy. The tool that was based on the SUBTLEXus frequency norms was more than 97% accurate in making lexicality judgments. The phonological-similarity criterion was approximately 91% accurate, and the overall classification accuracy of the semantic classifier ranged from 86% to 90%. Overall, the results highlight the potential of tools from the field of natural language processing for the development of highly reliable, cost-effective diagnostic tools suitable for collecting high-quality measurement data for research and clinical purposes.

Student Self-evaluation After Nursing Examinations: That's a Wrap.

PubMed

Butzlaff, Alice; Gaylle, Debrayh; O'Leary Kelley, Colleen

2018-04-13

Examination wrappers are a self-evaluation tool that uses metacognition to help students reflect on test performance. After examinations, rather than focus on points earned, students learn to self-identify study strategies and recognize methods of test preparation. The purpose of the study was to determine if the use of an examination wrapper after each test would encourage students to self-evaluate performance and adjust study strategies. A total of 120 undergraduate nursing students completed self-evaluations after each examination, which were analyzed using content analysis. Three general patterns emerged from student self-evaluation: effective and ineffective study strategies, understanding versus memorization of content, and nurse educator assistance.

Indicators and Measurement Tools for Health Systems Integration: A Knowledge Synthesis

PubMed Central

Oelke, Nelly D.; da Silva Lima, Maria Alice Dias; Stiphout, Michelle; Janke, Robert; Witt, Regina Rigatto; Van Vliet-Brown, Cheryl; Schill, Kaela; Rostami, Mahnoush; Hepp, Shelanne; Birney, Arden; Al-Roubaiai, Fatima; Marques, Giselda Quintana

2017-01-01

Background: Despite far reaching support for integrated care, conceptualizing and measuring integrated care remains challenging. This knowledge synthesis aimed to identify indicator domains and tools to measure progress towards integrated care. Methods: We used an established framework and a Delphi survey with integration experts to identify relevant measurement domains. For each domain, we searched and reviewed the literature for relevant tools. Findings: From 7,133 abstracts, we retrieved 114 unique tools. We found many quality tools to measure care coordination, patient engagement and team effectiveness/performance. In contrast, there were few tools in the domains of performance measurement and information systems, alignment of organizational goals and resource allocation. The search yielded 12 tools that measure overall integration or three or more indicator domains. Discussion: Our findings highlight a continued gap in tools to measure foundational components that support integrated care. In the absence of such targeted tools, “overall integration” tools may be useful for a broad assessment of the overall state of a system. Conclusions: Continued progress towards integrated care depends on our ability to evaluate the success of strategies across different levels and context. This study has identified 114 tools that measure integrated care across 16 domains, supporting efforts towards a unified measurement framework. PMID:29588637

How to perform a critically appraised topic: part 2, appraise, evaluate, generate, and recommend.

PubMed

Kelly, Aine Marie; Cronin, Paul

2011-11-01

This article continues the discussion of a critically appraised topic started in Part 1. A critically appraised topic is a practical tool for learning and applying critical appraisal skills. This article outlines steps 4-7 involved in performing a critically appraised topic for studies of diagnostic tests: Appraise, Appraise the literature; Evaluate, evaluate the strength of the evidence from the literature; Generate, generate graphs of conditional probability; and Recommend, draw conclusions and make recommendations. For steps 4-7 of performing a critically appraised topic, the main study results are summarized and translated into clinically useful measures of accuracy, efficacy, or risk.

Evaluation of colonoscopic allergen provocation as a diagnostic tool in dogs with proven food hypersensitivity reactions.

PubMed

Allenspach, K; Vaden, S L; Harris, T S; Gröne, A; Doherr, M G; Griot-Wenk, M E; Bischoff, S C; Gaschen, F

2006-01-01

To evaluate the colonoscopic allergen provocation (COLAP) test as a new tool for the diagnosis of IgE-mediated food allergy. Oral food challenges as well as COLAP testing were performed in a colony of nine research dogs with proven immediate-type food allergic reactions. In addition, COLAP was performed in five healthy dogs. When compared with the oral challenge test, COLAP accurately determined 18 of 23 (73 per cent) positive oral challenge reactions (73 per cent) in dogs with food allergies and was negative in the healthy dogs. The accuracy of this new test may be higher than that for gastric sensitivity testing. Therefore, COLAP holds promise as a new test to confirm the diagnosis of suspect IgE-mediated food allergy in dogs.

A Novel Approach to Practice-Based Learning and Improvement Using a Web-Based Audit and Feedback Module.

PubMed

Boggan, Joel C; Cheely, George; Shah, Bimal R; Heffelfinger, Randy; Springall, Deanna; Thomas, Samantha M; Zaas, Aimee; Bae, Jonathan

2014-09-01

Systematically engaging residents in large programs in quality improvement (QI) is challenging. To coordinate a shared QI project in a large residency program using an online tool. A web-based QI tool guided residents through a 2-phase evaluation of performance of foot examinations in patients with diabetes. In phase 1, residents completed reviews of health records with online data entry. Residents were then presented with personal performance data relative to peers and were prompted to develop improvement plans. In phase 2, residents again reviewed personal performance. Rates of performance were compared at the program and clinic levels for each phase, with data presented for residents. Acceptability was measured by the number of residents completing each phase. Feasibility was measured by estimated faculty, programmer, and administrator time and costs. Seventy-nine of 86 eligible residents (92%) completed improvement plans and reviewed 1471 patients in phase 1, whereas 68 residents (79%) reviewed 1054 patient charts in phase 2. Rates of performance of examination increased significantly between phases (from 52% to 73% for complete examination, P < .001). Development of the tool required 130 hours of programmer time. Project analysis and management required 6 hours of administrator and faculty time monthly. An online tool developed and implemented for program-wide QI initiatives successfully engaged residents to participate in QI activities. Residents using this tool demonstrated improvement in a selected quality target. This tool could be adapted by other graduate medical education programs or for faculty development.

Evaluation and Assessment of a Biomechanics Computer-Aided Instruction.

ERIC Educational Resources Information Center

Washington, N.; Parnianpour, M.; Fraser, J. M.

1999-01-01

Describes the Biomechanics Tutorial, a computer-aided instructional tool that was developed at Ohio State University to expedite the transition from lecture to application for undergraduate students. Reports evaluation results that used statistical analyses and student questionnaires to show improved performance on posttests as well as positive…

Predictive Heterosis in Multibreed Evaluations Using Quantitative and Molecular Approaches

USDA-ARS?s Scientific Manuscript database

Heterosis is the extra genetic boost in performance obtained by crossing two cattle breeds. It is an important tool for increasing the efficiency of beef production. It is also important to adjust data used to calculate genetic evaluations for differences in heterosis. Good estimates of heterosis...

Evaluating the mitigation of greenhouse gas emissions and adaptation in dairy production.

USDA-ARS?s Scientific Manuscript database

Process-level modeling at the farm scale provides a tool for evaluating strategies for both mitigating greenhouse gas emissions and adapting to climate change. The Integrated Farm System Model (IFSM) simulates representative crop, beef or dairy farms over many years of weather to predict performance...

Tools of Individual Evaluation and Prestige Recognition in Spain: How Sexenio 'Mints the Golden Coin of Authority'

ERIC Educational Resources Information Center

Marini, Giulio

2018-01-01

Individual experiences in dealing with individual evaluations are studied through a national documental analysis and qualitative interviews. The analysis considers three main individual assessments designed to measure individual credentials or performance: "sexenio" (research and third mission), "quinquenio" (teaching) and…

Principal Evaluation: Standards, Rubrics, and Tools for Effective Performance

ERIC Educational Resources Information Center

Stronge, James H.; Xu, Xianxuan; Leeper, Lauri M.; Tonneson, Virginia C.

2013-01-01

Effective principals run effective schools--this much we know. Accurately measuring principal effectiveness, however, has long been an elusive goal for school administrators. In this indispensable book, author James H. Stronge details the steps and resources necessary for designing a comprehensive principal evaluation system that is based on sound…

Exploring Non-Instructional Factors in Student Evaluations

ERIC Educational Resources Information Center

Ali, Holi Ibrahim Holi; Al Ajmi, Ahmed Ali Saleh

2013-01-01

The use of student ratings to measure instructors' teaching performance and effectiveness in tertiary education has been an important but controversial tool in the improvement of teaching quality during the past few decades. This is an attempt to explore non-instructional factors of student evaluations by discussing and reviewing relevant…

Revamping Teacher Evaluation

ERIC Educational Resources Information Center

Zatynski, Mandy

2012-01-01

In the past two years, as concerns over teacher quality have swelled, teacher evaluation has emerged as a crucial tool for principals and other administrators to improve instructor performance. More states are seeking federal waivers to the stringent benchmarks of No Child Left Behind; others are vying for Race to the Top funds. Both require…

Tools For Evaluating The Benefits Of Green Infrastructure For Urban Water Management: Informational Brief (WERF Report INFR5SG09b)

EPA Science Inventory

This report identifies the practical challenges for evaluating the benefits of green infrastructure. It also discusses a more systematic approach to integrate cost-effective, high-performance urban water infrastructure practices with other environmental, social, and economic goa...

Drug Abuse Films, Second Edition.

ERIC Educational Resources Information Center

National Coordinating Council on Drug Education, Washington, DC.

This second edition updates and expands a 1971 evaluation of films and audiovisuals related to drug education performed by the National Coordinating Council on Drug Education. Materials in this edition are evaluated both for accuracy and effectiveness as a communications tool. They are separated into two sections--films and other audiovisuals…

Using vignettes to study nurse practitioners' performance in suspected domestic violence situations.

PubMed

Gagan, M J

2000-01-01

Vignettes have often been used to evaluate students or collect data in nursing research. The format is familiar to most nursing students as well as nurses and nurse researchers. This article presents the development and testing of the Nurse Practitioner Performance Tool (NPPT) which used vignettes as an approach to nurse practitioner performance evaluation. In this example, vignettes were used in a quasi-experimental design to collect data from Adult and Family Nurse Practitioners (A/FNP). The focus was on the diagnosis and intervention performance of the A/FNPs when addressing suspected cases of domestic violence.

Benchmarking short sequence mapping tools

PubMed Central

2013-01-01

Background The development of next-generation sequencing instruments has led to the generation of millions of short sequences in a single run. The process of aligning these reads to a reference genome is time consuming and demands the development of fast and accurate alignment tools. However, the current proposed tools make different compromises between the accuracy and the speed of mapping. Moreover, many important aspects are overlooked while comparing the performance of a newly developed tool to the state of the art. Therefore, there is a need for an objective evaluation method that covers all the aspects. In this work, we introduce a benchmarking suite to extensively analyze sequencing tools with respect to various aspects and provide an objective comparison. Results We applied our benchmarking tests on 9 well known mapping tools, namely, Bowtie, Bowtie2, BWA, SOAP2, MAQ, RMAP, GSNAP, Novoalign, and mrsFAST (mrFAST) using synthetic data and real RNA-Seq data. MAQ and RMAP are based on building hash tables for the reads, whereas the remaining tools are based on indexing the reference genome. The benchmarking tests reveal the strengths and weaknesses of each tool. The results show that no single tool outperforms all others in all metrics. However, Bowtie maintained the best throughput for most of the tests while BWA performed better for longer read lengths. The benchmarking tests are not restricted to the mentioned tools and can be further applied to others. Conclusion The mapping process is still a hard problem that is affected by many factors. In this work, we provided a benchmarking suite that reveals and evaluates the different factors affecting the mapping process. Still, there is no tool that outperforms all of the others in all the tests. Therefore, the end user should clearly specify his needs in order to choose the tool that provides the best results. PMID:23758764

Statistical Process Control: A Quality Tool for a Venous Thromboembolic Disease Registry.

PubMed

Posadas-Martinez, Maria Lourdes; Rojas, Liliana Paloma; Vazquez, Fernando Javier; De Quiros, Fernan Bernaldo; Waisman, Gabriel Dario; Giunta, Diego Hernan

2016-01-01

We aim to describe Statistical Control Process as a quality tool for the Institutional Registry of Venous Thromboembolic Disease (IRTD), a registry developed in a community-care tertiary hospital in Buenos Aires, Argentina. The IRTD is a prospective cohort. The process of data acquisition began with the creation of a computerized alert generated whenever physicians requested imaging or laboratory study to diagnose venous thromboembolism, which defined eligible patients. The process then followed a structured methodology for patient's inclusion, evaluation, and posterior data entry. To control this process, process performance indicators were designed to be measured monthly. These included the number of eligible patients, the number of included patients, median time to patient's evaluation, and percentage of patients lost to evaluation. Control charts were graphed for each indicator. The registry was evaluated in 93 months, where 25,757 patients were reported and 6,798 patients met inclusion criteria. The median time to evaluation was 20 hours (SD, 12) and 7.7% of the total was lost to evaluation. Each indicator presented trends over time, caused by structural changes and improvement cycles, and therefore the central limit suffered inflexions. Statistical process control through process performance indicators allowed us to control the performance of the registry over time to detect systematic problems. We postulate that this approach could be reproduced for other clinical registries.

Student Use of NABPLaw Online in a Pharmacy Laws Project

PubMed Central

Hammer, Dana P.; Hartnett, Cassandra J.; Williams, Donald H.

2006-01-01

Objectives To evaluate students’ frequency of use and degree of usefulness of NABPLaw Online, a pharmacy-specific, online, licensed resource produced by the National Association of Boards of Pharmacy (NABP). Methods Students usage of various information resources, including NABPLaw Online were evaluated through (1) usage statistics gathered by NABP, (2) students’ response to a questionnaire, and (3) citation analysis performed on students’ project reports. Results Students used NABPLaw Online less frequently than other online tools, partly related to the relevance of the tool to their projects, and partly related to ease of use in comparison to other tools. Conclusions Although it was not extensively used, NABPLaw Online represents a unique resource for students researching multistate aspects of pharmacy practice law. PMID:17149444

Supportive supervision and constructive relationships with healthcare workers support CHW performance: Use of a qualitative framework to evaluate CHW programming in Uganda.

PubMed

Ludwick, Teralynn; Turyakira, Eleanor; Kyomuhangi, Teddy; Manalili, Kimberly; Robinson, Sheila; Brenner, Jennifer L

2018-02-13

While evidence supports community health worker (CHW) capacity to improve maternal and newborn health in less-resourced countries, key implementation gaps remain. Tools for assessing CHW performance and evidence on what programmatic components affect performance are lacking. This study developed and tested a qualitative evaluative framework and tool to assess CHW team performance in a district program in rural Uganda. A new assessment framework was developed to collect and analyze qualitative evidence based on CHW perspectives on seven program components associated with effectiveness (selection; training; community embeddedness; peer support; supportive supervision; relationship with other healthcare workers; retention and incentive structures). Focus groups were conducted with four high/medium-performing CHW teams and four low-performing CHW teams selected through random, stratified sampling. Content analysis involved organizing focus group transcripts according to the seven program effectiveness components, and assigning scores to each component per focus group. Four components, 'supportive supervision', 'good relationships with other healthcare workers', 'peer support', and 'retention and incentive structures' received the lowest overall scores. Variances in scores between 'high'/'medium'- and 'low'-performing CHW teams were largest for 'supportive supervision' and 'good relationships with other healthcare workers.' Our analysis suggests that in the Bushenyi intervention context, CHW team performance is highly correlated with the quality of supervision and relationships with other healthcare workers. CHWs identified key performance-related issues of absentee supervisors, referral system challenges, and lack of engagement/respect by health workers. Other less-correlated program components warrant further study and may have been impacted by relatively consistent program implementation within our limited study area. Applying process-oriented measurement tools are needed to better understand CHW performance-related factors and build a supportive environment for CHW program effectiveness and sustainability. Findings from a qualitative, multi-component tool developed and applied in this study suggest that factors related to (1) supportive supervision and (2) relationships with other healthcare workers may be strongly associated with variances in performance outcomes within a program. Careful consideration of supervisory structure and health worker orientation during program implementation are among strategies proposed to increase CHW performance.

The Dutch Health Care Performance Report: seven years of health care performance assessment in the Netherlands.

PubMed

van den Berg, Michael J; Kringos, Dionne S; Marks, Lisanne K; Klazinga, Niek S

2014-01-09

In 2006, the first edition of a monitoring tool for the performance of the Dutch health care system was released: the Dutch Health Care Performance Report (DHCPR). The Netherlands was among the first countries in the world developing such a comprehensive tool for reporting performance on quality, access, and affordability of health care. The tool contains 125 performance indicators; the choice for specific indicators resulted from a dialogue between researchers and policy makers. In the 'policy cycle', the DHCPR can rationally be placed between evaluation (accountability) and agenda-setting (for strategic decision making). In this paper, we reflect on important lessons learned after seven years of health care system performance assessment. These lessons entail the importance of a good conceptual framework for health system performance assessment, the importance of repeated measurement, the strength of combining multiple perspectives (e.g., patient, professional, objective, subjective) on the same issue, the importance of a central role for the patients' perspective in performance assessment, how to deal with the absence of data in relevant domains, the value of international benchmarking and the continuous exchange between researchers and policy makers.

Simulating the Daylight Performance of Complex Fenestration Systems Using Bidirectional Scattering Distribution Functions within Radiance

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ward, Gregory; Mistrick, Ph.D., Richard; Lee, Eleanor

2011-01-21

We describe two methods which rely on bidirectional scattering distribution functions (BSDFs) to model the daylighting performance of complex fenestration systems (CFS), enabling greater flexibility and accuracy in evaluating arbitrary assemblies of glazing, shading, and other optically-complex coplanar window systems. Two tools within Radiance enable a) efficient annual performance evaluations of CFS, and b) accurate renderings of CFS despite the loss of spatial resolution associated with low-resolution BSDF datasets for inhomogeneous systems. Validation, accuracy, and limitations of the methods are discussed.

Roles and methods of performance evaluation of hospital academic leadership.

PubMed

Zhou, Ying; Yuan, Huikang; Li, Yang; Zhao, Xia; Yi, Lihua

2016-01-01

The rapidly advancing implementation of public hospital reform urgently requires the identification and classification of a pool of exceptional medical specialists, corresponding with incentives to attract and retain them, providing a nucleus of distinguished expertise to ensure public hospital preeminence. This paper examines the significance of academic leadership, from a strategic management perspective, including various tools, methods and mechanisms used in the theory and practice of performance evaluation, and employed in the selection, training and appointment of academic leaders. Objective methods of assessing leadership performance are also provided for reference.

7 CFR 275.19 - Monitoring and evaluation.

Code of Federal Regulations, 2012 CFR

2012-01-01

... AGRICULTURE FOOD STAMP AND FOOD DISTRIBUTION PROGRAM PERFORMANCE REPORTING SYSTEM Corrective Action § 275.19... data available through program management tools and other sources. (c) In instances where the State...

7 CFR 275.19 - Monitoring and evaluation.

Code of Federal Regulations, 2011 CFR

2011-01-01

... AGRICULTURE FOOD STAMP AND FOOD DISTRIBUTION PROGRAM PERFORMANCE REPORTING SYSTEM Corrective Action § 275.19... data available through program management tools and other sources. (c) In instances where the State...

7 CFR 275.19 - Monitoring and evaluation.

Code of Federal Regulations, 2013 CFR

2013-01-01

... AGRICULTURE FOOD STAMP AND FOOD DISTRIBUTION PROGRAM PERFORMANCE REPORTING SYSTEM Corrective Action § 275.19... data available through program management tools and other sources. (c) In instances where the State...

7 CFR 275.19 - Monitoring and evaluation.

Code of Federal Regulations, 2014 CFR

2014-01-01

... AGRICULTURE FOOD STAMP AND FOOD DISTRIBUTION PROGRAM PERFORMANCE REPORTING SYSTEM Corrective Action § 275.19... data available through program management tools and other sources. (c) In instances where the State...

Evaluation of a French medical multi-terminology indexer for the manual annotation of natural language medical reports of healthcare-associated infections.

PubMed

Sakji, Saoussen; Gicquel, Quentin; Pereira, Suzanne; Kergourlay, Ivan; Proux, Denys; Darmoni, Stéfan; Metzger, Marie-Hélène

2010-01-01

Surveillance of healthcare-associated infections is essential to prevention. A new collaborative project, namely ALADIN, was launched in January 2009 and aims to develop an automated detection tool based on natural language processing of medical documents. The objective of this study was to evaluate the annotation of natural language medical reports of healthcare-associated infections. A software MS Access application (NosIndex) has been developed to interface ECMT XML answer and manual annotation work. ECMT performances were evaluated by an infection control practitioner (ICP). Precision was evaluated for the 2 modules and recall only for the default module. Exclusion rate was defined as ratio between medical terms not found by ECMT and total number of terms evaluated. The medical discharge summaries were randomly selected in 4 medical wards. From the 247 medical terms evaluated, ECMT proposed 428 and 3,721 codes, respectively for the default and expansion modules. The precision was higher with the default module (P1=0.62) than with the expansion (P2=0.47). Performances of ECMT as support tool for the medical annotation were satisfactory.

Engine System Model Development for Nuclear Thermal Propulsion

NASA Technical Reports Server (NTRS)

Nelson, Karl W.; Simpson, Steven P.

2006-01-01

In order to design, analyze, and evaluate conceptual Nuclear Thermal Propulsion (NTP) engine systems, an improved NTP design and analysis tool has been developed. The NTP tool utilizes the Rocket Engine Transient Simulation (ROCETS) system tool and many of the routines from the Enabler reactor model found in Nuclear Engine System Simulation (NESS). Improved non-nuclear component models and an external shield model were added to the tool. With the addition of a nearly complete system reliability model, the tool will provide performance, sizing, and reliability data for NERVA-Derived NTP engine systems. A new detailed reactor model is also being developed and will replace Enabler. The new model will allow more flexibility in reactor geometry and include detailed thermal hydraulics and neutronics models. A description of the reactor, component, and reliability models is provided. Another key feature of the modeling process is the use of comprehensive spreadsheets for each engine case. The spreadsheets include individual worksheets for each subsystem with data, plots, and scaled figures, making the output very useful to each engineering discipline. Sample performance and sizing results with the Enabler reactor model are provided including sensitivities. Before selecting an engine design, all figures of merit must be considered including the overall impacts on the vehicle and mission. Evaluations based on key figures of merit of these results and results with the new reactor model will be performed. The impacts of clustering and external shielding will also be addressed. Over time, the reactor model will be upgraded to design and analyze other NTP concepts with CERMET and carbide fuel cores.

Dental Students' Perceptions of Digital Assessment Software for Preclinical Tooth Preparation Exercises.

PubMed

Park, Carly F; Sheinbaum, Justin M; Tamada, Yasushi; Chandiramani, Raina; Lian, Lisa; Lee, Cliff; Da Silva, John; Ishikawa-Nagai, Shigemi

2017-05-01

Objective self-assessment is essential to learning and continued competence in dentistry. A computer-assisted design/computer-assisted manufacturing (CAD/CAM) learning software (prepCheck, Sirona) allows students to objectively assess their performance in preclinical prosthodontics. The aim of this study was to evaluate students' perceptions of CAD/CAM learning software for preclinical prosthodontics exercises. In 2014, all third-year dental students at Harvard School of Dental Medicine (n=36) were individually instructed by a trained faculty member in using prepCheck. Each student completed a preclinical formative exercise (#18) and summative examination (#30) for ceramometal crown preparation and evaluated the preparation using five assessment tools (reduction, margin width, surface finish, taper, and undercut) in prepCheck. The students then rated each of the five tools for usefulness, user-friendliness, and frequency of use on a scale from 1=lowest to 5=highest. Faculty members graded the tooth preparations as pass (P), marginal-pass (MP), or fail (F). The survey response rate was 100%. The tools for undercut and taper had the highest scores for usefulness, user-friendliness, and frequency of use. The reduction tool score was significantly lower in all categories (p<0.01). There were significant differences in usefulness (p<0.05) and user-friendliness (p<0.05) scores among the P, MP, and F groups. These results suggest that the prepCheck taper and undercut tools were useful for the students' learning process in a preclinical exercise. The students' perceptions of prepCheck and their preclinical performance were related, and those students who performed poorest rated the software as significantly more useful.

Performance of a computer-based assessment of cognitive function measures in two cohorts of seniors

USDA-ARS?s Scientific Manuscript database

Computer-administered assessment of cognitive function is being increasingly incorporated in clinical trials, however its performance in these settings has not been systematically evaluated. The Seniors Health and Activity Research Program (SHARP) pilot trial (N=73) developed a computer-based tool f...

An overview of the BioCreative 2012 Workshop Track III: interactive text mining task

PubMed Central

Arighi, Cecilia N.; Carterette, Ben; Cohen, K. Bretonnel; Krallinger, Martin; Wilbur, W. John; Fey, Petra; Dodson, Robert; Cooper, Laurel; Van Slyke, Ceri E.; Dahdul, Wasila; Mabee, Paula; Li, Donghui; Harris, Bethany; Gillespie, Marc; Jimenez, Silvia; Roberts, Phoebe; Matthews, Lisa; Becker, Kevin; Drabkin, Harold; Bello, Susan; Licata, Luana; Chatr-aryamontri, Andrew; Schaeffer, Mary L.; Park, Julie; Haendel, Melissa; Van Auken, Kimberly; Li, Yuling; Chan, Juancarlos; Muller, Hans-Michael; Cui, Hong; Balhoff, James P.; Chi-Yang Wu, Johnny; Lu, Zhiyong; Wei, Chih-Hsuan; Tudor, Catalina O.; Raja, Kalpana; Subramani, Suresh; Natarajan, Jeyakumar; Cejuela, Juan Miguel; Dubey, Pratibha; Wu, Cathy

2013-01-01

In many databases, biocuration primarily involves literature curation, which usually involves retrieving relevant articles, extracting information that will translate into annotations and identifying new incoming literature. As the volume of biological literature increases, the use of text mining to assist in biocuration becomes increasingly relevant. A number of groups have developed tools for text mining from a computer science/linguistics perspective, and there are many initiatives to curate some aspect of biology from the literature. Some biocuration efforts already make use of a text mining tool, but there have not been many broad-based systematic efforts to study which aspects of a text mining tool contribute to its usefulness for a curation task. Here, we report on an effort to bring together text mining tool developers and database biocurators to test the utility and usability of tools. Six text mining systems presenting diverse biocuration tasks participated in a formal evaluation, and appropriate biocurators were recruited for testing. The performance results from this evaluation indicate that some of the systems were able to improve efficiency of curation by speeding up the curation task significantly (∼1.7- to 2.5-fold) over manual curation. In addition, some of the systems were able to improve annotation accuracy when compared with the performance on the manually curated set. In terms of inter-annotator agreement, the factors that contributed to significant differences for some of the systems included the expertise of the biocurator on the given curation task, the inherent difficulty of the curation and attention to annotation guidelines. After the task, annotators were asked to complete a survey to help identify strengths and weaknesses of the various systems. The analysis of this survey highlights how important task completion is to the biocurators’ overall experience of a system, regardless of the system’s high score on design, learnability and usability. In addition, strategies to refine the annotation guidelines and systems documentation, to adapt the tools to the needs and query types the end user might have and to evaluate performance in terms of efficiency, user interface, result export and traditional evaluation metrics have been analyzed during this task. This analysis will help to plan for a more intense study in BioCreative IV. PMID:23327936

An overview of the BioCreative 2012 Workshop Track III: interactive text mining task.

PubMed

Arighi, Cecilia N; Carterette, Ben; Cohen, K Bretonnel; Krallinger, Martin; Wilbur, W John; Fey, Petra; Dodson, Robert; Cooper, Laurel; Van Slyke, Ceri E; Dahdul, Wasila; Mabee, Paula; Li, Donghui; Harris, Bethany; Gillespie, Marc; Jimenez, Silvia; Roberts, Phoebe; Matthews, Lisa; Becker, Kevin; Drabkin, Harold; Bello, Susan; Licata, Luana; Chatr-aryamontri, Andrew; Schaeffer, Mary L; Park, Julie; Haendel, Melissa; Van Auken, Kimberly; Li, Yuling; Chan, Juancarlos; Muller, Hans-Michael; Cui, Hong; Balhoff, James P; Chi-Yang Wu, Johnny; Lu, Zhiyong; Wei, Chih-Hsuan; Tudor, Catalina O; Raja, Kalpana; Subramani, Suresh; Natarajan, Jeyakumar; Cejuela, Juan Miguel; Dubey, Pratibha; Wu, Cathy

2013-01-01

In many databases, biocuration primarily involves literature curation, which usually involves retrieving relevant articles, extracting information that will translate into annotations and identifying new incoming literature. As the volume of biological literature increases, the use of text mining to assist in biocuration becomes increasingly relevant. A number of groups have developed tools for text mining from a computer science/linguistics perspective, and there are many initiatives to curate some aspect of biology from the literature. Some biocuration efforts already make use of a text mining tool, but there have not been many broad-based systematic efforts to study which aspects of a text mining tool contribute to its usefulness for a curation task. Here, we report on an effort to bring together text mining tool developers and database biocurators to test the utility and usability of tools. Six text mining systems presenting diverse biocuration tasks participated in a formal evaluation, and appropriate biocurators were recruited for testing. The performance results from this evaluation indicate that some of the systems were able to improve efficiency of curation by speeding up the curation task significantly (∼1.7- to 2.5-fold) over manual curation. In addition, some of the systems were able to improve annotation accuracy when compared with the performance on the manually curated set. In terms of inter-annotator agreement, the factors that contributed to significant differences for some of the systems included the expertise of the biocurator on the given curation task, the inherent difficulty of the curation and attention to annotation guidelines. After the task, annotators were asked to complete a survey to help identify strengths and weaknesses of the various systems. The analysis of this survey highlights how important task completion is to the biocurators' overall experience of a system, regardless of the system's high score on design, learnability and usability. In addition, strategies to refine the annotation guidelines and systems documentation, to adapt the tools to the needs and query types the end user might have and to evaluate performance in terms of efficiency, user interface, result export and traditional evaluation metrics have been analyzed during this task. This analysis will help to plan for a more intense study in BioCreative IV.

Evaluating the performance of a fault detection and diagnostic system for vapor compression equipment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Breuker, M.S.; Braun, J.E.

This paper presents a detailed evaluation of the performance of a statistical, rule-based fault detection and diagnostic (FDD) technique presented by Rossi and Braun (1997). Steady-state and transient tests were performed on a simple rooftop air conditioner over a range of conditions and fault levels. The steady-state data without faults were used to train models that predict outputs for normal operation. The transient data with faults were used to evaluate FDD performance. The effect of a number of design variables on FDD sensitivity for different faults was evaluated and two prototype systems were specified for more complete evaluation. Good performancemore » was achieved in detecting and diagnosing five faults using only six temperatures (2 input and 4 output) and linear models. The performance improved by about a factor of two when ten measurements (three input and seven output) and higher order models were used. This approach for evaluating and optimizing the performance of the statistical, rule-based FDD technique could be used as a design and evaluation tool when applying this FDD method to other packaged air-conditioning systems. Furthermore, the approach could also be modified to evaluate the performance of other FDD methods.« less

Evaluation of a telerobotic system to assist surgeons in microsurgery

NASA Technical Reports Server (NTRS)

Das, H.; Zak, H.; Johnson, J.; Crouch, J.; Frambach, D.

1999-01-01

A tool was developed that assists surgeons in manipulating surgical instruments more precisely than is possible manually. The tool is a telemanipulator that scales down the surgeon's hand motion and filters tremor in the motion. The signals measured from the surgeon's hand are transformed and used to drive a six-degrees-of-freedom robot to position the surgical instrument mounted on its tip. A pilot study comparing the performance of the telemanipulator system against manual instrument positioning was conducted at the University of Southern California School of Medicine. The results show that a telerobotic tool can improve the performance of a microsurgeon by increasing the precision with which he can position surgical instruments, but this is achieved at the cost of increased time in performing the task. We believe that this technology will extend the capabilities of microsurgeons and allow more surgeons to perform highly skilled procedures currently performed only by the best surgeons. It will also enable performance of new surgical procedures that are beyond the capabilities of even the most skilled surgeons. Copyright 1999 Wiley-Liss, Inc.

Objective Assessment of Laparoscopic Force and Psychomotor Skills in a Novel Virtual Reality-Based Haptic Simulator.

PubMed

Prasad, M S Raghu; Manivannan, Muniyandi; Manoharan, Govindan; Chandramohan, S M

2016-01-01

Most of the commercially available virtual reality-based laparoscopic simulators do not effectively evaluate combined psychomotor and force-based laparoscopic skills. Consequently, the lack of training on these critical skills leads to intraoperative errors. To assess the effectiveness of the novel virtual reality-based simulator, this study analyzed the combined psychomotor (i.e., motion or movement) and force skills of residents and expert surgeons. The study also examined the effectiveness of real-time visual force feedback and tool motion during training. Bimanual fundamental (i.e., probing, pulling, sweeping, grasping, and twisting) and complex tasks (i.e., tissue dissection) were evaluated. In both tasks, visual feedback on applied force and tool motion were provided. The skills of the participants while performing the early tasks were assessed with and without visual feedback. Participants performed 5 repetitions of fundamental and complex tasks. Reaction force and instrument acceleration were used as metrics. Surgical Gastroenterology, Government Stanley Medical College and Hospital; Institute of Surgical Gastroenterology, Madras Medical College and Rajiv Gandhi Government General Hospital. Residents (N = 25; postgraduates and surgeons with <2 years of laparoscopic surgery) and expert surgeons (N = 25; surgeons with >4 and ≤10 years of laparoscopic surgery). Residents applied large forces compared with expert surgeons and performed abrupt tool movements (p < 0.001). However, visual + haptic feedback improved the performance of residents (p < 0.001). In complex tasks, visual + haptic feedback did not influence the applied force of expert surgeons, but influenced their tool motion (p < 0.001). Furthermore, in complex tissue sweeping task, expert surgeons applied more force, but were within the tissue damage limits. In both groups, exertion of large forces and abrupt tool motion were observed during grasping, probing or pulling, and tissue sweeping maneuvers (p < 0.001). Modern day curriculum-based training should evaluate the skills of residents with robust force and psychomotor-based exercises for proficient laparoscopy. Visual feedback on force and motion during training has the potential to enhance the learning curve of residents. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

An Intelligent Tool for Activity Data Collection

PubMed Central

Jehad Sarkar, A. M.

2011-01-01

Activity recognition systems using simple and ubiquitous sensors require a large variety of real-world sensor data for not only evaluating their performance but also training the systems for better functioning. However, a tremendous amount of effort is required to setup an environment for collecting such data. For example, expertise and resources are needed to design and install the sensors, controllers, network components, and middleware just to perform basic data collections. It is therefore desirable to have a data collection method that is inexpensive, flexible, user-friendly, and capable of providing large and diverse activity datasets. In this paper, we propose an intelligent activity data collection tool which has the ability to provide such datasets inexpensively without physically deploying the testbeds. It can be used as an inexpensive and alternative technique to collect human activity data. The tool provides a set of web interfaces to create a web-based activity data collection environment. It also provides a web-based experience sampling tool to take the user’s activity input. The tool generates an activity log using its activity knowledge and the user-given inputs. The activity knowledge is mined from the web. We have performed two experiments to validate the tool’s performance in producing reliable datasets. PMID:22163832

Combining web-based tools for transparent evaluation of data for risk assessment: developmental effects of bisphenol A on the mammary gland as a case study.

PubMed

Molander, Linda; Hanberg, Annika; Rudén, Christina; Ågerstrand, Marlene; Beronius, Anna

2017-03-01

Different tools have been developed that facilitate systematic and transparent evaluation and handling of toxicity data in the risk assessment process. The present paper sets out to explore the combined use of two web-based tools for study evaluation and identification of reliable data relevant to health risk assessment. For this purpose, a case study was performed using in vivo toxicity studies investigating low-dose effects of bisphenol A on mammary gland development. The reliability of the mammary gland studies was evaluated using the Science in Risk Assessment and Policy (SciRAP) criteria for toxicity studies. The Health Assessment Workspace Collaborative (HAWC) was used for characterizing and visualizing the mammary gland data in terms of type of effects investigated and reported, and the distribution of these effects within the dose interval. It was then investigated whether there was any relationship between study reliability and the type of effects reported and/or their distribution in the dose interval. The combination of the SciRAP and HAWC tools allowed for transparent evaluation and visualization of the studies investigating developmental effects of BPA on the mammary gland. The use of these tools showed that there were no apparent differences in the type of effects and their distribution in the dose interval between the five studies assessed as most reliable and the whole data set. Combining the SciRAP and HAWC tools was found to be a useful approach for evaluating in vivo toxicity studies and identifying reliable and sensitive information relevant to regulatory risk assessment of chemicals. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Stochastic Simulation Tool for Aerospace Structural Analysis

NASA Technical Reports Server (NTRS)

Knight, Norman F.; Moore, David F.

2006-01-01

Stochastic simulation refers to incorporating the effects of design tolerances and uncertainties into the design analysis model and then determining their influence on the design. A high-level evaluation of one such stochastic simulation tool, the MSC.Robust Design tool by MSC.Software Corporation, has been conducted. This stochastic simulation tool provides structural analysts with a tool to interrogate their structural design based on their mathematical description of the design problem using finite element analysis methods. This tool leverages the analyst's prior investment in finite element model development of a particular design. The original finite element model is treated as the baseline structural analysis model for the stochastic simulations that are to be performed. A Monte Carlo approach is used by MSC.Robust Design to determine the effects of scatter in design input variables on response output parameters. The tool was not designed to provide a probabilistic assessment, but to assist engineers in understanding cause and effect. It is driven by a graphical-user interface and retains the engineer-in-the-loop strategy for design evaluation and improvement. The application problem for the evaluation is chosen to be a two-dimensional shell finite element model of a Space Shuttle wing leading-edge panel under re-entry aerodynamic loading. MSC.Robust Design adds value to the analysis effort by rapidly being able to identify design input variables whose variability causes the most influence in response output parameters.

Simulation for Prediction of Entry Article Demise (SPEAD): An Analysis Tool for Spacecraft Safety Analysis and Ascent/Reentry Risk Assessment

NASA Technical Reports Server (NTRS)

Ling, Lisa

2014-01-01

For the purpose of performing safety analysis and risk assessment for a potential off-nominal atmospheric reentry resulting in vehicle breakup, a synthesis of trajectory propagation coupled with thermal analysis and the evaluation of node failure is required to predict the sequence of events, the timeline, and the progressive demise of spacecraft components. To provide this capability, the Simulation for Prediction of Entry Article Demise (SPEAD) analysis tool was developed. The software and methodology have been validated against actual flights, telemetry data, and validated software, and safety/risk analyses were performed for various programs using SPEAD. This report discusses the capabilities, modeling, validation, and application of the SPEAD analysis tool.

Development and optimization of the Suna trap as a tool for mosquito monitoring and control

PubMed Central

2014-01-01

Background Monitoring of malaria vector populations provides information about disease transmission risk, as well as measures of the effectiveness of vector control. The Suna trap is introduced and evaluated with regard to its potential as a new, standardized, odour-baited tool for mosquito monitoring and control. Methods Dual-choice experiments with female Anopheles gambiae sensu lato in a laboratory room and semi-field enclosure, were used to compare catch rates of odour-baited Suna traps and MM-X traps. The relative performance of the Suna trap, CDC light trap and MM-X trap as monitoring tools was assessed inside a human-occupied experimental hut in a semi-field enclosure. Use of the Suna trap as a tool to prevent mosquito house entry was also evaluated in the semi-field enclosure. The optimal hanging height of Suna traps was determined by placing traps at heights ranging from 15 to 105 cm above ground outside houses in western Kenya. Results In the laboratory the mean proportion of An. gambiae s.l. caught in the Suna trap was 3.2 times greater than the MM-X trap (P < 0.001), but the traps performed equally in semi-field conditions (P = 0.615). As a monitoring tool , the Suna trap outperformed an unlit CDC light trap (P < 0.001), but trap performance was equal when the CDC light trap was illuminated (P = 0.127). Suspending a Suna trap outside an experimental hut reduced entry rates by 32.8% (P < 0.001). Under field conditions, suspending the trap at 30 cm above ground resulted in the greatest catch sizes (mean 25.8 An. gambiae s.l. per trap night). Conclusions The performance of the Suna trap equals that of the CDC light trap and MM-X trap when used to sample An. gambiae inside a human-occupied house under semi-field conditions. The trap is effective in sampling mosquitoes outside houses in the field, and the use of a synthetic blend of attractants negates the requirement of a human bait. Hanging a Suna trap outside a house can reduce An. gambiae house entry and its use as a novel tool for reducing malaria transmission risk will be evaluated in peri-domestic settings in sub-Saharan Africa. PMID:24998771

Evaluating auditory perception and communication demands required to carry out work tasks and complimentary hearing resources and skills for older workers with hearing loss.

PubMed

Jennings, M B; Shaw, L; Hodgins, H; Kuchar, D A; Bataghva, L Poost-Foroosh

2010-01-01

For older workers with acquired hearing loss, this loss as well as the changing nature of work and the workforce, may lead to difficulties and disadvantages in obtaining and maintaining employment. Currently there are very few instruments that can assist workplaces, employers and workers to prepare for older workers with hearing loss or with the evaluation of auditory perception demands of work, especially those relevant to communication, and safety sensitive workplaces that require high levels of communication. This paper introduces key theoretical considerations that informed the development of a new framework, The Audiologic Ergonomic (AE) Framework to guide audiologists, work rehabilitation professionals and workers in developing tools to support the identification and evaluation of auditory perception demands in the workplace, the challenges to communication and the subsequent productivity and safety in the performance of work duties by older workers with hearing loss. The theoretical concepts underpinning this framework are discussed along with next steps in developing tools such as the Canadian Hearing Demands Tool (C-HearD Tool) in advancing approaches to evaluate auditory perception and communication demands in the workplace.

Evaluation of a Tactical Surface Metering Tool for Charlotte Douglas International Airport via Human-in-the-Loop Simulation

NASA Technical Reports Server (NTRS)

Verma, Savita; Lee, Hanbong; Dulchinos, Victoria L.; Martin, Lynne; Stevens, Lindsay; Jung, Yoon; Chevalley, Eric; Jobe, Kim; Parke, Bonny

2017-01-01

NASA has been working with the FAA and aviation industry partners to develop and demonstrate new concepts and technologies that integrate arrival, departure, and surface traffic management capabilities. In March 2017, NASA conducted a human-in-the-loop (HITL) simulation for integrated surface and airspace operations, modeling Charlotte Douglas International Airport, to evaluate the operational procedures and information requirements for the tactical surface metering tool, and data exchange elements between the airline controlled ramp and ATC Tower. In this paper, we focus on the calibration of the tactical surface metering tool using various metrics measured from the HITL simulation results. Key performance metrics include gate hold times from pushback advisories, taxi-in-out times, runway throughput, and departure queue size. Subjective metrics presented in this paper include workload, situational awareness, and acceptability of the metering tool and its calibration.

Evaluation of a Tactical Surface Metering Tool for Charlotte Douglas International Airport Via Human-in-the-Loop Simulation

NASA Technical Reports Server (NTRS)

Verma, Savita; Lee, Hanbong; Martin, Lynne; Stevens, Lindsay; Jung, Yoon; Dulchinos, Victoria; Chevalley, Eric; Jobe, Kim; Parke, Bonny

2017-01-01

NASA has been working with the FAA and aviation industry partners to develop and demonstrate new concepts and technologies that integrate arrival, departure, and surface traffic management capabilities. In March 2017, NASA conducted a human-in-the-loop (HITL) simulation for integrated surface and airspace operations, modeling Charlotte Douglas International Airport, to evaluate the operational procedures and information requirements for the tactical surface metering tool, and data exchange elements between the airline controlled ramp and ATC Tower. In this paper, we focus on the calibration of the tactical surface metering tool using various metrics measured from the HITL simulation results. Key performance metrics include gate hold times from pushback advisories, taxi-in/out times, runway throughput, and departure queue size. Subjective metrics presented in this paper include workload, situational awareness, and acceptability of the metering tool and its calibration

Forward impact extrusion of surface textured steel blanks using coated tooling

NASA Astrophysics Data System (ADS)

Hild, Rafael; Feuerhack, Andreas; Trauth, Daniel; Arghavani, Mostafa; Kruppe, Nathan C.; Brögelmann, Tobias; Bobzin, Kirsten; Klocke, Fritz

2017-10-01

A method to enable dry metal forming by the means of a self-lubricating coating and surface textures was researched using an innovative Pin-On-Cylinder-Tribometer. The experimental analysis was complemented by a numerical model of the complex contact conditions between coated tools and the surface textured specimen at the micro-level. Based on the results, the explanation of the tribological interactions between surface textured specimens and the tool in dry full forward extrusion is the objective of this work. Therefore, experimental dry extrusion tests were performed using a tool system. The extruded specimens were evaluated regarding their geometry as well as by the required punch force. Thereby, the effectiveness and the feasibility of dry metal forming on the example of full forward extrusion was evaluated. Thus, one more step towards the technical realization of dry metal forming of low alloy steels under industrial conditions was realized.

Rapid Accurate Identification of Tuberculous Meningitis Among South African Children Using a Novel Clinical Decision Tool.

PubMed

Goenka, Anu; Jeena, Prakash M; Mlisana, Koleka; Solomon, Tom; Spicer, Kevin; Stephenson, Rebecca; Verma, Arpana; Dhada, Barnesh; Griffiths, Michael J

2018-03-01

Early diagnosis of tuberculous meningitis (TBM) is crucial to achieve optimum outcomes. There is no effective rapid diagnostic test for use in children. We aimed to develop a clinical decision tool to facilitate the early diagnosis of childhood TBM. Retrospective case-control study was performed across 7 hospitals in KwaZulu-Natal, South Africa (2010-2014). We identified the variables most predictive of microbiologically confirmed TBM in children (3 months to 15 years) by univariate analysis. These variables were modelled into a clinical decision tool and performance tested on an independent sample group. Of 865 children with suspected TBM, 3% (25) were identified with microbiologically confirmed TBM. Clinical information was retrieved for 22 microbiologically confirmed cases of TBM and compared with 66 controls matched for age, ethnicity, sex and geographical origin. The 9 most predictive variables among the confirmed cases were used to develop a clinical decision tool (CHILD TB LP): altered Consciousness; caregiver HIV infected; Illness length >7 days; Lethargy; focal neurologic Deficit; failure to Thrive; Blood/serum sodium <132 mmol/L; CSF >10 Lymphocytes ×10/L; CSF Protein >0.65 g/L. This tool successfully classified an independent sample of 7 cases and 21 controls with a sensitivity of 100% and specificity of 90%. The CHILD TB LP decision tool accurately classified microbiologically confirmed TBM. We propose that CHILD TB LP is prospectively evaluated as a novel rapid diagnostic tool for use in the initial evaluation of children with suspected neurologic infection presenting to hospitals in similar settings.

Program audit, A management tool

NASA Technical Reports Server (NTRS)

Miller, T. J.

1971-01-01

Program gives in-depth view of organizational performance at all levels of the management structure, and provides means by which managers can effectively and efficiently evaluate adequacy of management direction, policies, and procedures.

A comprehensive evaluation of assembly scaffolding tools

PubMed Central

2014-01-01

Background Genome assembly is typically a two-stage process: contig assembly followed by the use of paired sequencing reads to join contigs into scaffolds. Scaffolds are usually the focus of reported assembly statistics; longer scaffolds greatly facilitate the use of genome sequences in downstream analyses, and it is appealing to present larger numbers as metrics of assembly performance. However, scaffolds are highly prone to errors, especially when generated using short reads, which can directly result in inflated assembly statistics. Results Here we provide the first independent evaluation of scaffolding tools for second-generation sequencing data. We find large variations in the quality of results depending on the tool and dataset used. Even extremely simple test cases of perfect input, constructed to elucidate the behaviour of each algorithm, produced some surprising results. We further dissect the performance of the scaffolders using real and simulated sequencing data derived from the genomes of Staphylococcus aureus, Rhodobacter sphaeroides, Plasmodium falciparum and Homo sapiens. The results from simulated data are of high quality, with several of the tools producing perfect output. However, at least 10% of joins remains unidentified when using real data. Conclusions The scaffolders vary in their usability, speed and number of correct and missed joins made between contigs. Results from real data highlight opportunities for further improvements of the tools. Overall, SGA, SOPRA and SSPACE generally outperform the other tools on our datasets. However, the quality of the results is highly dependent on the read mapper and genome complexity. PMID:24581555

Employee Performance in the Context of the Problems of Measurement and Evaluation in Practice

NASA Astrophysics Data System (ADS)

Szabó, Peter; Mĺkva, Miroslava; Vaňová, Jaromíra; Marková, Petra

2017-09-01

Employee performance is a condition and an assumption for the performance and success of a company on the market. In order to ensure competitive ability, the quality of human resources, their management, and related measurement and performance assessment are at the forefront of company interest. Employee assessment affects the performance, development and motivation of people and also provides the necessary information about the employees. It allows the organization to monitor employee performance and compare their work with other collaborators. Many companies have the problem of setting up evaluation system so that it carried itself elements of responsibility and objectivity. The result of conceptual work in this area is the ultimate use of tools whose deployment, if possible, motivates employees to perform better. The aim of the paper is to refer to problems that arise in companies in evaluating the performance of employees.

PERTS: A Prototyping Environment for Real-Time Systems

NASA Technical Reports Server (NTRS)

Liu, Jane W. S.; Lin, Kwei-Jay; Liu, C. L.

1993-01-01

PERTS is a prototyping environment for real-time systems. It is being built incrementally and will contain basic building blocks of operating systems for time-critical applications, tools, and performance models for the analysis, evaluation and measurement of real-time systems and a simulation/emulation environment. It is designed to support the use and evaluation of new design approaches, experimentations with alternative system building blocks, and the analysis and performance profiling of prototype real-time systems.

Reliable assessment of laparoscopic performance in the operating room using videotape analysis.

PubMed

Chang, Lily; Hogle, Nancy J; Moore, Brianna B; Graham, Mark J; Sinanan, Mika N; Bailey, Robert; Fowler, Dennis L

2007-06-01

The Global Operative Assessment of Laparoscopic Skills (GOALS) is a valid assessment tool for objectively evaluating the technical performance of laparoscopic skills in surgery residents. We hypothesized that GOALS would reliably differentiate between an experienced (expert) and an inexperienced (novice) laparoscopic surgeon (construct validity) based on a blinded videotape review of a laparoscopic cholecystectomy procedure. Ten board-certified surgeons actively engaged in the practice and teaching of laparoscopy reviewed and evaluated the videotaped operative performance of one novice and one expert laparoscopic surgeon using GOALS. Each reviewer recorded a score for both the expert and the novice videotape reviews in each of the 5 domains in GOALS (depth perception, bimanual dexterity, efficiency, tissue handling, and overall competence). The scores for the expert and the novice were compared and statistically analyzed using single-factor analysis of variance (ANOVA). The expert scored significantly higher than the novice did in the domains of depth perception (p = .005), bimanual dexterity (p = .001), efficiency (p = .001), and overall competence ( p = .001). Interrater reliability for the reviewers of the novice tape was Cronbach alpha = .93 and the expert tape was Cronbach alpha = .87. There was no difference between the two for tissue handling. The Global Operative Assessment of Laparoscopic Skills is a valid, objective assessment tool for evaluating technical surgical performance when used to blindly evaluate an intraoperative videotape recording of a laparoscopic procedure.

Modeling and Evaluating Pilot Performance in NextGen: Review of and Recommendations Regarding Pilot Modeling Efforts, Architectures, and Validation Studies

NASA Technical Reports Server (NTRS)

Wickens, Christopher; Sebok, Angelia; Keller, John; Peters, Steve; Small, Ronald; Hutchins, Shaun; Algarin, Liana; Gore, Brian Francis; Hooey, Becky Lee; Foyle, David C.

2013-01-01

NextGen operations are associated with a variety of changes to the national airspace system (NAS) including changes to the allocation of roles and responsibilities among operators and automation, the use of new technologies and automation, additional information presented on the flight deck, and the entire concept of operations (ConOps). In the transition to NextGen airspace, aviation and air operations designers need to consider the implications of design or system changes on human performance and the potential for error. To ensure continued safety of the NAS, it will be necessary for researchers to evaluate design concepts and potential NextGen scenarios well before implementation. One approach for such evaluations is through human performance modeling. Human performance models (HPMs) provide effective tools for predicting and evaluating operator performance in systems. HPMs offer significant advantages over empirical, human-in-the-loop testing in that (1) they allow detailed analyses of systems that have not yet been built, (2) they offer great flexibility for extensive data collection, (3) they do not require experimental participants, and thus can offer cost and time savings. HPMs differ in their ability to predict performance and safety with NextGen procedures, equipment and ConOps. Models also vary in terms of how they approach human performance (e.g., some focus on cognitive processing, others focus on discrete tasks performed by a human, while others consider perceptual processes), and in terms of their associated validation efforts. The objectives of this research effort were to support the Federal Aviation Administration (FAA) in identifying HPMs that are appropriate for predicting pilot performance in NextGen operations, to provide guidance on how to evaluate the quality of different models, and to identify gaps in pilot performance modeling research, that could guide future research opportunities. This research effort is intended to help the FAA evaluate pilot modeling efforts and select the appropriate tools for future modeling efforts to predict pilot performance in NextGen operations.

Evaluation plan for space station network interface units

NASA Technical Reports Server (NTRS)

Weaver, Alfred C.

1990-01-01

Outlined here is a procedure for evaluating network interface units (NIUs) produced for the Space Station program. The procedures should be equally applicable to the data management system (DMS) testbed NIUs produced by Honeywell and IBM. The evaluation procedures are divided into four areas. Performance measurement tools are hardware and software that must be developed in order to evaluate NIU performance. Performance tests are a series of tests, each of which documents some specific characteristic of NIU and/or network performance. In general, these performance tests quantify the speed, capacity, latency, and reliability of message transmission under a wide variety of conditions. Functionality tests are a series of tests and code inspections that demonstrate the functionality of the particular subset of ISO protocols which have been implemented in a given NIU. Conformance tests are a series of tests which would expose whether or not selected features within the ISO protocols are present and interoperable.

New machine learning tools for predictive vegetation mapping after climate change: Bagging and Random Forest perform better than Regression Tree Analysis

Treesearch

L.R. Iverson; A.M. Prasad; A. Liaw

2004-01-01

More and better machine learning tools are becoming available for landscape ecologists to aid in understanding species-environment relationships and to map probable species occurrence now and potentially into the future. To thal end, we evaluated three statistical models: Regression Tree Analybib (RTA), Bagging Trees (BT) and Random Forest (RF) for their utility in...

Critical evaluation of reverse engineering tool Imagix 4D!

PubMed

Yadav, Rashmi; Patel, Ravindra; Kothari, Abhay

2016-01-01

The comprehension of legacy codes is difficult to understand. Various commercial reengineering tools are available that have unique working styles, and are equipped with their inherent capabilities and shortcomings. The focus of the available tools is in visualizing static behavior not the dynamic one. Therefore, it is difficult for people who work in software product maintenance, code understanding reengineering/reverse engineering. Consequently, the need for a comprehensive reengineering/reverse engineering tool arises. We found the usage of Imagix 4D to be good as it generates the maximum pictorial representations in the form of flow charts, flow graphs, class diagrams, metrics and, to a partial extent, dynamic visualizations. We evaluated Imagix 4D with the help of a case study involving a few samples of source code. The behavior of the tool was analyzed on multiple small codes and a large code gcc C parser. Large code evaluation was performed to uncover dead code, unstructured code, and the effect of not including required files at preprocessing level. The utility of Imagix 4D to prepare decision density and complexity metrics for a large code was found to be useful in getting to know how much reengineering is required. At the outset, Imagix 4D offered limitations in dynamic visualizations, flow chart separation (large code) and parsing loops. The outcome of evaluation will eventually help in upgrading Imagix 4D and posed a need of full featured tools in the area of software reengineering/reverse engineering. It will also help the research community, especially those who are interested in the realm of software reengineering tool building.

A tool for urban soundscape evaluation applying Support Vector Machines for developing a soundscape classification model.

PubMed

Torija, Antonio J; Ruiz, Diego P; Ramos-Ridao, Angel F

2014-06-01

To ensure appropriate soundscape management in urban environments, the urban-planning authorities need a range of tools that enable such a task to be performed. An essential step during the management of urban areas from a sound standpoint should be the evaluation of the soundscape in such an area. In this sense, it has been widely acknowledged that a subjective and acoustical categorization of a soundscape is the first step to evaluate it, providing a basis for designing or adapting it to match people's expectations as well. In this sense, this work proposes a model for automatic classification of urban soundscapes. This model is intended for the automatic classification of urban soundscapes based on underlying acoustical and perceptual criteria. Thus, this classification model is proposed to be used as a tool for a comprehensive urban soundscape evaluation. Because of the great complexity associated with the problem, two machine learning techniques, Support Vector Machines (SVM) and Support Vector Machines trained with Sequential Minimal Optimization (SMO), are implemented in developing model classification. The results indicate that the SMO model outperforms the SVM model in the specific task of soundscape classification. With the implementation of the SMO algorithm, the classification model achieves an outstanding performance (91.3% of instances correctly classified). © 2013 Elsevier B.V. All rights reserved.

Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia.

PubMed

Alkhayal, Abdullah; Aldhukair, Shahla; Alselaim, Nahar; Aldekhayel, Salah; Alhabdan, Sultan; Altaweel, Waleed; Magzoub, Mohi Elden; Zamakhshary, Mohammed

2012-01-01

After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors' perspective, and program directors' attitudes toward the available objective tools to assess technical skills. This study was a cross-sectional survey of surgical program directors (PDs). The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs' perspective and the PDs' attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated. Seventy-one (61%) participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative evaluation, whereas 15% perform only formative evaluations of their residents, and the remaining 22% conduct both summative and formative evaluations of their residents' technical skills. Operative portfolios are kept by 53% of programs. The percentage of programs with mechanisms for remediation is 29% (19 of 65). The survey showed that surgical training programs use different tools to assess surgical skills competency. Having a clear remediation mechanism was highly associated with reporting remediation, which reflects the capability to detect struggling residents. Surgical training leadership should invest more in standardizing the assessment of surgical skills.

Comparison of seven fall risk assessment tools in community-dwelling Korean older women.

PubMed

Kim, Taekyoung; Xiong, Shuping

2017-03-01

This study aimed to compare seven widely used fall risk assessment tools in terms of validity and practicality, and to provide a guideline for choosing appropriate fall risk assessment tools for elderly Koreans. Sixty community-dwelling Korean older women (30 fallers and 30 matched non-fallers) were evaluated. Performance measures of all tools were compared between the faller and non-faller groups through two sample t-tests. Receiver Operating Characteristic curves were generated with odds ratios for discriminant analysis. Results showed that four tools had significant discriminative power, and the shortened version of Falls Efficacy Scale (SFES) showed excellent discriminant validity, followed by Berg Balance Scale (BBS) with acceptable discriminant validity. The Mini Balance Evaluation System Test and Timed Up and Go, however, had limited discriminant validities. In terms of practicality, SFES was also excellent. These findings suggest that SFES is the most suitable tool for assessing the fall risks of community-dwelling Korean older women, followed by BBS. Practitioner Summary: There is no general guideline on which fall risk assessment tools are suitable for community-dwelling Korean older women. This study compared seven widely used assessment tools in terms of validity and practicality. Results suggested that the short Falls Efficacy Scale is the most suitable tool, followed by Berg Balance Scale.

Evaluation of a Micro-Force Sensing Handheld Robot for Vitreoretinal Surgery.

PubMed

Gonenc, Berk; Balicki, Marcin A; Handa, James; Gehlbach, Peter; Riviere, Cameron N; Taylor, Russell H; Iordachita, Iulian

2012-12-20

Highly accurate positioning is fundamental to the performance of vitreoretinal microsurgery. Of vitreoretinal procedures, membrane peeling is among the most prone to complications since extremely delicate manipulation of retinal tissue is required. Associated tool-to-tissue interaction forces are usually below the threshold of human perception, and the surgical tools are moved very slowly, within the 0.1-0.5 mm/s range. During the procedure, unintentional tool motion and excessive forces can easily give rise to vision loss or irreversible damage to the retina. A successful surgery includes two key features: controlled tremor-free tool motion and control of applied force. In this study, we present the potential benefits of a micro-force sensing robot in vitreoretinal surgery. Our main contribution is implementing fiber Bragg grating based force sensing in an active tremor canceling handheld micromanipulator, known as Micron, to measure tool-to-tissue interaction forces in real time. Implemented auditory sensory substitution assists in reducing and limiting forces. In order to test the functionality and performance, the force sensing Micron was evaluated in peeling experiments with adhesive bandages and with the inner shell membrane from chicken eggs. Our findings show that the combination of active tremor canceling together with auditory sensory substitution is the most promising aid that keeps peeling forces below 7 mN with a significant reduction in 2-20 Hz oscillations.

Tools for monitoring system suitability in LC MS/MS centric proteomic experiments.

PubMed

Bereman, Michael S

2015-03-01

With advances in liquid chromatography coupled to tandem mass spectrometry technologies combined with the continued goals of biomarker discovery, clinical applications of established biomarkers, and integrating large multiomic datasets (i.e. "big data"), there remains an urgent need for robust tools to assess instrument performance (i.e. system suitability) in proteomic workflows. To this end, several freely available tools have been introduced that monitor a number of peptide identification (ID) and/or peptide ID free metrics. Peptide ID metrics include numbers of proteins, peptides, or peptide spectral matches identified from a complex mixture. Peptide ID free metrics include retention time reproducibility, full width half maximum, ion injection times, and integrated peptide intensities. The main driving force in the development of these tools is to monitor both intra- and interexperiment performance variability and to identify sources of variation. The purpose of this review is to summarize and evaluate these tools based on versatility, automation, vendor neutrality, metrics monitored, and visualization capabilities. In addition, the implementation of a robust system suitability workflow is discussed in terms of metrics, type of standard, and frequency of evaluation along with the obstacles to overcome prior to incorporating a more proactive approach to overall quality control in liquid chromatography coupled to tandem mass spectrometry based proteomic workflows. © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Evaluation of the efficacy of six nutritional screening tools to predict malnutrition in the elderly.

PubMed

Poulia, Kalliopi-Anna; Yannakoulia, Mary; Karageorgou, Dimitra; Gamaletsou, Maria; Panagiotakos, Demosthenes B; Sipsas, Nikolaos V; Zampelas, Antonis

2012-06-01

Malnutrition in the elderly is a multifactorial problem, more prevalent in hospitals and care homes. The absence of a gold standard in evaluating nutritional risk led us to evaluate the efficacy of six nutritional screening tools used in the elderly. Two hundred forty eight elderly patients (129 men, 119 female women, aged 75.2 ± 8.5 years) were examined. Nutritional screening was performed on admission using the following tools: Nutritional Risk Index (NRI), Geriatric Nutritional Risk Index (GNRI), Subjective Global Assessment (SGA), Mini Nutritional Assessment - Screening Form (MNA-SF), Malnutrition Universal Screening Tool (MUST) and Nutritional Risk Screening 2002 (NRS 2002). A combined index for malnutrition was also calculated. Nutritional risk and/or malnutrition varied greatly, ranging from 47.2 to 97.6%, depending on the nutritional screening tool used. MUST was the most valid screening tool (validity coefficient = 0.766, CI 95%: 0.690-0.841), while SGA was in better agreement with the combined index (κ = 0.707, p = 0.000). NRS 2002 although was the highest in sensitivity (99.4%), it was the lowest in specificity (6.1%) and positive predictive value (68.2%). MUST seem to be the most valid in the evaluation of the risk for malnutrition in the elderly upon admission to the hospital. NRS 2002 was found to overestimate nutritional risk in the elderly. Copyright © 2011 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

A Business Analytics Software Tool for Monitoring and Predicting Radiology Throughput Performance.

PubMed

Jones, Stephen; Cournane, Seán; Sheehy, Niall; Hederman, Lucy

2016-12-01

Business analytics (BA) is increasingly being utilised by radiology departments to analyse and present data. It encompasses statistical analysis, forecasting and predictive modelling and is used as an umbrella term for decision support and business intelligence systems. The primary aim of this study was to determine whether utilising BA technologies could contribute towards improved decision support and resource management within radiology departments. A set of information technology requirements were identified with key stakeholders, and a prototype BA software tool was designed, developed and implemented. A qualitative evaluation of the tool was carried out through a series of semi-structured interviews with key stakeholders. Feedback was collated, and emergent themes were identified. The results indicated that BA software applications can provide visibility of radiology performance data across all time horizons. The study demonstrated that the tool could potentially assist with improving operational efficiencies and management of radiology resources.

External validation of Global Evaluative Assessment of Robotic Skills (GEARS).

PubMed

Aghazadeh, Monty A; Jayaratna, Isuru S; Hung, Andrew J; Pan, Michael M; Desai, Mihir M; Gill, Inderbir S; Goh, Alvin C

2015-11-01

We demonstrate the construct validity, reliability, and utility of Global Evaluative Assessment of Robotic Skills (GEARS), a clinical assessment tool designed to measure robotic technical skills, in an independent cohort using an in vivo animal training model. Using a cross-sectional observational study design, 47 voluntary participants were categorized as experts (>30 robotic cases completed as primary surgeon) or trainees. The trainee group was further divided into intermediates (≥5 but ≤30 cases) or novices (<5 cases). All participants completed a standardized in vivo robotic task in a porcine model. Task performance was evaluated by two expert robotic surgeons and self-assessed by the participants using the GEARS assessment tool. Kruskal-Wallis test was used to compare the GEARS performance scores to determine construct validity; Spearman's rank correlation measured interobserver reliability; and Cronbach's alpha was used to assess internal consistency. Performance evaluations were completed on nine experts and 38 trainees (14 intermediate, 24 novice). Experts demonstrated superior performance compared to intermediates and novices overall and in all individual domains (p < 0.0001). In comparing intermediates and novices, the overall performance difference trended toward significance (p = 0.0505), while the individual domains of efficiency and autonomy were significantly different between groups (p = 0.0280 and 0.0425, respectively). Interobserver reliability between expert ratings was confirmed with a strong correlation observed (r = 0.857, 95 % CI [0.691, 0.941]). Experts and participant scoring showed less agreement (r = 0.435, 95 % CI [0.121, 0.689] and r = 0.422, 95 % CI [0.081, 0.0672]). Internal consistency was excellent for experts and participants (α = 0.96, 0.98, 0.93). In an independent cohort, GEARS was able to differentiate between different robotic skill levels, demonstrating excellent construct validity. As a standardized assessment tool, GEARS maintained consistency and reliability for an in vivo robotic surgical task and may be applied for skills evaluation in a broad range of robotic procedures.

Constructing Benchmark Databases and Protocols for Medical Image Analysis: Diabetic Retinopathy

PubMed Central

Kauppi, Tomi; Kämäräinen, Joni-Kristian; Kalesnykiene, Valentina; Sorri, Iiris; Uusitalo, Hannu; Kälviäinen, Heikki

2013-01-01

We address the performance evaluation practices for developing medical image analysis methods, in particular, how to establish and share databases of medical images with verified ground truth and solid evaluation protocols. Such databases support the development of better algorithms, execution of profound method comparisons, and, consequently, technology transfer from research laboratories to clinical practice. For this purpose, we propose a framework consisting of reusable methods and tools for the laborious task of constructing a benchmark database. We provide a software tool for medical image annotation helping to collect class label, spatial span, and expert's confidence on lesions and a method to appropriately combine the manual segmentations from multiple experts. The tool and all necessary functionality for method evaluation are provided as public software packages. As a case study, we utilized the framework and tools to establish the DiaRetDB1 V2.1 database for benchmarking diabetic retinopathy detection algorithms. The database contains a set of retinal images, ground truth based on information from multiple experts, and a baseline algorithm for the detection of retinopathy lesions. PMID:23956787

Development of a smart timber bridge girder with fiber optic sensors

Treesearch

James Wacker; Ursula Deza; Brent M. Phares; Terry J. Wipf

2010-01-01

Past timber bridge evaluation and maintenance efforts in the USA have principally focused on the internal integrity of timber components using various non-destructive evaluation tools to supplement visual inspection data. This project is part of a comprehensive effort to develop smart structure concepts for improving the long-term performance, maintenance, and...

Projected outcomes of a public-industry partnership for enhancing corn nitrogen research and datasets

USDA-ARS?s Scientific Manuscript database

Research is needed over a wide geographic range of soil and weather scenarios to evaluate methods and tools for corn N fertilizer applications. The objectives of this research were to conduct standardized corn N rate response field studies to evaluate the performance of multiple public-domain N deci...

Simulating forage crop production in a northern climate with the Integrated Farm System Model

USDA-ARS?s Scientific Manuscript database

Whole-farm simulation models are useful tools for evaluating the effect of management practices and climate variability on the agro-environmental and economic performance of farms. A few process-based farm-scale models have been developed, but none have been evaluated in a northern region with a sho...

A practical assessment of physician biopsychosocial performance.

PubMed

Margalit, Alon Pa; Glick, Shimon M; Benbassat, Jochanan; Cohen, Ayala; Margolis, Carmi Z

2007-10-01

A biopsychosocial approach to care seems to improve patient satisfaction and health outcomes. Nevertheless, this approach is not widely practiced, possibly because its precepts have not been translated into observable skills. To identify the skill components of a biopsychosocial consultation and develop an tool for their evaluation. We approached three e-mail discussion groups of family physicians and pooled their responses to the question "what types of observed physician behavior would characterize a biopsychosocial consultation?" We received 35 responses describing 37 types of behavior, all of which seemed to cluster around one of three aspects: patient-centered interview; system-centered and family-centered approach to care; or problem-solving orientation. Using these categories, we developed a nine-item evaluation tool. We used the evaluation tool to score videotaped encounters of patients with two types of doctors: family physicians who were identified by peer ratings to have a highly biopsychosocial orientation (n = 9) or a highly biomedical approach (n = 4); and 44 general practitioners, before and after they had participated in a program that taught a biopsychosocial approach to care. The evaluation tool was found to demonstrate high reliability (alpha = 0.90) and acceptable interobserver variability. The average scores of the physicians with a highly biopsychosocial orientation were significantly higher than those of physicians with a highly biomedical approach. There were significant differences between the scores of the teaching-program participants before and after the program. A biopsychosocial approach to patient care can be characterized using a valid and easy-to-apply evaluation tool.

Biomimetic Dissolution: A Tool to Predict Amorphous Solid Dispersion Performance.

PubMed

Puppolo, Michael M; Hughey, Justin R; Dillon, Traciann; Storey, David; Jansen-Varnum, Susan

2017-11-01

The presented study describes the development of a membrane permeation non-sink dissolution method that can provide analysis of complete drug speciation and emulate the in vivo performance of poorly water-soluble Biopharmaceutical Classification System class II compounds. The designed membrane permeation methodology permits evaluation of free/dissolved/unbound drug from amorphous solid dispersion formulations with the use of a two-cell apparatus, biorelevant dissolution media, and a biomimetic polymer membrane. It offers insight into oral drug dissolution, permeation, and absorption. Amorphous solid dispersions of felodipine were prepared by hot melt extrusion and spray drying techniques and evaluated for in vitro performance. Prior to ranking performance of extruded and spray-dried felodipine solid dispersions, optimization of the dissolution methodology was performed for parameters such as agitation rate, membrane type, and membrane pore size. The particle size and zeta potential were analyzed during dissolution experiments to understand drug/polymer speciation and supersaturation sustainment of felodipine solid dispersions. Bland-Altman analysis was performed to measure the agreement or equivalence between dissolution profiles acquired using polymer membranes and porcine intestines and to establish the biomimetic nature of the treated polymer membranes. The utility of the membrane permeation dissolution methodology is seen during the evaluation of felodipine solid dispersions produced by spray drying and hot melt extrusion. The membrane permeation dissolution methodology can suggest formulation performance and be employed as a screening tool for selection of candidates to move forward to pharmacokinetic studies. Furthermore, the presented model is a cost-effective technique.

Underground coal mine instrumentation and test

NASA Technical Reports Server (NTRS)

Burchill, R. F.; Waldron, W. D.

1976-01-01

The need to evaluate mechanical performance of mine tools and to obtain test performance data from candidate systems dictate that an engineering data recording system be built. Because of the wide range of test parameters which would be evaluated, a general purpose data gathering system was designed and assembled to permit maximum versatility. A primary objective of this program was to provide a specific operating evaluation of a longwall mining machine vibration response under normal operating conditions. A number of mines were visited and a candidate for test evaluation was selected, based upon management cooperation, machine suitability, and mine conditions. Actual mine testing took place in a West Virginia mine.

Automated near-real-time clinical performance feedback for anesthesiology residents: one piece of the milestones puzzle.

PubMed

Ehrenfeld, Jesse M; McEvoy, Matthew D; Furman, William R; Snyder, Dylan; Sandberg, Warren S

2014-01-01

Anesthesiology residencies are developing trainee assessment tools to evaluate 25 milestones that map to the six core competencies. The effort will be facilitated by development of automated methods to capture, assess, and report trainee performance to program directors, the Accreditation Council for Graduate Medical Education and the trainees themselves. The authors leveraged a perioperative information management system to develop an automated, near-real-time performance capture and feedback tool that provides objective data on clinical performance and requires minimal administrative effort. Before development, the authors surveyed trainees about satisfaction with clinical performance feedback and about preferences for future feedback. Resident performance on 24,154 completed cases has been incorporated into the authors' automated dashboard, and trainees now have access to their own performance data. Eighty percent (48 of 60) of the residents responded to the feedback survey. Overall, residents "agreed/strongly agreed" that they desire frequent updates on their clinical performance on defined quality metrics and that they desired to see how they compared with the residency as a whole. Before deployment of the new tool, they "disagreed" that they were receiving feedback in a timely manner. Survey results were used to guide the format of the feedback tool that has been implemented. The authors demonstrate the implementation of a system that provides near-real-time feedback concerning resident performance on an extensible series of quality metrics, and which is responsive to requests arising from resident feedback about desired reporting mechanisms.

Automated Near Real-Time Clinical Performance Feedback for Anesthesiology Residents: One Piece of the Milestones Puzzle

PubMed Central

Ehrenfeld, Jesse M.; McEvoy, Matthew D.; Furman, William R.; Snyder, Dylan; Sandberg, Warren S.

2014-01-01

Background Anesthesiology residencies are developing trainee assessment tools to evaluate 25 milestones that map to the 6 core competencies. The effort will be facilitated by development of automated methods to capture, assess, and report trainee performance to program directors, the Accreditation Council for Graduate Medical Education and the trainees themselves. Methods We leveraged a perioperative information management system to develop an automated, near-real-time performance capture and feedback tool that provides objective data on clinical performance and requires minimal administrative effort. Prior to development, we surveyed trainees about satisfaction with clinical performance feedback and about preferences for future feedback. Results Resident performance on 24,154 completed cases has been incorporated into our automated dashboard, and trainees now have access to their own performance data. Eighty percent (48 of 60) of our residents responded to the feedback survey. Overall, residents ‘agreed/strongly agreed’ that they desire frequent updates on their clinical performance on defined quality metrics and that they desired to see how they compared to the residency as a whole. Prior to deployment of the new tool, they ‘disagreed’ that they were receiving feedback in a timely manner. Survey results were used to guide the format of the feedback tool that has been implemented. Conclusions We demonstrate the implementation of a system that provides near real-time feedback concerning resident performance on an extensible series of quality metrics, and which is responsive to requests arising from resident feedback about desired reporting mechanisms. PMID:24398735

Development of an Evaluation Methodology for Triple Bottom Line Reports Using International Standards on Reporting

NASA Astrophysics Data System (ADS)

Skouloudis, Antonis; Evangelinos, Konstantinos; Kourmousis, Fotis

2009-08-01

The purpose of this article is twofold. First, evaluation scoring systems for triple bottom line (TBL) reports to date are examined and potential methodological weaknesses and problems are highlighted. In this context, a new assessment methodology is presented based explicitly on the most widely acknowledged standard on non-financial reporting worldwide, the Global Reporting Initiative (GRI) guidelines. The set of GRI topics and performance indicators was converted into scoring criteria while the generic scoring devise was set from 0 to 4 points. Secondly, the proposed benchmark tool was applied to the TBL reports published by Greek companies. Results reveal major gaps in reporting practices, stressing the need for the further development of internal systems and processes in order to collect essential non-financial performance data. A critical overview of the structure and rationale of the evaluation tool in conjunction with the Greek case study is discussed while recommendations for future research on the field of this relatively new form of reporting are suggested.

Development of an evaluation methodology for triple bottom line reports using international standards on reporting.

PubMed

Skouloudis, Antonis; Evangelinos, Konstantinos; Kourmousis, Fotis

2009-08-01

The purpose of this article is twofold. First, evaluation scoring systems for triple bottom line (TBL) reports to date are examined and potential methodological weaknesses and problems are highlighted. In this context, a new assessment methodology is presented based explicitly on the most widely acknowledged standard on non-financial reporting worldwide, the Global Reporting Initiative (GRI) guidelines. The set of GRI topics and performance indicators was converted into scoring criteria while the generic scoring devise was set from 0 to 4 points. Secondly, the proposed benchmark tool was applied to the TBL reports published by Greek companies. Results reveal major gaps in reporting practices, stressing the need for the further development of internal systems and processes in order to collect essential non-financial performance data. A critical overview of the structure and rationale of the evaluation tool in conjunction with the Greek case study is discussed while recommendations for future research on the field of this relatively new form of reporting are suggested.

Strategic Analysis Overview

NASA Technical Reports Server (NTRS)

Cirillo, William M.; Earle, Kevin D.; Goodliff, Kandyce E.; Reeves, J. D.; Stromgren, Chel; Andraschko, Mark R.; Merrill, R. Gabe

2008-01-01

NASA s Constellation Program employs a strategic analysis methodology in providing an integrated analysis capability of Lunar exploration scenarios and to support strategic decision-making regarding those scenarios. The strategic analysis methodology integrates the assessment of the major contributors to strategic objective satisfaction performance, affordability, and risk and captures the linkages and feedbacks between all three components. Strategic analysis supports strategic decision making by senior management through comparable analysis of alternative strategies, provision of a consistent set of high level value metrics, and the enabling of cost-benefit analysis. The tools developed to implement the strategic analysis methodology are not element design and sizing tools. Rather, these models evaluate strategic performance using predefined elements, imported into a library from expert-driven design/sizing tools or expert analysis. Specific components of the strategic analysis tool set include scenario definition, requirements generation, mission manifesting, scenario lifecycle costing, crew time analysis, objective satisfaction benefit, risk analysis, and probabilistic evaluation. Results from all components of strategic analysis are evaluated a set of pre-defined figures of merit (FOMs). These FOMs capture the high-level strategic characteristics of all scenarios and facilitate direct comparison of options. The strategic analysis methodology that is described in this paper has previously been applied to the Space Shuttle and International Space Station Programs and is now being used to support the development of the baseline Constellation Program lunar architecture. This paper will present an overview of the strategic analysis methodology and will present sample results from the application of the strategic analysis methodology to the Constellation Program lunar architecture.

A framework for outcome-level evaluation of in-service training of health care workers.

PubMed

O'Malley, Gabrielle; Perdue, Thomas; Petracca, Frances

2013-10-01

In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President's Emergency Plan for AIDS Relief's Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the ability of evaluators to determine training outcomes. In addition, a group of user-friendly resources, the Training Evaluation Framework and Tools (TEFT) were created to help evaluators and stakeholders understand and apply the framework. Feedback from pilot users suggests that using the framework and accompanying tools may support outcome evaluation planning. Further assessment will assist in strengthening guidelines and tools for operationalization.

A framework for outcome-level evaluation of in-service training of health care workers

PubMed Central

2013-01-01

Background In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President’s Emergency Plan for AIDS Relief’s Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. Methods A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. Results The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the ability of evaluators to determine training outcomes. In addition, a group of user-friendly resources, the Training Evaluation Framework and Tools (TEFT) were created to help evaluators and stakeholders understand and apply the framework. Conclusions Feedback from pilot users suggests that using the framework and accompanying tools may support outcome evaluation planning. Further assessment will assist in strengthening guidelines and tools for operationalization. PMID:24083635

Assessing the validity and reliability of the Malagasy version of Oral Impacts on Daily Performance (OIDP): a cross-sectional study.

PubMed

Razanamihaja, Noeline; Ranivoharilanto, Eva

2017-01-01

Evaluating health needs includes measures of the impact of state of health on the quality of life. This entails evaluating the psychosocial aspects of health. To achieve this, several tools for measuring the quality of life related to oral health have been developed. However, it is vital to evaluate the psychometric properties of these tools so they can be used in a new context and on a new population. The purpose of this study was to evaluate the reliability and validity of the Malagasy version of a questionnaire for studying the impacts of oral-dental health on daily activities (Oral Impacts on Daily Performance), and analyse the interrelations between the scores obtained and the oral health indicators. A cross-sectional study was performed for the transcultural adaptation of the Oral Impacts on Daily Performance questionnaire forward translated and back-translated from English to Malagasy and from Malagasy to English, respectively. The psychometric characteristics of the Malagasy version of the Oral Impacts on Daily Performance were then evaluated in terms of internal reliability, test-retest, and construct, criteria and discriminant validity. Four hundred and six adults responded in face-to-face interviews to the Malagasy version of the Oral Impacts on Daily Performance questionnaire. Nearly 74% of the participants indicated impacts of their oral health on their performance in their daily lives during the 6 months prior to the survey. The activities most affected were: "smiling", "eating" and "sleeping and relaxing". Cronbach's alpha was 0.87. The construct validity was demonstrated by a significant association between the Oral Impacts on Daily Performance scores and the subjective evaluation of oral health ( p <0.001). Discriminant validity was demonstrated by the fact that the Oral Impacts on Daily Performance scores were significantly higher in subjects with more than ten missing teeth, compared to those with fewer than ten missing teeth ( p  < 0.001). The Malagasy version of the Oral Impacts on Daily Performance index is a valid and reliable measure for use in Malagasy adults over 55 years old.

Spanish validation of the social stigma scale: Community Attitudes towards Mental Illness.

PubMed

Ochoa, Susana; Martínez-Zambrano, Francisco; Vila-Badia, Regina; Arenas, Oti; Casas-Anguera, Emma; García-Morales, Esther; Villellas, Raúl; Martín, José Ramón; Pérez-Franco, María Belén; Valduciel, Tamara; García-Franco, Mar; Miguel, Jose; Balsera, Joaquim; Pascual, Gemma; Julia, Eugènia; Casellas, Diana; Haro, Josep Maria

2016-01-01

The stigma against people with mental illness is very high. In Spain there are currently no tools to assess this construct. The aim of this study was to validate the Spanish version of the Community Attitudes towards Mental Illness questionnaire in an adolescent population, and determining its internal consistency and temporal stability. Another analysis by gender will be also performed. A translation and back-translation of the Community Attitudes towards Mental Illness was performed. A total of 150 students of between 14 and 18 years-old were evaluated with this tool in two stages. Internal consistency was tested using Cronbach α; and intraclass correlation coefficient was used for test-retest reliability. Gender-stratified analyses were also performed. The Cronbach α was 0.861 for the first evaluation and 0.909 for the second evaluation. The values of the intraclass correlation coefficient ranged from 0.775 to 0.339 in the item by item analysis, and between 0.88 and 0.81 in the subscales. In the segmentation by gender, it was found that girls scored between 0.797 and 0.863 in the intraclass correlation coefficient, and boys scored between 0.889 and 0.774. In conclusion, the Community Attitudes towards Mental Illness is a reliable tool for the assessment of social stigma. Although reliable results have been found for boys and girls, our results found some gender differences in the analysis. Copyright © 2014 SEP y SEPB. Published by Elsevier España. All rights reserved.

Qualitative evaluation: A critical and interpretative complementary approach to improve health programs and services

PubMed Central

Tayabas, Luz María Tejada; León, Teresita Castillo; ESPINO, JOEL MONARREZ

2014-01-01

This short essay aims at commenting on the origin, development, rationale, and main characteristics of qualitative evaluation (QE), emphasizing the value of this methodological tool to evaluate health programs and services. During the past decades, different approaches have come to light proposing complementary alternatives to appraise the performance of public health programs, mainly focusing on the implementation process involved rather than on measuring the impact of such actions. QE is an alternative tool that can be used to illustrate and understand the process faced when executing health programs. It can also lead to useful suggestions to modify its implementation from the stakeholders’ perspectives, as it uses a qualitative approach that considers participants as reflective subjects, generators of meanings. This implies that beneficiaries become involved in an active manner in the evaluated phenomena with the aim of improving the health programs or services that they receive. With this work we want to encourage evaluators in the field of public health to consider the use of QE as a complementary tool for program evaluation to be able to identify areas of opportunity to improve programs’ implementation processes from the perspective of intended beneficiaries. PMID:25152220

The use of neural network technology to model swimming performance.

PubMed

Silva, António José; Costa, Aldo Manuel; Oliveira, Paulo Moura; Reis, Victor Machado; Saavedra, José; Perl, Jurgen; Rouboa, Abel; Marinho, Daniel Almeida

2007-01-01

to identify the factors which are able to explain the performance in the 200 meters individual medley and 400 meters front crawl events in young swimmers, to model the performance in those events using non-linear mathematic methods through artificial neural networks (multi-layer perceptrons) and to assess the neural network models precision to predict the performance. A sample of 138 young swimmers (65 males and 73 females) of national level was submitted to a test battery comprising four different domains: kinanthropometric evaluation, dry land functional evaluation (strength and flexibility), swimming functional evaluation (hydrodynamics, hydrostatic and bioenergetics characteristics) and swimming technique evaluation. To establish a profile of the young swimmer non-linear combinations between preponderant variables for each gender and swim performance in the 200 meters medley and 400 meters font crawl events were developed. For this purpose a feed forward neural network was used (Multilayer Perceptron) with three neurons in a single hidden layer. The prognosis precision of the model (error lower than 0.8% between true and estimated performances) is supported by recent evidence. Therefore, we consider that the neural network tool can be a good approach in the resolution of complex problems such as performance modeling and the talent identification in swimming and, possibly, in a wide variety of sports. Key pointsThe non-linear analysis resulting from the use of feed forward neural network allowed us the development of four performance models.The mean difference between the true and estimated results performed by each one of the four neural network models constructed was low.The neural network tool can be a good approach in the resolution of the performance modeling as an alternative to the standard statistical models that presume well-defined distributions and independence among all inputs.The use of neural networks for sports sciences application allowed us to create very realistic models for swimming performance prediction based on previous selected criterions that were related with the dependent variable (performance).

ATLAS offline software performance monitoring and optimization

NASA Astrophysics Data System (ADS)

Chauhan, N.; Kabra, G.; Kittelmann, T.; Langenberg, R.; Mandrysch, R.; Salzburger, A.; Seuster, R.; Ritsch, E.; Stewart, G.; van Eldik, N.; Vitillo, R.; Atlas Collaboration

2014-06-01

In a complex multi-developer, multi-package software environment, such as the ATLAS offline framework Athena, tracking the performance of the code can be a non-trivial task in itself. In this paper we describe improvements in the instrumentation of ATLAS offline software that have given considerable insight into the performance of the code and helped to guide the optimization work. The first tool we used to instrument the code is PAPI, which is a programing interface for accessing hardware performance counters. PAPI events can count floating point operations, cycles, instructions and cache accesses. Triggering PAPI to start/stop counting for each algorithm and processed event results in a good understanding of the algorithm level performance of ATLAS code. Further data can be obtained using Pin, a dynamic binary instrumentation tool. Pin tools can be used to obtain similar statistics as PAPI, but advantageously without requiring recompilation of the code. Fine grained routine and instruction level instrumentation is also possible. Pin tools can additionally interrogate the arguments to functions, like those in linear algebra libraries, so that a detailed usage profile can be obtained. These tools have characterized the extensive use of vector and matrix operations in ATLAS tracking. Currently, CLHEP is used here, which is not an optimal choice. To help evaluate replacement libraries a testbed has been setup allowing comparison of the performance of different linear algebra libraries (including CLHEP, Eigen and SMatrix/SVector). Results are then presented via the ATLAS Performance Management Board framework, which runs daily with the current development branch of the code and monitors reconstruction and Monte-Carlo jobs. This framework analyses the CPU and memory performance of algorithms and an overview of results are presented on a web page. These tools have provided the insight necessary to plan and implement performance enhancements in ATLAS code by identifying the most common operations, with the call parameters well understood, and allowing improvements to be quantified in detail.

77 FR 39250 - Notice of Proposed Information Collection for Public Comment: Home Equity Conversion Mortgage...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-07-02

... Information Collection for Public Comment: Home Equity Conversion Mortgage Counseling Client Survey AGENCY... subject proposal. The HECM Counseling Session Evaluation is sent to counseling recipients as part of HUD's performance review of a HECM counseling agency. It is used by HUD as a performance tool to obtain information...

Relationship between body condition score at calving and reproductive performance in young postpartum cows grazing native range

USDA-ARS?s Scientific Manuscript database

Body condition score is used as a management tool to predict competency of reproduction in beef cows. Therefore, a retrospective study was performed to evaluate association of BCS at calving with subsequent pregnancy rate, days to first estrus, nutrient status (assessed by blood metabolites), and c...

Peer Review of a Formal Verification/Design Proof Methodology

NASA Technical Reports Server (NTRS)

1983-01-01

The role of formal verification techniques in system validation was examined. The value and the state of the art of performance proving for fault-tolerant compuers were assessed. The investigation, development, and evaluation of performance proving tools were reviewed. The technical issues related to proof methodologies are examined. The technical issues discussed are summarized.

Evaluating supplier quality performance using analytical hierarchy process

NASA Astrophysics Data System (ADS)

Kalimuthu Rajoo, Shanmugam Sundram; Kasim, Maznah Mat; Ahmad, Nazihah

2013-09-01

This paper elaborates the importance of evaluating supplier quality performance to an organization. Supplier quality performance evaluation reflects the actual performance of the supplier exhibited at customer's end. It is critical in enabling the organization to determine the area of improvement and thereafter works with supplier to close the gaps. Success of the customer partly depends on supplier's quality performance. Key criteria as quality, cost, delivery, technology support and customer service are categorized as main factors in contributing to supplier's quality performance. 18 suppliers' who were manufacturing automotive application parts evaluated in year 2010 using weight point system. There were few suppliers with common rating which led to common ranking observed by few suppliers'. Analytical Hierarchy Process (AHP), a user friendly decision making tool for complex and multi criteria problems was used to evaluate the supplier's quality performance challenging the weight point system that was used for 18 suppliers'. The consistency ratio was checked for criteria and sub-criteria. Final results of AHP obtained with no overlap ratings, therefore yielded a better decision making methodology as compared to weight point rating system.

A Novel Approach to Practice-Based Learning and Improvement Using a Web-Based Audit and Feedback Module

PubMed Central

Boggan, Joel C.; Cheely, George; Shah, Bimal R.; Heffelfinger, Randy; Springall, Deanna; Thomas, Samantha M.; Zaas, Aimee; Bae, Jonathan

2014-01-01

Background Systematically engaging residents in large programs in quality improvement (QI) is challenging. Objective To coordinate a shared QI project in a large residency program using an online tool. Methods A web-based QI tool guided residents through a 2-phase evaluation of performance of foot examinations in patients with diabetes. In phase 1, residents completed reviews of health records with online data entry. Residents were then presented with personal performance data relative to peers and were prompted to develop improvement plans. In phase 2, residents again reviewed personal performance. Rates of performance were compared at the program and clinic levels for each phase, with data presented for residents. Acceptability was measured by the number of residents completing each phase. Feasibility was measured by estimated faculty, programmer, and administrator time and costs. Results Seventy-nine of 86 eligible residents (92%) completed improvement plans and reviewed 1471 patients in phase 1, whereas 68 residents (79%) reviewed 1054 patient charts in phase 2. Rates of performance of examination increased significantly between phases (from 52% to 73% for complete examination, P < .001). Development of the tool required 130 hours of programmer time. Project analysis and management required 6 hours of administrator and faculty time monthly. Conclusions An online tool developed and implemented for program-wide QI initiatives successfully engaged residents to participate in QI activities. Residents using this tool demonstrated improvement in a selected quality target. This tool could be adapted by other graduate medical education programs or for faculty development. PMID:26279782

Data Visualization Saliency Model: A Tool for Evaluating Abstract Data Visualizations

DOE PAGES

Matzen, Laura E.; Haass, Michael J.; Divis, Kristin M.; ...

2017-08-29

Evaluating the effectiveness of data visualizations is a challenging undertaking and often relies on one-off studies that test a visualization in the context of one specific task. Researchers across the fields of data science, visualization, and human-computer interaction are calling for foundational tools and principles that could be applied to assessing the effectiveness of data visualizations in a more rapid and generalizable manner. One possibility for such a tool is a model of visual saliency for data visualizations. Visual saliency models are typically based on the properties of the human visual cortex and predict which areas of a scene havemore » visual features (e.g. color, luminance, edges) that are likely to draw a viewer's attention. While these models can accurately predict where viewers will look in a natural scene, they typically do not perform well for abstract data visualizations. In this paper, we discuss the reasons for the poor performance of existing saliency models when applied to data visualizations. We introduce the Data Visualization Saliency (DVS) model, a saliency model tailored to address some of these weaknesses, and we test the performance of the DVS model and existing saliency models by comparing the saliency maps produced by the models to eye tracking data obtained from human viewers. In conclusion, we describe how modified saliency models could be used as general tools for assessing the effectiveness of visualizations, including the strengths and weaknesses of this approach.« less

Data Visualization Saliency Model: A Tool for Evaluating Abstract Data Visualizations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Matzen, Laura E.; Haass, Michael J.; Divis, Kristin M.

Evaluating the effectiveness of data visualizations is a challenging undertaking and often relies on one-off studies that test a visualization in the context of one specific task. Researchers across the fields of data science, visualization, and human-computer interaction are calling for foundational tools and principles that could be applied to assessing the effectiveness of data visualizations in a more rapid and generalizable manner. One possibility for such a tool is a model of visual saliency for data visualizations. Visual saliency models are typically based on the properties of the human visual cortex and predict which areas of a scene havemore » visual features (e.g. color, luminance, edges) that are likely to draw a viewer's attention. While these models can accurately predict where viewers will look in a natural scene, they typically do not perform well for abstract data visualizations. In this paper, we discuss the reasons for the poor performance of existing saliency models when applied to data visualizations. We introduce the Data Visualization Saliency (DVS) model, a saliency model tailored to address some of these weaknesses, and we test the performance of the DVS model and existing saliency models by comparing the saliency maps produced by the models to eye tracking data obtained from human viewers. In conclusion, we describe how modified saliency models could be used as general tools for assessing the effectiveness of visualizations, including the strengths and weaknesses of this approach.« less

IDEAS: A multidisciplinary computer-aided conceptual design system for spacecraft

NASA Technical Reports Server (NTRS)

Ferebee, M. J., Jr.

1984-01-01

During the conceptual development of advanced aerospace vehicles, many compromises must be considered to balance economy and performance of the total system. Subsystem tradeoffs may need to be made in order to satisfy system-sensitive attributes. Due to the increasingly complex nature of aerospace systems, these trade studies have become more difficult and time-consuming to complete and involve interactions of ever-larger numbers of subsystems, components, and performance parameters. The current advances of computer-aided synthesis, modeling and analysis techniques have greatly helped in the evaluation of competing design concepts. Langley Research Center's Space Systems Division is currently engaged in trade studies for a variety of systems which include advanced ground-launched space transportation systems, space-based orbital transfer vehicles, large space antenna concepts and space stations. The need for engineering analysis tools to aid in the rapid synthesis and evaluation of spacecraft has led to the development of the Interactive Design and Evaluation of Advanced Spacecraft (IDEAS) computer-aided design system. The ADEAS system has been used to perform trade studies of competing technologies and requirements in order to pinpoint possible beneficial areas for research and development. IDEAS is presented as a multidisciplinary tool for the analysis of advanced space systems. Capabilities range from model generation and structural and thermal analysis to subsystem synthesis and performance analysis.

Contamination and Surface Preparation Effects on Composite Bonding

NASA Technical Reports Server (NTRS)

Kutscha, Eileen O.; Vahey, Paul G.; Belcher, Marcus A.; VanVoast, Peter J.; Grace, William B.; Blohowiak, Kay Y.; Palmieri, Frank L.; Connell, John W.

2017-01-01

Results presented here demonstrate the effect of several prebond surface contaminants (hydrocarbon, machining fluid, latex, silicone, peel ply residue, release film) on bond quality, as measured by fracture toughness and failure modes of carbon fiber reinforced epoxy substrates bonded in secondary and co-bond configurations with paste and film adhesives. Additionally, the capability of various prebond surface property measurement tools to detect contaminants and potentially predict subsequent bond performance of three different adhesives is also shown. Surface measurement methods included water contact angle, Dyne solution wettability, optically stimulated electron emission spectroscopy, surface free energy, inverse gas chromatography, and Fourier transform infrared spectroscopy with chemometrics analysis. Information will also be provided on the effectiveness of mechanical and energetic surface treatments to recover a bondable surface after contamination. The benefits and drawbacks of the various surface analysis tools to detect contaminants and evaluate prebond surfaces after surface treatment were assessed as well as their ability to correlate to bond performance. Surface analysis tools were also evaluated for their potential use as in-line quality control of adhesive bonding parameters in the manufacturing environment.

Evaluation of an open source tool for indexing and searching enterprise radiology and pathology reports

NASA Astrophysics Data System (ADS)

Kim, Woojin; Boonn, William

2010-03-01

Data mining of existing radiology and pathology reports within an enterprise health system can be used for clinical decision support, research, education, as well as operational analyses. In our health system, the database of radiology and pathology reports exceeds 13 million entries combined. We are building a web-based tool to allow search and data analysis of these combined databases using freely available and open source tools. This presentation will compare performance of an open source full-text indexing tool to MySQL's full-text indexing and searching and describe implementation procedures to incorporate these capabilities into a radiology-pathology search engine.

Introducing the Professionalism Mini-Evaluation Exercise (P-MEX) in Japan: results from a multicenter, cross-sectional study.

PubMed

Tsugawa, Yusuke; Ohbu, Sadayoshi; Cruess, Richard; Cruess, Sylvia; Okubo, Tomoya; Takahashi, Osamu; Tokuda, Yasuharu; Heist, Brian S; Bito, Seiji; Itoh, Toshiyuki; Aoki, Akiko; Chiba, Tsutomu; Fukui, Tsuguya

2011-08-01

Despite the growing importance of and interest in medical professionalism, there is no standardized tool for its measurement. The authors sought to verify the validity, reliability, and generalizability of the Professionalism Mini-Evaluation Exercise (P-MEX), a previously developed and tested tool, in the context of Japanese hospitals. A multicenter, cross-sectional evaluation study was performed to investigate the validity, reliability, and generalizability of the P-MEX in seven Japanese hospitals. In 2009-2010, 378 evaluators (attending physicians, nurses, peers, and junior residents) completed 360-degree assessments of 165 residents and fellows using the P-MEX. The content validity and criterion-related validity were examined, and the construct validity of the P-MEX was investigated by performing confirmatory factor analysis through a structural equation model. The reliability was tested using generalizability analysis. The contents of the P-MEX achieved good acceptance in a preliminary working group, and the poststudy survey revealed that 302 (79.9%) evaluators rated the P-MEX items as appropriate, indicating good content validity. The correlation coefficient between P-MEX scores and external criteria was 0.78 (P < .001), demonstrating good criterion-related validity. Confirmatory factor analysis verified high path coefficient (0.60-0.99) and adequate goodness of fit of the model. The generalizability analysis yielded a high dependability coefficient, suggesting good reliability, except when evaluators were peers or junior residents. Findings show evidence of adequate validity, reliability, and generalizability of the P-MEX in Japanese hospital settings. The P-MEX is the only evaluation tool for medical professionalism verified in both a Western and East Asian cultural context.

Recognizing chemicals in patents: a comparative analysis.

PubMed

Habibi, Maryam; Wiegandt, David Luis; Schmedding, Florian; Leser, Ulf

2016-01-01

Recently, methods for Chemical Named Entity Recognition (NER) have gained substantial interest, driven by the need for automatically analyzing todays ever growing collections of biomedical text. Chemical NER for patents is particularly essential due to the high economic importance of pharmaceutical findings. However, NER on patents has essentially been neglected by the research community for long, mostly because of the lack of enough annotated corpora. A recent international competition specifically targeted this task, but evaluated tools only on gold standard patent abstracts instead of full patents; furthermore, results from such competitions are often difficult to extrapolate to real-life settings due to the relatively high homogeneity of training and test data. Here, we evaluate the two state-of-the-art chemical NER tools, tmChem and ChemSpot, on four different annotated patent corpora, two of which consist of full texts. We study the overall performance of the tools, compare their results at the instance level, report on high-recall and high-precision ensembles, and perform cross-corpus and intra-corpus evaluations. Our findings indicate that full patents are considerably harder to analyze than patent abstracts and clearly confirm the common wisdom that using the same text genre (patent vs. scientific) and text type (abstract vs. full text) for training and testing is a pre-requisite for achieving high quality text mining results.

A Management System for Computer Performance Evaluation.

DTIC Science & Technology

1981-12-01

AD-AIlS 538 AIR FORCE INST OF TECH WRIGHT-PATTERSON AFB OH SCHOO-ETC F/6 S/1 MANAGEMENT SYSTEM FOR COMPUTER PERFORMANCE EVALUATION. (U DEC 81 H K...release; distribution unlimited. AFIT/GCS/1,Y/81 D)-i PREFACE As an installation manager of a Burroughs 3500 1 erncountered many problems concerning its...techniques to select, and finally, how do I organize the effort. As a manager I felt that I needed a reference or tool that would broaden my OPE

Cultural competency assessment tool for hospitals: evaluating hospitals' adherence to the culturally and linguistically appropriate services standards.

PubMed

Weech-Maldonado, Robert; Dreachslin, Janice L; Brown, Julie; Pradhan, Rohit; Rubin, Kelly L; Schiller, Cameron; Hays, Ron D

2012-01-01

The U.S. national standards for culturally and linguistically appropriate services (CLAS) in health care provide guidelines on policies and practices aimed at developing culturally competent systems of care. The Cultural Competency Assessment Tool for Hospitals (CCATH) was developed as an organizational tool to assess adherence to the CLAS standards. First, we describe the development of the CCATH and estimate the reliability and validity of the CCATH measures. Second, we discuss the managerial implications of the CCATH as an organizational tool to assess cultural competency. We pilot tested an initial draft of the CCATH, revised it based on a focus group and cognitive interviews, and then administered it in a field test with a sample of California hospitals. The reliability and validity of the CCATH were evaluated using factor analysis, analysis of variance, and Cronbach's alphas. Exploratory and confirmatory factor analyses identified 12 CCATH composites: leadership and strategic planning, data collection on inpatient population, data collection on service area, performance management systems and quality improvement, human resources practices, diversity training, community representation, availability of interpreter services, interpreter services policies, quality of interpreter services, translation of written materials, and clinical cultural competency practices. All the CCATH scales had internal consistency reliability of .65 or above, and the reliability was .70 or above for 9 of the 12 scales. Analysis of variance results showed that not-for-profit hospitals have higher CCATH scores than for-profit hospitals in five CCATH scales and higher CCATH scores than government hospitals in two CCATH scales. The CCATH showed adequate psychometric properties. Managers and policy makers can use the CCATH as a tool to evaluate hospital performance in cultural competency and identify and target improvements in hospital policies and practices that undergird the provision of CLAS.

A community resource benchmarking predictions of peptide binding to MHC-I molecules.

PubMed

Peters, Bjoern; Bui, Huynh-Hoa; Frankild, Sune; Nielson, Morten; Lundegaard, Claus; Kostem, Emrah; Basch, Derek; Lamberth, Kasper; Harndahl, Mikkel; Fleri, Ward; Wilson, Stephen S; Sidney, John; Lund, Ole; Buus, Soren; Sette, Alessandro

2006-06-09

Recognition of peptides bound to major histocompatibility complex (MHC) class I molecules by T lymphocytes is an essential part of immune surveillance. Each MHC allele has a characteristic peptide binding preference, which can be captured in prediction algorithms, allowing for the rapid scan of entire pathogen proteomes for peptide likely to bind MHC. Here we make public a large set of 48,828 quantitative peptide-binding affinity measurements relating to 48 different mouse, human, macaque, and chimpanzee MHC class I alleles. We use this data to establish a set of benchmark predictions with one neural network method and two matrix-based prediction methods extensively utilized in our groups. In general, the neural network outperforms the matrix-based predictions mainly due to its ability to generalize even on a small amount of data. We also retrieved predictions from tools publicly available on the internet. While differences in the data used to generate these predictions hamper direct comparisons, we do conclude that tools based on combinatorial peptide libraries perform remarkably well. The transparent prediction evaluation on this dataset provides tool developers with a benchmark for comparison of newly developed prediction methods. In addition, to generate and evaluate our own prediction methods, we have established an easily extensible web-based prediction framework that allows automated side-by-side comparisons of prediction methods implemented by experts. This is an advance over the current practice of tool developers having to generate reference predictions themselves, which can lead to underestimating the performance of prediction methods they are not as familiar with as their own. The overall goal of this effort is to provide a transparent prediction evaluation allowing bioinformaticians to identify promising features of prediction methods and providing guidance to immunologists regarding the reliability of prediction tools.

Identifying poor performance among doctors in NHS organizations.

PubMed

Locke, Rachel; Scallan, Samantha; Leach, Camilla; Rickenbach, Mark

2013-10-01

To account for the means by which poor performance among career doctors is identified by National Health Service organizations, whether the tools are considered effective and how these processes may be strengthened in the light of revalidation and the requirement for doctors to demonstrate their fitness to practice. This study sought to look beyond the 'doctor as individual'; as well as considering the typical approaches to managing the practice of an individual, the systems within which the doctor is working were reviewed, as these are also relevant to standards of performance. A qualitative review was undertaken consisting of a literature review of current practice, a policy review of current documentation from 15 trusts in one deanery locality, and 14 semi-structured interviews with respondents with an overview of processes in use. The framework for the analysis of the data considered tools at three levels: individual, team and organizational. Tools are, in the main, reactive--with an individual focus. They rely on colleagues and others to speak out, so their effectiveness is hindered by a reluctance to do so. Tools can lack an evidence base for their use, and there is limited linking of data across contexts and tools. There is more work to be done in evaluating current tools and developing stronger processes. Linkage between data sources needs to be improved and proactive tools at the organizational level need further development to help with the early identification of performance issues. This would also assist in balancing a wider systems approach with a current over emphasis on individual doctors. © 2012 John Wiley & Sons Ltd.

Efficacy of a novel educational curriculum using a simulation laboratory on resident performance of hysteroscopic sterilization.

PubMed

Chudnoff, Scott G; Liu, Connie S; Levie, Mark D; Bernstein, Peter; Banks, Erika H

2010-09-01

To assess whether a novel educational curriculum using a simulation teaching laboratory improves resident knowledge, comfort with, and surgical performance of hysteroscopic sterilization. An educational prospective, pretest/posttest study. The Montefiore Institute of Minimally Invasive Surgery Laboratory. PATIENT(S)/SUBJECT(S): Thirty-four OB/GYN residents in an academic medical center. Hysteroscopic sterilization simulation laboratory and a brief didactic lecture. Differences in scores on validated skill assessment tools: Task specific checklist, Global Rating Scale (GRS), pass fail assessment, and a multiple-choice examination to evaluate knowledge and attitude. In the entire cohort improvements were observed on all evaluation tools after the simulation laboratory, with 31% points (SD+/-11.5, 95% confidence interval [CI] 27.3-35.3) higher score on the written evaluation; 63% points (SD+/-15.7, 95% CI 57.8-68.8) higher score on the task specific checklist; and 54% points (SD+/-13.6, 95% CI 48.8-58.3) higher score on the GRS. Higher PGY status was correlated with better pretest performance, but was not statistically significant in posttest scores. Residents reported an improvement in comfort performing the procedure after the laboratory. Simulation laboratory teaching significantly improved resident knowledge, comfort level, and technical skill performance of hysteroscopic sterilization. Copyright (c) 2010 American Society for Reproductive Medicine. Published by Elsevier Inc. All rights reserved.

Clinical Trial Assessment of Infrastructure Matrix Tool to Improve the Quality of Research Conduct in the Community.

PubMed

Dimond, Eileen P; Zon, Robin T; Weiner, Bryan J; St Germain, Diane; Denicoff, Andrea M; Dempsey, Kandie; Carrigan, Angela C; Teal, Randall W; Good, Marjorie J; McCaskill-Stevens, Worta; Grubbs, Stephen S; Dimond, Eileen P; Zon, Robin T; Weiner, Bryan J; St Germain, Diane; Denicoff, Andrea M; Dempsey, Kandie; Carrigan, Angela C; Teal, Randall W; Good, Marjorie J; McCaskill-Stevens, Worta; Grubbs, Stephen S

2016-01-01

Several publications have described minimum standards and exemplary attributes for clinical trial sites to improve research quality. The National Cancer Institute (NCI) Community Cancer Centers Program (NCCCP) developed the clinical trial Best Practice Matrix tool to facilitate research program improvements through annual self-assessments and benchmarking. The tool identified nine attributes, each with three progressive levels, to score clinical trial infrastructural elements from less to more exemplary. The NCCCP sites correlated tool use with research program improvements, and the NCI pursued a formative evaluation to refine the interpretability and measurability of the tool. From 2011 to 2013, 21 NCCCP sites self-assessed their programs with the tool annually. During 2013 to 2014, NCI collaborators conducted a five-step formative evaluation of the matrix tool. Sites reported significant increases in level-three scores across the original nine attributes combined (P<.001). Two specific attributes exhibited significant change: clinical trial portfolio diversity and management (P=.0228) and clinical trial communication (P=.0281). The formative evaluation led to revisions, including renaming the Best Practice Matrix as the Clinical Trial Assessment of Infrastructure Matrix (CT AIM), expanding infrastructural attributes from nine to 11, clarifying metrics, and developing a new scoring tool. Broad community input, cognitive interviews, and pilot testing improved the usability and functionality of the tool. Research programs are encouraged to use the CT AIM to assess and improve site infrastructure. Experience within the NCCCP suggests that the CT AIM is useful for improving quality, benchmarking research performance, reporting progress, and communicating program needs with institutional leaders. The tool model may also be useful in disciplines beyond oncology.

Simulation Performance and National Council Licensure Examination for Registered Nurses Outcomes: Field Research Perspectives.

PubMed

Brackney, Dana E; Lane, Susan Hayes; Dawson, Tyia; Koontz, Angie

2017-11-01

This descriptive field study examines processes used to evaluate simulation for senior-level Bachelor of Science in Nursing (BSN) students in a capstone course, discusses challenges related to simulation evaluation, and reports the relationship between faculty evaluation of student performance and National Council Licensure Examination for Registered Nurses (NCLEX-RN) first-time passing rates. Researchers applied seven terms used to rank BSN student performance (n = 41, female, ages 22-24 years) in a senior-level capstone simulation. Faculty evaluation was correlated with students' NCLEX-RN outcomes. Students evaluated as "lacking confidence" and "flawed" were less likely to pass the NCLEX-RN on the first attempt. Faculty evaluation of capstone simulation performance provided additional evidence of student preparedness for practice in the RN role, as evidenced by the relationship between the faculty assessment and NCLEX-RN success. Simulation has been broadly accepted as a powerful educational tool that may also contribute to verification of student achievement of program outcomes and readiness for the RN role.

Evaluation of EIT system performance.

PubMed

Yasin, Mamatjan; Böhm, Stephan; Gaggero, Pascal O; Adler, Andy

2011-07-01

An electrical impedance tomography (EIT) system images internal conductivity from surface electrical stimulation and measurement. Such systems necessarily comprise multiple design choices from cables and hardware design to calibration and image reconstruction. In order to compare EIT systems and study the consequences of changes in system performance, this paper describes a systematic approach to evaluate the performance of the EIT systems. The system to be tested is connected to a saline phantom in which calibrated contrasting test objects are systematically positioned using a position controller. A set of evaluation parameters are proposed which characterize (i) data and image noise, (ii) data accuracy, (iii) detectability of single contrasts and distinguishability of multiple contrasts, and (iv) accuracy of reconstructed image (amplitude, resolution, position and ringing). Using this approach, we evaluate three different EIT systems and illustrate the use of these tools to evaluate and compare performance. In order to facilitate the use of this approach, all details of the phantom, test objects and position controller design are made publicly available including the source code of the evaluation and reporting software.

Balanced scorecards for performance management.

PubMed

Park, Eun-Jun; Huber, Diane L

2007-01-01

Nurse administrators who manage nursing case management programs are challenged to demonstrate the improved quality of patient care and financial outcomes to their organization that result from such programs. This article introduces the balanced scorecard and discusses its benefits and practical concerns for adopting the scorecard. The balanced scorecard is a useful performance management tool used to both evaluate and direct case management performance in meeting organizational missions and strategies.

Simulation-based ongoing professional practice evaluation in psychiatry: a novel tool for performance assessment.

PubMed

Gorrindo, Tristan; Goldfarb, Elizabeth; Birnbaum, Robert J; Chevalier, Lydia; Meller, Benjamin; Alpert, Jonathan; Herman, John; Weiss, Anthony

2013-07-01

Ongoing professional practice evaluation (OPPE) activities consist of a quantitative, competency-based evaluation of clinical performance. Hospitals must design assessments that measure clinical competencies, are scalable, and minimize impact on the clinician's daily routines. A psychiatry department at a large academic medical center designed and implemented an interactive Web-based psychiatric simulation focusing on violence risk assessment as a tool for a departmentwide OPPE. Of 412 invited clinicians in a large psychiatry department, 410 completed an online simulation in April-May 2012. Participants received scheduled e-mail reminders with instructions describing how to access the simulation. Using the Computer Simulation Assessment Tool, participants viewed an introductory video and were then asked to conduct a risk assessment, acting as a clinician in the encounter by selecting actions from a series of drop-down menus. Each action was paired with a corresponding video segment of a clinical encounter with a standardized patient. Participants were scored on the basis of their actions within the simulation (Measure 1) and by their responses to the open-ended questions in which they were asked to integrate the information from the simulation in a summative manner (Measure 2). Of the 410 clinicians, 381 (92.9%) passed Measure 1,359 (87.6%) passed Measure 2, and 5 (1.2%) failed both measures. Seventy-five (18.3%) participants were referred for focused professional practice evaluation (FPPE) after failing either Measure 1, Measure 2, or both. Overall, Web-based simulation and e-mail engagement tools were a scalable and efficient way to assess a large number of clinicians in OPPE and to identify those who required FPPE.

Performance of office workers under various enclosure conditions in state-of-the-art open workplaces

NASA Astrophysics Data System (ADS)

Yoon, Heakyung Cecilia

The objective of this thesis is to more firmly establish the importance of physical attributes of workstations on the performance of workers undertaking a range of complex tasks while subjected to the visual and noise distractions prevalent in state-of-the-art North American office settings. This study investigates objective and subjective evaluations of noise and performance given a range of current physical work environments. The study provides criteria for architects, interior designers and managers, to select distraction-free office environments to deliver better performance. The concluding chapter helps to establish the importance of designing more acoustically responsible work settings in state-of-the-art office projects. With 102 subjects (23 native speakers of English per each of three workstation types), controlled experiments were completed over a six month testing period in three different work settings---four foot partitions on two sides, seated privacy with six foot partitions on three sides, and a closed office with eight foot partitions, a door and a ceiling, with two acoustic environments (office sounds with and without speech at a controlled 45 dBA level at the receiver), the experimental results were statistically significant. Another finding was the lack of a significant effect of background sound variations on simple or complex task performance. That implies the current acoustical evaluation tool, the Articulation Index, may not be an appropriate tool to adequately and conclusively assess the acoustic impact of open workplaces on individual performance. Concerning the impact of acoustic conditions on occupant performance from the experiments, Articulation Index values do not reflect the potential relation of workstation designs and subjects' performance and moods. However, NIC connected with speech privacy rating has the potential to be a better evaluation tool than AI for open workplaces. From the results of this thesis, it is predicted that fully enclosed workstations will improve the individual performance of knowledge workers whose main tasks are complex, as well as improve the moods of occupants towards collaborations with their co-workers.

Implementing a Quantitative Analysis Design Tool for Future Generation Interfaces

DTIC Science & Technology

2012-03-01

with Remotely Piloted Aircraft (RPA) has resulted in the need of a platform to evaluate interface design. The Vigilant Spirit Control Station ( VSCS ...Spirit interface. A modified version of the HCI Index was successfully applied to perform a quantitative analysis of the baseline VSCS interface and...time of the original VSCS interface. These results revealed the effectiveness of the tool and demonstrated in the design of future generation

A Portfolio Analysis Tool for Measuring NASAs Aeronautics Research Progress toward Planned Strategic Outcomes

NASA Technical Reports Server (NTRS)

Tahmasebi, Farhad; Pearce, Robert

2016-01-01

Description of a tool for portfolio analysis of NASA's Aeronautics research progress toward planned community strategic Outcomes is presented. The strategic planning process for determining the community Outcomes is also briefly described. Stakeholder buy-in, partnership performance, progress of supporting Technical Challenges, and enablement forecast are used as the criteria for evaluating progress toward Outcomes. A few illustrative examples are also presented.

Investigation, Development, and Evaluation of Performance Proving for Fault-tolerant Computers

NASA Technical Reports Server (NTRS)

Levitt, K. N.; Schwartz, R.; Hare, D.; Moore, J. S.; Melliar-Smith, P. M.; Shostak, R. E.; Boyer, R. S.; Green, M. W.; Elliott, W. D.

1983-01-01

A number of methodologies for verifying systems and computer based tools that assist users in verifying their systems were developed. These tools were applied to verify in part the SIFT ultrareliable aircraft computer. Topics covered included: STP theorem prover; design verification of SIFT; high level language code verification; assembly language level verification; numerical algorithm verification; verification of flight control programs; and verification of hardware logic.

Tool Support for Software Lookup Table Optimization

DOE PAGES

Wilcox, Chris; Strout, Michelle Mills; Bieman, James M.

2011-01-01

A number of scientific applications are performance-limited by expressions that repeatedly call costly elementary functions. Lookup table (LUT) optimization accelerates the evaluation of such functions by reusing previously computed results. LUT methods can speed up applications that tolerate an approximation of function results, thereby achieving a high level of fuzzy reuse. One problem with LUT optimization is the difficulty of controlling the tradeoff between performance and accuracy. The current practice of manual LUT optimization adds programming effort by requiring extensive experimentation to make this tradeoff, and such hand tuning can obfuscate algorithms. In this paper we describe a methodology andmore » tool implementation to improve the application of software LUT optimization. Our Mesa tool implements source-to-source transformations for C or C++ code to automate the tedious and error-prone aspects of LUT generation such as domain profiling, error analysis, and code generation. We evaluate Mesa with five scientific applications. Our results show a performance improvement of 3.0× and 6.9× for two molecular biology algorithms, 1.4× for a molecular dynamics program, 2.1× to 2.8× for a neural network application, and 4.6× for a hydrology calculation. We find that Mesa enables LUT optimization with more control over accuracy and less effort than manual approaches.« less

Piloting a programme tool to evaluate malaria case investigation and reactive case detection activities: results from 3 settings in the Asia Pacific.

PubMed

Cotter, Chris; Sudathip, Prayuth; Herdiana, Herdiana; Cao, Yuanyuan; Liu, Yaobao; Luo, Alex; Ranasinghe, Neil; Bennett, Adam; Cao, Jun; Gosling, Roly D

2017-08-22

Case investigation and reactive case detection (RACD) activities are widely-used in low transmission settings to determine the suspected origin of infection and identify and treat malaria infections nearby to the index patient household. Case investigation and RACD activities are time and resource intensive, include methodologies that vary across eliminating settings, and have no standardized metrics or tools available to monitor and evaluate them. In response to this gap, a simple programme tool was developed for monitoring and evaluating (M&E) RACD activities and piloted by national malaria programmes. During the development phase, four modules of the RACD M&E tool were created to assess and evaluate key case investigation and RACD activities and costs. A pilot phase was then carried out by programme implementers between 2013 and 2015, during which malaria surveillance teams in three different settings (China, Indonesia, Thailand) piloted the tool over a period of 3 months each. This study describes summary results of the pilots and feasibility and impact of the tool on programmes. All three study areas implemented the RACD M&E tool modules, and pilot users reported the tool and evaluation process were helpful to identify gaps in RACD programme activities. In the 45 health facilities evaluated, 71.8% (97/135; min 35.3-max 100.0%) of the proper notification and reporting forms and 20.0% (27/135; min 0.0-max 100.0%) of standard operating procedures (SOPs) were available to support malaria elimination activities. The tool highlighted gaps in reporting key data indicators on the completeness for malaria case reporting (98.8%; min 93.3-max 100.0%), case investigations (65.6%; min 61.8-max 78.4%) and RACD activities (70.0%; min 64.7-max 100.0%). Evaluation of the SOPs showed that knowledge and practices of malaria personnel varied within and between study areas. Average monthly costs for conducting case investigation and RACD activities showed variation between study areas (min USD $844.80-max USD $2038.00) for the malaria personnel, commodities, services and other costs required to carry out the activities. The RACD M&E tool was implemented in the three pilot areas, identifying key gaps that led to impacts on programme decision making. Study findings support the need for routine M&E of malaria case reporting, case investigation and RACD activities. Scale-up of the RACD M&E tool in malaria-eliminating settings will contribute to improved programme performance to the high level that is required to reach elimination.

Freva - Freie Univ Evaluation System Framework for Scientific HPC Infrastructures in Earth System Modeling

NASA Astrophysics Data System (ADS)

Kadow, C.; Illing, S.; Schartner, T.; Grieger, J.; Kirchner, I.; Rust, H.; Cubasch, U.; Ulbrich, U.

2017-12-01

The Freie Univ Evaluation System Framework (Freva - freva.met.fu-berlin.de) is a software infrastructure for standardized data and tool solutions in Earth system science (e.g. www-miklip.dkrz.de, cmip-eval.dkrz.de). Freva runs on high performance computers to handle customizable evaluation systems of research projects, institutes or universities. It combines different software technologies into one common hybrid infrastructure, including all features present in the shell and web environment. The database interface satisfies the international standards provided by the Earth System Grid Federation (ESGF). Freva indexes different data projects into one common search environment by storing the meta data information of the self-describing model, reanalysis and observational data sets in a database. This implemented meta data system with its advanced but easy-to-handle search tool supports users, developers and their plugins to retrieve the required information. A generic application programming interface (API) allows scientific developers to connect their analysis tools with the evaluation system independently of the programming language used. Users of the evaluation techniques benefit from the common interface of the evaluation system without any need to understand the different scripting languages. The integrated web-shell (shellinabox) adds a degree of freedom in the choice of the working environment and can be used as a gate to the research projects HPC. Plugins are able to integrate their e.g. post-processed results into the database of the user. This allows e.g. post-processing plugins to feed statistical analysis plugins, which fosters an active exchange between plugin developers of a research project. Additionally, the history and configuration sub-system stores every analysis performed with the evaluation system in a database. Configurations and results of the tools can be shared among scientists via shell or web system. Furthermore, if configurations match while starting an evaluation plugin, the system suggests to use results already produced by other users - saving CPU/h, I/O, disk space and time. The efficient interaction between different technologies improves the Earth system modeling science framed by Freva.

Three-dimensional non-destructive optical evaluation of laser-processing performance using optical coherence tomography.

PubMed

Kim, Youngseop; Choi, Eun Seo; Kwak, Wooseop; Shin, Yongjin; Jung, Woonggyu; Ahn, Yeh-Chan; Chen, Zhongping

2008-06-01

We demonstrate the use of optical coherence tomography (OCT) as a non-destructive diagnostic tool for evaluating laser-processing performance by imaging the features of a pit and a rim. A pit formed on a material at different laser-processing conditions is imaged using both a conventional scanning electron microscope (SEM) and OCT. Then using corresponding images, the geometrical characteristics of the pit are analyzed and compared. From the results, we could verify the feasibility and the potential of the application of OCT to the monitoring of the laser-processing performance.

Space station common module power system network topology and hardware development

NASA Technical Reports Server (NTRS)

Landis, D. M.

1985-01-01

Candidate power system newtork topologies for the space station common module are defined and developed and the necessary hardware for test and evaluation is provided. Martin Marietta's approach to performing the proposed program is presented. Performance of the tasks described will assure systematic development and evaluation of program results, and will provide the necessary management tools, visibility, and control techniques for performance assessment. The plan is submitted in accordance with the data requirements given and includes a comprehensive task logic flow diagram, time phased manpower requirements, a program milestone schedule, and detailed descriptions of each program task.

Three-dimensional non-destructive optical evaluation of laser-processing performance using optical coherence tomography

PubMed Central

Kim, Youngseop; Choi, Eun Seo; Kwak, Wooseop; Shin, Yongjin; Jung, Woonggyu; Ahn, Yeh-Chan; Chen, Zhongping

2014-01-01

We demonstrate the use of optical coherence tomography (OCT) as a non-destructive diagnostic tool for evaluating laser-processing performance by imaging the features of a pit and a rim. A pit formed on a material at different laser-processing conditions is imaged using both a conventional scanning electron microscope (SEM) and OCT. Then using corresponding images, the geometrical characteristics of the pit are analyzed and compared. From the results, we could verify the feasibility and the potential of the application of OCT to the monitoring of the laser-processing performance. PMID:24932051

Design and evaluation of a slave manipulator with roll-pitch-roll wrist and automatic tool loading mechanism in telerobotic surgery.

PubMed

Kim, Ki-Young; Lee, Jung-Ju

2012-12-01

As there is a shortage of scrub nurses in many hospitals, automatic surgical tool exchanging mechanism without human labour has been studied. Minimally invasive robotic surgeries (MIRS) also require scrub nurses. A surgical tool loading mechanism without a scrub nurse's assistance for MIRS is proposed. Many researchers have developed minimally invasive surgical instruments with a wrist joint that can be movable inside the abdomen. However, implementation of a distal rolling joint on a gripper is rare. To implement surgical tool exchanging without a scrub nurse's assistance, a slave manipulator and a tool loader were developed to load and unload a surgical tool unit. A surgical tool unit with a roll-pitch-roll wrist was developed. Several experiments were performed to validate the effectiveness of the slave manipulator and the surgical tool unit. The slave manipulator and the tool loader were able to successfully unload and load the surgical tool unit without human assistance. The total duration of unloading and loading the surgical tool unit was 97 s. Motion tracking experiments of the distal rolling joint were performed. The maximum positioning error of the step input response was 2°. The advantage of the proposed slave manipulator and tool loader is that other robotic systems or human labour are not needed for surgical tool loading. The feasibility of the distal rolling joint in MIS is verified. Copyright © 2012 John Wiley & Sons, Ltd.

Deterministic and fuzzy-based methods to evaluate community resilience

NASA Astrophysics Data System (ADS)

Kammouh, Omar; Noori, Ali Zamani; Taurino, Veronica; Mahin, Stephen A.; Cimellaro, Gian Paolo

2018-04-01

Community resilience is becoming a growing concern for authorities and decision makers. This paper introduces two indicator-based methods to evaluate the resilience of communities based on the PEOPLES framework. PEOPLES is a multi-layered framework that defines community resilience using seven dimensions. Each of the dimensions is described through a set of resilience indicators collected from literature and they are linked to a measure allowing the analytical computation of the indicator's performance. The first method proposed in this paper requires data on previous disasters as an input and returns as output a performance function for each indicator and a performance function for the whole community. The second method exploits a knowledge-based fuzzy modeling for its implementation. This method allows a quantitative evaluation of the PEOPLES indicators using descriptive knowledge rather than deterministic data including the uncertainty involved in the analysis. The output of the fuzzy-based method is a resilience index for each indicator as well as a resilience index for the community. The paper also introduces an open source online tool in which the first method is implemented. A case study illustrating the application of the first method and the usage of the tool is also provided in the paper.

The Getting To Outcomes Demonstration and Evaluation: An Illustration of the Prevention Support System

PubMed Central

Hunter, Sarah B.; Ebener, Patricia; Paddock, Susan M.; Stillman, Lindsey; Imm, Pamela; Wandersman, Abraham

2010-01-01

Communities are increasingly being required by state and federal funders to achieve outcomes and be accountable, yet are often not provided the guidance or the tools needed to successfully meet this challenge. To improve the likelihood of achieving positive outcomes, the Getting To Outcomes (GTO) intervention (manual, training, technical assistance) is designed to provide the necessary guidance and tools, tailored to community needs, in order to build individual capacity and program performance. GTO is an example of a Prevention Support System intervention, which as conceptualized by the Interactive Systems Framework, plays a key role in bridging the gap between prevention science (Prevention Synthesis and Translation System) and prevention practice (Prevention Delivery System). We evaluated the impact of GTO on individual capacity and program performance using survey- and interview-based methods. We tracked the implementation of GTO and gathered user feedback about its utility and acceptability. The evaluation of GTO suggests that it can build individual capacity and program performance and as such demonstrates that the Prevention Support System can successfully fulfill its intended role. Lessons learned from the implementation of GTO relevant to illuminating the framework are discussed. PMID:18278551

Performance of an online translation tool when applied to patient educational material.

PubMed

Khanna, Raman R; Karliner, Leah S; Eck, Matthias; Vittinghoff, Eric; Koenig, Christopher J; Fang, Margaret C

2011-11-01

Language barriers may prevent clinicians from tailoring patient educational material to the needs of individuals with limited English proficiency. Online translation tools could fill this gap, but their accuracy is unknown. We evaluated the accuracy of an online translation tool for patient educational material. We selected 45 sentences from a pamphlet available in both English and Spanish, and translated it into Spanish using GoogleTranslate™ (GT). Three bilingual Spanish speakers then performed a blinded evaluation on these 45 sentences, comparing GT-translated sentences to those translated professionally, along four domains: fluency (grammatical correctness), adequacy (information preservation), meaning (connotation maintenance), and severity (perceived dangerousness of an error if present). In addition, evaluators indicated whether they had a preference for either the GT-translated or professionally translated sentences. The GT-translated sentences had significantly lower fluency scores compared to the professional translation (3.4 vs. 4.7, P < 0.001), but similar adequacy (4.2 vs. 4.5, P = 0.19) and meaning (4.5 vs. 4.8, P = 0.29) scores. The GT-translated sentences were more likely to have any error (39% vs. 22%, P = 0.05), but not statistically more likely to have a severe error (4% vs. 2%, P = 0.61). Evaluators preferred the professional translation for complex sentences, but not for simple ones. When applied to patient educational material, GT performed comparably to professional human translation in terms of preserving information and meaning, though it was slightly worse in preserving grammar. In situations where professional human translations are unavailable or impractical, online translation may someday fill an important niche. Copyright © 2011 Society of Hospital Medicine.

Automated Guideway Ground Transportation Network Simulation

DOT National Transportation Integrated Search

1975-08-01

The report discusses some automated guideway management problems relating to ground transportation systems and provides an outline of the types of models and algorithms that could be used to develop simulation tools for evaluating system performance....

Materials Flow through Industry Supply Chain Modeling Tool | Advanced

Science.gov Websites

efficiency. It also performs supply chain scale analyses to quantify the impacts and benefits of next , read Evaluating opportunities to improve material and energy impacts in commodity supply chains

Data and software tools for gamma radiation spectral threat detection and nuclide identification algorithm development and evaluation

NASA Astrophysics Data System (ADS)

Portnoy, David; Fisher, Brian; Phifer, Daniel

2015-06-01

The detection of radiological and nuclear threats is extremely important to national security. The federal government is spending significant resources developing new detection systems and attempting to increase the performance of existing ones. The detection of illicit radionuclides that may pose a radiological or nuclear threat is a challenging problem complicated by benign radiation sources (e.g., cat litter and medical treatments), shielding, and large variations in background radiation. Although there is a growing acceptance within the community that concentrating efforts on algorithm development (independent of the specifics of fully assembled systems) has the potential for significant overall system performance gains, there are two major hindrances to advancements in gamma spectral analysis algorithms under the current paradigm: access to data and common performance metrics along with baseline performance measures. Because many of the signatures collected during performance measurement campaigns are classified, dissemination to algorithm developers is extremely limited. This leaves developers no choice but to collect their own data if they are lucky enough to have access to material and sensors. This is often combined with their own definition of metrics for measuring performance. These two conditions make it all but impossible for developers and external reviewers to make meaningful comparisons between algorithms. Without meaningful comparisons, performance advancements become very hard to achieve and (more importantly) recognize. The objective of this work is to overcome these obstacles by developing and freely distributing real and synthetically generated gamma-spectra data sets as well as software tools for performance evaluation with associated performance baselines to national labs, academic institutions, government agencies, and industry. At present, datasets for two tracks, or application domains, have been developed: one that includes temporal spectral data at 1 s time intervals, which represents data collected by a mobile system operating in a dynamic radiation background environment; and one that represents static measurements with a foreground spectrum (background plus source) and a background spectrum. These data include controlled variations in both Source Related Factors (nuclide, nuclide combinations, activities, distances, collection times, shielding configurations, and background spectra) and Detector Related Factors (currently only gain shifts, but resolution changes and non-linear energy calibration errors will be added soon). The software tools will allow the developer to evaluate the performance impact of each of these factors. Although this first implementation is somewhat limited in scope, considering only NaI-based detection systems and two application domains, it is hoped that (with community feedback) a wider range of detector types and applications will be included in the future. This article describes the methods used for dataset creation, the software validation/performance measurement tools, the performance metrics used, and examples of baseline performance.

[A quality evaluation tableau for health institutions: an educational tool].

PubMed

Moll, Marie Christine; Decavel, Frédérique; Merlet, Christine

2009-09-01

For a few years, health institutions have had to comply with the certification and the need to establish the new governance. Thanks to the accreditation version 2 (obtained in 2005), the elaboration of the hospital project (adopted in October, 2006) and the organization in poles since 2006, the quality oriented management became a priority axis at the University Hospital of Angers. The strategic adaptation to quality requirements leads to develop the hospital management, more especially at the level of the clinical, medico technical and administrative poles. The elements of the hospital project including the part about the quality, risk and evaluation aim at being adapted by every pole according to the level of its project. This adaptation which is imposed to each pole manager requires a practical and educational accompaniment allowing at the same time to realize a diagnosis of the progress of the quality approach, a measure of the impact of the global impregnation within the institution and a comparison between pole. A eight axis dashboard with criteria and a user guide were developed from certification ISO 9001, the EFQM manual and the certification manual version 2 of the Healthcare High Authorities. The criteria are transcribed in an EXCEL grid ready to use. Succeeding in estimating your own quality system means that you demonstrate the maturity of the quality approach. The results of this evaluation confirmed those of the certification. The dashboard is a management structuring tool at the service of the multidisciplinary team. Two considerations emerge from these results: First of all, for the hospital top management, the axes to be improved emerge as a priority to determine and target the next annual action plans. The results also allow to support the auto evaluation for the certification version 2010 planned in January of the same year. It is a pragmatic tool which allows auto evaluation and comparison to estimate the pole performances. It is a strategic driving tool for the hospital, for the quality department and for the pole. The pole quality system evaluation dashboard for Healthcare institutions is a simple tool which allies strategy and performance for a better efficiency of the actions to be taken to improve the quality, the risk management and the evaluation of the poles for a better service to the hospital users.

Applying operational research and data mining to performance based medical personnel motivation system.

PubMed

Niaksu, Olegas; Zaptorius, Jonas

2014-01-01

This paper presents the methodology suitable for creation of a performance related remuneration system in healthcare sector, which would meet requirements for efficiency and sustainable quality of healthcare services. Methodology for performance indicators selection, ranking and a posteriori evaluation has been proposed and discussed. Priority Distribution Method is applied for unbiased performance criteria weighting. Data mining methods are proposed to monitor and evaluate the results of motivation system.We developed a method for healthcare specific criteria selection consisting of 8 steps; proposed and demonstrated application of Priority Distribution Method for the selected criteria weighting. Moreover, a set of data mining methods for evaluation of the motivational system outcomes was proposed. The described methodology for calculating performance related payment needs practical approbation. We plan to develop semi-automated tools for institutional and personal performance indicators monitoring. The final step would be approbation of the methodology in a healthcare facility.

A reliability and validity study of the Palliative Performance Scale

PubMed Central

Ho, Francis; Lau, Francis; Downing, Michael G; Lesperance, Mary

2008-01-01

Background The Palliative Performance Scale (PPS) was first introduced in1996 as a new tool for measurement of performance status in palliative care. PPS has been used in many countries and has been translated into other languages. Methods This study evaluated the reliability and validity of PPS. A web-based, case scenarios study with a test-retest format was used to determine reliability. Fifty-three participants were recruited and randomly divided into two groups, each evaluating 11 cases at two time points. The validity study was based on the content validation of 15 palliative care experts conducted over telephone interviews, with discussion on five themes: PPS as clinical assessment tool, the usefulness of PPS, PPS scores affecting decision making, the problems in using PPS, and the adequacy of PPS instruction. Results The intraclass correlation coefficients for absolute agreement were 0.959 and 0.964 for Group 1, at Time-1 and Time-2; 0.951 and 0.931 for Group 2, at Time-1 and Time-2 respectively. Results showed that the participants were consistent in their scoring over the two times, with a mean Cohen's kappa of 0.67 for Group 1 and 0.71 for Group 2. In the validity study, all experts agreed that PPS is a valuable clinical assessment tool in palliative care. Many of them have already incorporated PPS as part of their practice standard. Conclusion The results of the reliability study demonstrated that PPS is a reliable tool. The validity study found that most experts did not feel a need to further modify PPS and, only two experts requested that some performance status measures be defined more clearly. Areas of PPS use include prognostication, disease monitoring, care planning, hospital resource allocation, clinical teaching and research. PPS is also a good communication tool between palliative care workers. PMID:18680590

Information theoretic analysis of edge detection in visual communication

NASA Astrophysics Data System (ADS)

Jiang, Bo; Rahman, Zia-ur

2010-08-01

Generally, the designs of digital image processing algorithms and image gathering devices remain separate. Consequently, the performance of digital image processing algorithms is evaluated without taking into account the artifacts introduced into the process by the image gathering process. However, experiments show that the image gathering process profoundly impacts the performance of digital image processing and the quality of the resulting images. Huck et al. proposed one definitive theoretic analysis of visual communication channels, where the different parts, such as image gathering, processing, and display, are assessed in an integrated manner using Shannon's information theory. In this paper, we perform an end-to-end information theory based system analysis to assess edge detection methods. We evaluate the performance of the different algorithms as a function of the characteristics of the scene, and the parameters, such as sampling, additive noise etc., that define the image gathering system. The edge detection algorithm is regarded to have high performance only if the information rate from the scene to the edge approaches the maximum possible. This goal can be achieved only by jointly optimizing all processes. People generally use subjective judgment to compare different edge detection methods. There is not a common tool that can be used to evaluate the performance of the different algorithms, and to give people a guide for selecting the best algorithm for a given system or scene. Our information-theoretic assessment becomes this new tool to which allows us to compare the different edge detection operators in a common environment.

Evaluation of operational, economic, and environmental performance of mixed and selective collection of municipal solid waste: Porto case study.

PubMed

Teixeira, Carlos A; Russo, Mário; Matos, Cristina; Bentes, Isabel

2014-12-01

This article describes an accurate methodology for an operational, economic, and environmental assessment of municipal solid waste collection. The proposed methodological tool uses key performance indicators to evaluate independent operational and economic efficiency and performance of municipal solid waste collection practices. These key performance indicators are then used in life cycle inventories and life cycle impact assessment. Finally, the life cycle assessment environmental profiles provide the environmental assessment. We also report a successful application of this tool through a case study in the Portuguese city of Porto. Preliminary results demonstrate the applicability of the methodological tool to real cases. Some of the findings focus a significant difference between average mixed and selective collection effective distance (2.14 km t(-1); 16.12 km t(-1)), fuel consumption (3.96 L t(-1); 15.37 L t(-1)), crew productivity (0.98 t h(-1) worker(-1); 0.23 t h(-1) worker(-1)), cost (45.90 € t(-1); 241.20 € t(-1)), and global warming impact (19.95 kg CO2eq t(-1); 57.47 kg CO2eq t(-1)). Preliminary results consistently indicate: (a) higher global performance of mixed collection as compared with selective collection; (b) dependency of collection performance, even in urban areas, on the waste generation rate and density; (c) the decline of selective collection performances with decreasing source-separated material density and recycling collection rate; and (d) that the main threats to collection route efficiency are the extensive collection distances, high fuel consumption vehicles, and reduced crew productivity. © The Author(s) 2014.

RADC thermal guide for reliability engineers

NASA Astrophysics Data System (ADS)

Morrison, G. N.; Kallis, J. M.; Strattan, L. A.; Jones, I. R.; Lena, A. L.

1982-06-01

This guide was developed to provide a reliability engineer, who is not proficient in thermal design and analysis techniques, with the tools for managing and evaluating the thermal design and production of electronic equipment. It defines the requirements and tasks that should be addressed in system equipment specifications and statements of work, and describes how to evaluate performance.

[Recommendations for the evaluation and follow-up of the continuous quality improvement].

PubMed

Maurellet-Evrard, S; Daunizeau, A

2013-06-01

Continual improvement of the quality in a medical laboratory is based on the implementation of tools for systematically evaluate the quality management system and its ability to meet the objectives defined. Monitoring through audit and management review, addressing complaints and nonconformities and performing client satisfaction survey are the key for the continual improvement.

Peer reviews: taking on new meanings.

PubMed

Raia, Lucille

2011-01-01

Peer reviews in nursing are historically used to gauge performance within an individual's scope of practice or as a tool to evaluate a sentinel or adverse event. Quality of care measures, clinical pertinence, and evaluating standards of care have begun as parallel strategies to replace the former uses in assuring the right care at the right time in the right setting.

Surveillance strategies for Classical Swine Fever in wild boar – a comprehensive evaluation study to ensure powerful surveillance

PubMed Central

Schulz, Katja; Peyre, Marisa; Staubach, Christoph; Schauer, Birgit; Schulz, Jana; Calba, Clémentine; Häsler, Barbara; Conraths, Franz J.

2017-01-01

Surveillance of Classical Swine Fever (CSF) should not only focus on livestock, but must also include wild boar. To prevent disease transmission into commercial pig herds, it is therefore vital to have knowledge about the disease status in wild boar. In the present study, we performed a comprehensive evaluation of alternative surveillance strategies for Classical Swine Fever (CSF) in wild boar and compared them with the currently implemented conventional approach. The evaluation protocol was designed using the EVA tool, a decision support tool to help in the development of an economic and epidemiological evaluation protocol for surveillance. To evaluate the effectiveness of the surveillance strategies, we investigated their sensitivity and timeliness. Acceptability was analysed and finally, the cost-effectiveness of the surveillance strategies was determined. We developed 69 surveillance strategies for comparative evaluation between the existing approach and the novel proposed strategies. Sampling only within sub-adults resulted in a better acceptability and timeliness than the currently implemented strategy. Strategies that were completely based on passive surveillance performance did not achieve the desired detection probability of 95%. In conclusion, the results of the study suggest that risk-based approaches can be an option to design more effective CSF surveillance strategies in wild boar. PMID:28266576