A framework for outcome-level evaluation of in-service training of health care workers

PubMed Central

2013-01-01

Background In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President’s Emergency Plan for AIDS Relief’s Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. Methods A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. Results The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the ability of evaluators to determine training outcomes. In addition, a group of user-friendly resources, the Training Evaluation Framework and Tools (TEFT) were created to help evaluators and stakeholders understand and apply the framework. Conclusions Feedback from pilot users suggests that using the framework and accompanying tools may support outcome evaluation planning. Further assessment will assist in strengthening guidelines and tools for operationalization. PMID:24083635

Advancing the use of performance evaluation in health care.

PubMed

Traberg, Andreas; Jacobsen, Peter; Duthiers, Nadia Monique

2014-01-01

The purpose of this paper is to develop a framework for health care performance evaluation that enables decision makers to identify areas indicative of corrective actions. The framework should provide information on strategic pro-/regress in an operational context that justifies the need for organizational adjustments. The study adopts qualitative methods for constructing the framework, subsequently implementing the framework in a Danish magnetic resonance imaging (MRI) unit. Workshops and interviews form the basis of the qualitative construction phase, and two internal and five external databases are used for a quantitative data collection. By aggregating performance outcomes, collective measures of performance are achieved. This enables easy and intuitive identification of areas not strategically aligned. In general, the framework has proven helpful in an MRI unit, where operational decision makers have been struggling with extensive amounts of performance information. The implementation of the framework in a single case in a public and highly political environment restricts the generalizing potential. The authors acknowledge that there may be more suitable approaches in organizations with different settings. The strength of the framework lies in the identification of performance problems prior to decision making. The quality of decisions is directly related to the individual decision maker. The only function of the framework is to support these decisions. The study demonstrates a more refined and transparent use of performance reporting by combining strategic weight assignment and performance aggregation in hierarchies. In this way, the framework accentuates performance as a function of strategic progress or regress, thus assisting decision makers in exerting operational effort in pursuit of strategic alignment.

A conceptual framework for evaluation of public health and primary care system performance in iran.

PubMed

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Akbari Sari, Ali; Mesdaghinia, Alireza

2015-01-26

The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report.

Evaluation Framework and Analyses for Thermal Energy Storage Integrated with Packaged Air Conditioning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kung, F.; Deru, M.; Bonnema, E.

2013-10-01

Few third-party guidance documents or tools are available for evaluating thermal energy storage (TES) integrated with packaged air conditioning (AC), as this type of TES is relatively new compared to TES integrated with chillers or hot water systems. To address this gap, researchers at the National Renewable Energy Laboratory conducted a project to improve the ability of potential technology adopters to evaluate TES technologies. Major project outcomes included: development of an evaluation framework to describe key metrics, methodologies, and issues to consider when assessing the performance of TES systems integrated with packaged AC; application of multiple concepts from the evaluationmore » framework to analyze performance data from four demonstration sites; and production of a new simulation capability that enables modeling of TES integrated with packaged AC in EnergyPlus. This report includes the evaluation framework and analysis results from the project.« less

Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases : a review and suggested reporting framework.

PubMed

Haji Ali Afzali, Hossein; Gray, Jodi; Karnon, Jonathan

2013-04-01

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess 'best practice' in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.

PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems

PubMed Central

Aqil, Anwer; Lippeveld, Theo; Hozumi, Dairiku

2009-01-01

The utility and effectiveness of routine health information systems (RHIS) in improving health system performance in developing countries has been questioned. This paper argues that the health system needs internal mechanisms to develop performance targets, track progress, and create and manage knowledge for continuous improvement. Based on documented RHIS weaknesses, we have developed the Performance of Routine Information System Management (PRISM) framework, an innovative approach to design, strengthen and evaluate RHIS. The PRISM framework offers a paradigm shift by putting emphasis on RHIS performance and incorporating the organizational, technical and behavioural determinants of performance. By describing causal pathways of these determinants, the PRISM framework encourages and guides the development of interventions for strengthening or reforming RHIS. Furthermore, it conceptualizes and proposes a methodology for measuring the impact of RHIS on health system performance. Ultimately, the PRISM framework, in spite of its challenges and competing paradigms, proposes a new agenda for building and sustaining information systems, for the promotion of an information culture, and for encouraging accountability in health systems. PMID:19304786

A Conceptual Framework to Help Evaluate the Quality of Institutional Performance

ERIC Educational Resources Information Center

Kettunen, Juha

2008-01-01

Purpose: This study aims to present a general conceptual framework which can be used to evaluate quality and institutional performance in higher education. Design/methodology/approach: The quality of higher education is at the heart of the setting up of the European Higher Education Area. Strategic management is widely used in higher education…

The participatory design of a performance oriented monitoring and evaluation system in an international development environment.

PubMed

Guerra-López, Ingrid; Hicks, Karen

2015-02-01

This article illustrates the application of the impact monitoring and evaluation process for the design and development of a performance monitoring and evaluation framework in the context of human and institutional capacity development. This participative process facilitated stakeholder ownership in several areas including the design, development, and use of a new monitoring and evaluation system, as well their targeted results and accomplishments through the use of timely performance data gathered through ongoing monitoring and evaluation. The process produced a performance indicator map, a comprehensive monitoring and evaluation framework, and data collection templates to promote the development, implementation, and sustainability of the monitoring and evaluation system of a farmer's trade union in an African country. Copyright © 2014 Elsevier Ltd. All rights reserved.

An open-source framework for large-scale, flexible evaluation of biomedical text mining systems.

PubMed

Baumgartner, William A; Cohen, K Bretonnel; Hunter, Lawrence

2008-01-29

Improved evaluation methodologies have been identified as a necessary prerequisite to the improvement of text mining theory and practice. This paper presents a publicly available framework that facilitates thorough, structured, and large-scale evaluations of text mining technologies. The extensibility of this framework and its ability to uncover system-wide characteristics by analyzing component parts as well as its usefulness for facilitating third-party application integration are demonstrated through examples in the biomedical domain. Our evaluation framework was assembled using the Unstructured Information Management Architecture. It was used to analyze a set of gene mention identification systems involving 225 combinations of system, evaluation corpus, and correctness measure. Interactions between all three were found to affect the relative rankings of the systems. A second experiment evaluated gene normalization system performance using as input 4,097 combinations of gene mention systems and gene mention system-combining strategies. Gene mention system recall is shown to affect gene normalization system performance much more than does gene mention system precision, and high gene normalization performance is shown to be achievable with remarkably low levels of gene mention system precision. The software presented in this paper demonstrates the potential for novel discovery resulting from the structured evaluation of biomedical language processing systems, as well as the usefulness of such an evaluation framework for promoting collaboration between developers of biomedical language processing technologies. The code base is available as part of the BioNLP UIMA Component Repository on SourceForge.net.

An open-source framework for large-scale, flexible evaluation of biomedical text mining systems

PubMed Central

Baumgartner, William A; Cohen, K Bretonnel; Hunter, Lawrence

2008-01-01

Background Improved evaluation methodologies have been identified as a necessary prerequisite to the improvement of text mining theory and practice. This paper presents a publicly available framework that facilitates thorough, structured, and large-scale evaluations of text mining technologies. The extensibility of this framework and its ability to uncover system-wide characteristics by analyzing component parts as well as its usefulness for facilitating third-party application integration are demonstrated through examples in the biomedical domain. Results Our evaluation framework was assembled using the Unstructured Information Management Architecture. It was used to analyze a set of gene mention identification systems involving 225 combinations of system, evaluation corpus, and correctness measure. Interactions between all three were found to affect the relative rankings of the systems. A second experiment evaluated gene normalization system performance using as input 4,097 combinations of gene mention systems and gene mention system-combining strategies. Gene mention system recall is shown to affect gene normalization system performance much more than does gene mention system precision, and high gene normalization performance is shown to be achievable with remarkably low levels of gene mention system precision. Conclusion The software presented in this paper demonstrates the potential for novel discovery resulting from the structured evaluation of biomedical language processing systems, as well as the usefulness of such an evaluation framework for promoting collaboration between developers of biomedical language processing technologies. The code base is available as part of the BioNLP UIMA Component Repository on SourceForge.net. PMID:18230184

A Conceptual Framework for Evaluation of Public Health and Primary Care System Performance in Iran

PubMed Central

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Sari, Ali Akbari; Mesdaghinia, Alireza

2015-01-01

Introduction: The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. Methods: We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. Results: We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. Conclusion: The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report. PMID:25946937

The Development of a Quality Management Framework for Evaluating Medical Device Reprocessing Practice in Healthcare Facilities.

PubMed

Lorv, Bailey; Horodyski, Robin; Welton, Cynthia; Vail, John; Simonetto, Luca; Jokanovic, Danilo; Sharma, Richa; Mahoney, Angela Rea; Savoy-Bird, Shay; Bains, Shalu

2017-01-01

There is increasing awareness of the importance of medical device reprocessing (MDR) for the provision of safe patient care. Although industry service standards are available to guide MDR practices, there remains a lack of published key performance indicators (KPIs) and targets that are necessary to evaluate MDR quality for feedback and improvement. This article outlines the development of an initial framework that builds on established guidelines and includes service standards, KPIs and targets for evaluating MDR operations. This framework can support healthcare facilities in strengthening existing practices and enables a platform for collaboration towards better MDR performance management.

Introducing an accountability framework for polio eradication in Ethiopia: results from the first year of implementation 2014-2015

PubMed Central

Kassahun, Aron; Braka, Fiona; Gallagher, Kathleen; Gebriel, Aregai Wolde; Nsubuga, Peter; M’pele-Kilebou, Pierre

2017-01-01

Introduction the World Health Organization (WHO), Ethiopia country office, introduced an accountability framework into its Polio Eradication Program in 2014 with the aim of improving the program's performance. Our study aims to evaluate staff performance and key program indicators following the introduction of the accountability framework. Methods the impact of the WHO accountability framework was reviewed after its first year of implementation from June 2014 to June 2015. We analyzed selected program and staff performance indicators associated with acute flaccid paralysis (AFP) surveillance from a database available at WHO. Data on managerial actions taken were also reviewed. Performance of a total of 38 staff was evaluated during our review. Results our review of results for the first four quarters of implementation of the polio eradication accountability framework showed improvement both at the program and individual level when compared with the previous year. Managerial actions taken during the study period based on the results from the monitoring tool included eleven written acknowledgments, six discussions regarding performance improvement, six rotations of staff, four written first-warning letters and nine non-renewal of contracts. Conclusion the introduction of the accountability framework resulted in improvement in staff performance and overall program indicators for AFP surveillance. PMID:28890753

A Computational Framework for Quantitative Evaluation of Movement during Rehabilitation

NASA Astrophysics Data System (ADS)

Chen, Yinpeng; Duff, Margaret; Lehrer, Nicole; Sundaram, Hari; He, Jiping; Wolf, Steven L.; Rikakis, Thanassis

2011-06-01

This paper presents a novel generalized computational framework for quantitative kinematic evaluation of movement in a rehabilitation clinic setting. The framework integrates clinical knowledge and computational data-driven analysis together in a systematic manner. The framework provides three key benefits to rehabilitation: (a) the resulting continuous normalized measure allows the clinician to monitor movement quality on a fine scale and easily compare impairments across participants, (b) the framework reveals the effect of individual movement components on the composite movement performance helping the clinician decide the training foci, and (c) the evaluation runs in real-time, which allows the clinician to constantly track a patient's progress and make appropriate adaptations to the therapy protocol. The creation of such an evaluation is difficult because of the sparse amount of recorded clinical observations, the high dimensionality of movement and high variations in subject's performance. We address these issues by modeling the evaluation function as linear combination of multiple normalized kinematic attributes y = Σwiφi(xi) and estimating the attribute normalization function φi(ṡ) by integrating distributions of idealized movement and deviated movement. The weights wi are derived from a therapist's pair-wise comparison using a modified RankSVM algorithm. We have applied this framework to evaluate upper limb movement for stroke survivors with excellent results—the evaluation results are highly correlated to the therapist's observations.

Software Engineering Support of the Third Round of Scientific Grand Challenge Investigations: Earth System Modeling Software Framework Survey

NASA Technical Reports Server (NTRS)

Talbot, Bryan; Zhou, Shu-Jia; Higgins, Glenn; Zukor, Dorothy (Technical Monitor)

2002-01-01

One of the most significant challenges in large-scale climate modeling, as well as in high-performance computing in other scientific fields, is that of effectively integrating many software models from multiple contributors. A software framework facilitates the integration task, both in the development and runtime stages of the simulation. Effective software frameworks reduce the programming burden for the investigators, freeing them to focus more on the science and less on the parallel communication implementation. while maintaining high performance across numerous supercomputer and workstation architectures. This document surveys numerous software frameworks for potential use in Earth science modeling. Several frameworks are evaluated in depth, including Parallel Object-Oriented Methods and Applications (POOMA), Cactus (from (he relativistic physics community), Overture, Goddard Earth Modeling System (GEMS), the National Center for Atmospheric Research Flux Coupler, and UCLA/UCB Distributed Data Broker (DDB). Frameworks evaluated in less detail include ROOT, Parallel Application Workspace (PAWS), and Advanced Large-Scale Integrated Computational Environment (ALICE). A host of other frameworks and related tools are referenced in this context. The frameworks are evaluated individually and also compared with each other.

The Context and Process for Performance Evaluations: Necessary Preconditions for the Use of Performance Evaluations as a Measure of Performance--A Critique of Perry

ERIC Educational Resources Information Center

McCarthy, Mary L.

2006-01-01

This article challenges Perry's research using performance evaluations to determine whether the educational background of child welfare workers is predictive of performance. Institutional theory, an understanding of street-level bureaucracies, and evaluations of field education performance measures are offered as necessary frameworks for Perry's…

Factors affecting construction performance: exploratory factor analysis

NASA Astrophysics Data System (ADS)

Soewin, E.; Chinda, T.

2018-04-01

The present work attempts to develop a multidimensional performance evaluation framework for a construction company by considering all relevant measures of performance. Based on the previous studies, this study hypothesizes nine key factors, with a total of 57 associated items. The hypothesized factors, with their associated items, are then used to develop questionnaire survey to gather data. The exploratory factor analysis (EFA) was applied to the collected data which gave rise 10 factors with 57 items affecting construction performance. The findings further reveal that the items constituting ten key performance factors (KPIs) namely; 1) Time, 2) Cost, 3) Quality, 4) Safety & Health, 5) Internal Stakeholder, 6) External Stakeholder, 7) Client Satisfaction, 8) Financial Performance, 9) Environment, and 10) Information, Technology & Innovation. The analysis helps to develop multi-dimensional performance evaluation framework for an effective measurement of the construction performance. The 10 key performance factors can be broadly categorized into economic aspect, social aspect, environmental aspect, and technology aspects. It is important to understand a multi-dimension performance evaluation framework by including all key factors affecting the construction performance of a company, so that the management level can effectively plan to implement an effective performance development plan to match with the mission and vision of the company.

Estimating Track Capacity Based on Rail Stresses and Metal Fatigue.

DOT National Transportation Integrated Search

2011-09-21

This paper describes a framework to evaluate the structural capacity of railroad track to train-induced loads. The framework is applied to estimate structural performance in terms of allowable limits for crosstie spacing. Evaluation of the load-carry...

Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol.

PubMed

Kasturi, Rangachar; Goldgof, Dmitry; Soundararajan, Padmanabhan; Manohar, Vasant; Garofolo, John; Bowers, Rachel; Boonstra, Matthew; Korzhova, Valentina; Zhang, Jing

2009-02-01

Common benchmark data sets, standardized performance metrics, and baseline algorithms have demonstrated considerable impact on research and development in a variety of application domains. These resources provide both consumers and developers of technology with a common framework to objectively compare the performance of different algorithms and algorithmic improvements. In this paper, we present such a framework for evaluating object detection and tracking in video: specifically for face, text, and vehicle objects. This framework includes the source video data, ground-truth annotations (along with guidelines for annotation), performance metrics, evaluation protocols, and tools including scoring software and baseline algorithms. For each detection and tracking task and supported domain, we developed a 50-clip training set and a 50-clip test set. Each data clip is approximately 2.5 minutes long and has been completely spatially/temporally annotated at the I-frame level. Each task/domain, therefore, has an associated annotated corpus of approximately 450,000 frames. The scope of such annotation is unprecedented and was designed to begin to support the necessary quantities of data for robust machine learning approaches, as well as a statistically significant comparison of the performance of algorithms. The goal of this work was to systematically address the challenges of object detection and tracking through a common evaluation framework that permits a meaningful objective comparison of techniques, provides the research community with sufficient data for the exploration of automatic modeling techniques, encourages the incorporation of objective evaluation into the development process, and contributes useful lasting resources of a scale and magnitude that will prove to be extremely useful to the computer vision research community for years to come.

Green and sustainable remediation (GSR) evaluation: framework, standards, and tool. A case study in Taiwan.

PubMed

Huang, Wen-Yen; Hung, Weiteng; Vu, Chi Thanh; Chen, Wei-Ting; Lai, Jhih-Wei; Lin, Chitsan

2016-11-01

Taiwan has a large number of poorly managed contaminated sites in need of remediation. This study proposes a framework, a set of standards, and a spreadsheet-based evaluation tool for implementing green and sustainable principles into remediation projects and evaluating the projects from this perspective. We performed a case study to understand how the framework would be applied. For the case study, we used a spreadsheet-based evaluation tool (SEFA) and performed field scale cultivation tests on a site contaminated with total petroleum hydrocarbons (TPHs). The site was divided into two lots: one treated by chemical oxidation and the other by bioremediation. We evaluated five core elements of green and sustainable remediation (GSR): energy, air, water resources, materials and wastes, and land and ecosystem. The proposed evaluation tool and field scale cultivation test were found to efficiently assess the effectiveness of the two remediation alternatives. The framework and related tools proposed herein can potentially be used to support decisions about the remediation of contaminated sites taking into account engineering management, cost effectiveness, and social reconciliation.

Proposed Framework for the Evaluation of Standalone Corpora Processing Systems: An Application to Arabic Corpora

PubMed Central

Al-Thubaity, Abdulmohsen; Alqifari, Reem

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework. PMID:25610910

Proposed framework for the evaluation of standalone corpora processing systems: an application to Arabic corpora.

PubMed

Al-Thubaity, Abdulmohsen; Al-Khalifa, Hend; Alqifari, Reem; Almazrua, Manal

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework.

Benchmarking high performance computing architectures with CMS’ skeleton framework

NASA Astrophysics Data System (ADS)

Sexton-Kennedy, E.; Gartung, P.; Jones, C. D.

2017-10-01

In 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel’s Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta, Mira. Because of this we have revived the 2012 benchmark to test it’s performance and conclusions on these new architectures. This talk will discuss the results of this exercise.

Strategic Frameworks for Human Performance and Human Systems Integration: Spotlight on General Systems Performance Theory and Nonlinear Causal Resource Analysis

DTIC Science & Technology

2010-12-14

Robotics Research Institute Auditorium Riverbend Campus, Fort Worth, TX Report Documentation Page Form ApprovedOMB No. 0704-0188 Public reporting burden...for HP and HSI. Format Participants were presented with information on frameworks and the benefits they can have in support of the various roles... benefit from understanding human performance. She concluded with the objectives of the workshop: 1. Evaluate strategic frameworks for representing

Individual performance review in hospital practice: the development of a framework and evaluation of doctors' attitudes to its value and implementation.

PubMed

Trebble, T M; Cruickshank, L; Hockey, P M; Heyworth, N; Powell, T; Clarke, N

2013-11-01

Appraisal, or independent performance review (IPR) is used in human resources management in the commercial and public sectors to evaluate the performance of an employee against agreed local organisational expectations and objectives, and to identify their requirements for development and effective management. IPR for NHS consultants may provide essential information for job planning, contribute towards medical appraisal for revalidation, and facilitate productivity and quality improvement. To develop a framework for IPR for consultants, and to determine attitudes on its value, process and content. Information from commercial, public and voluntary sector models and published and other literature sources were used to develop an IPR framework. This was assessed through a three-cycle action research methodology involving qualitative interviews with 22 consultants (predominantly with medical management roles). The domains of the IPR framework included: (1) performance against objectives; (2) behaviour and leadership; (3) talent management; (4) agreed future objectives. A number of themes were identified from the consultant interviews including: ineffective current appraisal systems reflecting a lack of valid performance data and allotted time; a lack of empowerment of medical managers to address performance issues; IPR as a more explicit system, offering value in evaluating doctors performance; and the dependence of successful implementation on the engagement of the Trust executive. IPR may have value for performance evaluation of consultants, contributing toward job planning and complementing medical appraisal. Support by their employing organisation and engagement with medical managers in design and implementation is likely to be essential.

Frameworks for evaluating health research capacity strengthening: a qualitative study

PubMed Central

2013-01-01

Background Health research capacity strengthening (RCS) projects are often complex and hard to evaluate. In order to inform health RCS evaluation efforts, we aimed to describe and compare key characteristics of existing health RCS evaluation frameworks: their process of development, purpose, target users, structure, content and coverage of important evaluation issues. A secondary objective was to explore what use had been made of the ESSENCE framework, which attempts to address one such issue: harmonising the evaluation requirements of different funders. Methods We identified and analysed health RCS evaluation frameworks published by seven funding agencies between 2004 and 2012, using a mixed methods approach involving structured qualitative analyses of documents, a stakeholder survey and consultations with key contacts in health RCS funding agencies. Results The frameworks were intended for use predominantly by the organisations themselves, and most were oriented primarily towards funders’ internal organisational performance requirements. The frameworks made limited reference to theories that specifically concern RCS. Generic devices, such as logical frameworks, were typically used to document activities, outputs and outcomes, but with little emphasis on exploring underlying assumptions or contextual constraints. Usage of the ESSENCE framework appeared limited. Conclusions We believe that there is scope for improving frameworks through the incorporation of more accessible information about how to do evaluation in practice; greater involvement of stakeholders, following evaluation capacity building principles; greater emphasis on explaining underlying rationales of frameworks; and structuring frameworks so that they separate generic and project-specific aspects of health RCS evaluation. The third and fourth of these improvements might assist harmonisation. PMID:24330628

New model framework and structure and the commonality evaluation model. [concerning unmanned spacecraft projects

NASA Technical Reports Server (NTRS)

1977-01-01

The development of a framework and structure for shuttle era unmanned spacecraft projects and the development of a commonality evaluation model is documented. The methodology developed for model utilization in performing cost trades and comparative evaluations for commonality studies is discussed. The model framework consists of categories of activities associated with the spacecraft system's development process. The model structure describes the physical elements to be treated as separate identifiable entities. Cost estimating relationships for subsystem and program-level components were calculated.

Building an Evaluation Framework for a Competency-Based Graduate Program at the University Of Southern Mississippi

ERIC Educational Resources Information Center

Gaudet, Cyndi H.; Annulis, Heather M.; Kmiec, John J., Jr.

2008-01-01

This article describes an ongoing project to build a comprehensive evaluation framework for the competency-based Master of Science in Workforce Training and Development (MSWTD) program at The University of Southern Mississippi (USM). First, it discusses some trends and issues in evaluating the performance of higher education programs in the United…

ATR evaluation through the synthesis of multiple performance measures

NASA Astrophysics Data System (ADS)

Bassham, Christopher B.; Klimack, William K.; Bauer, Kenneth W., Jr.

2002-07-01

This research demonstrates the application of decision analysis (DA) techniques to decisions made within Automatic Target Recognition (ATR) technology development. This work is accomplished to improve the means by which ATR technologies are evaluated. The first step in this research was to create a flexible decision analysis framework that could be applied to several decisions across different ATR programs evaluated by the Comprehensive ATR Scientific Evaluation (COMPASE) Center of the Air Force Research Laboratory (AFRL). For the purposes of this research, a single COMPASE Center representative provided the value, utility, and preference functions for the DA framework. The DA framework employs performance measures collected during ATR classification system (CS) testing to calculate value and utility scores. The authors gathered data from the Moving and Stationary Target Acquisition and Recognition (MSTAR) program to demonstrate how the decision framework could be used to evaluate three different ATR CSs. A decision-maker may use the resultant scores to gain insight into any of the decisions that occur throughout the lifecycle of ATR technologies. Additionally, a means of evaluating ATR CS self-assessment ability is presented. This represents a new criterion that emerged from this study, and no present evaluation metric is known.

Quality Attribute Techniques Framework

NASA Astrophysics Data System (ADS)

Chiam, Yin Kia; Zhu, Liming; Staples, Mark

The quality of software is achieved during its development. Development teams use various techniques to investigate, evaluate and control potential quality problems in their systems. These “Quality Attribute Techniques” target specific product qualities such as safety or security. This paper proposes a framework to capture important characteristics of these techniques. The framework is intended to support process tailoring, by facilitating the selection of techniques for inclusion into process models that target specific product qualities. We use risk management as a theory to accommodate techniques for many product qualities and lifecycle phases. Safety techniques have motivated the framework, and safety and performance techniques have been used to evaluate the framework. The evaluation demonstrates the ability of quality risk management to cover the development lifecycle and to accommodate two different product qualities. We identify advantages and limitations of the framework, and discuss future research on the framework.

On the Performance Evaluation of 3D Reconstruction Techniques from a Sequence of Images

NASA Astrophysics Data System (ADS)

Eid, Ahmed; Farag, Aly

2005-12-01

The performance evaluation of 3D reconstruction techniques is not a simple problem to solve. This is not only due to the increased dimensionality of the problem but also due to the lack of standardized and widely accepted testing methodologies. This paper presents a unified framework for the performance evaluation of different 3D reconstruction techniques. This framework includes a general problem formalization, different measuring criteria, and a classification method as a first step in standardizing the evaluation process. Performance characterization of two standard 3D reconstruction techniques, stereo and space carving, is also presented. The evaluation is performed on the same data set using an image reprojection testing methodology to reduce the dimensionality of the evaluation domain. Also, different measuring strategies are presented and applied to the stereo and space carving techniques. These measuring strategies have shown consistent results in quantifying the performance of these techniques. Additional experiments are performed on the space carving technique to study the effect of the number of input images and the camera pose on its performance.

MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans.

PubMed

Mendrik, Adriënne M; Vincken, Koen L; Kuijf, Hugo J; Breeuwer, Marcel; Bouvy, Willem H; de Bresser, Jeroen; Alansary, Amir; de Bruijne, Marleen; Carass, Aaron; El-Baz, Ayman; Jog, Amod; Katyal, Ranveer; Khan, Ali R; van der Lijn, Fedde; Mahmood, Qaiser; Mukherjee, Ryan; van Opbroek, Annegreet; Paneri, Sahil; Pereira, Sérgio; Persson, Mikael; Rajchl, Martin; Sarikaya, Duygu; Smedby, Örjan; Silva, Carlos A; Vrooman, Henri A; Vyas, Saurabh; Wang, Chunliang; Zhao, Liang; Biessels, Geert Jan; Viergever, Max A

2015-01-01

Many methods have been proposed for tissue segmentation in brain MRI scans. The multitude of methods proposed complicates the choice of one method above others. We have therefore established the MRBrainS online evaluation framework for evaluating (semi)automatic algorithms that segment gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) on 3T brain MRI scans of elderly subjects (65-80 y). Participants apply their algorithms to the provided data, after which their results are evaluated and ranked. Full manual segmentations of GM, WM, and CSF are available for all scans and used as the reference standard. Five datasets are provided for training and fifteen for testing. The evaluated methods are ranked based on their overall performance to segment GM, WM, and CSF and evaluated using three evaluation metrics (Dice, H95, and AVD) and the results are published on the MRBrainS13 website. We present the results of eleven segmentation algorithms that participated in the MRBrainS13 challenge workshop at MICCAI, where the framework was launched, and three commonly used freeware packages: FreeSurfer, FSL, and SPM. The MRBrainS evaluation framework provides an objective and direct comparison of all evaluated algorithms and can aid in selecting the best performing method for the segmentation goal at hand.

MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans

PubMed Central

Mendrik, Adriënne M.; Vincken, Koen L.; Kuijf, Hugo J.; Breeuwer, Marcel; Bouvy, Willem H.; de Bresser, Jeroen; Alansary, Amir; de Bruijne, Marleen; Carass, Aaron; El-Baz, Ayman; Jog, Amod; Katyal, Ranveer; Khan, Ali R.; van der Lijn, Fedde; Mahmood, Qaiser; Mukherjee, Ryan; van Opbroek, Annegreet; Paneri, Sahil; Pereira, Sérgio; Rajchl, Martin; Sarikaya, Duygu; Smedby, Örjan; Silva, Carlos A.; Vrooman, Henri A.; Vyas, Saurabh; Wang, Chunliang; Zhao, Liang; Biessels, Geert Jan; Viergever, Max A.

2015-01-01

Many methods have been proposed for tissue segmentation in brain MRI scans. The multitude of methods proposed complicates the choice of one method above others. We have therefore established the MRBrainS online evaluation framework for evaluating (semi)automatic algorithms that segment gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) on 3T brain MRI scans of elderly subjects (65–80 y). Participants apply their algorithms to the provided data, after which their results are evaluated and ranked. Full manual segmentations of GM, WM, and CSF are available for all scans and used as the reference standard. Five datasets are provided for training and fifteen for testing. The evaluated methods are ranked based on their overall performance to segment GM, WM, and CSF and evaluated using three evaluation metrics (Dice, H95, and AVD) and the results are published on the MRBrainS13 website. We present the results of eleven segmentation algorithms that participated in the MRBrainS13 challenge workshop at MICCAI, where the framework was launched, and three commonly used freeware packages: FreeSurfer, FSL, and SPM. The MRBrainS evaluation framework provides an objective and direct comparison of all evaluated algorithms and can aid in selecting the best performing method for the segmentation goal at hand. PMID:26759553

Developing a Multi-Dimensional Evaluation Framework for Faculty Teaching and Service Performance

ERIC Educational Resources Information Center

Baker, Diane F.; Neely, Walter P.; Prenshaw, Penelope J.; Taylor, Patrick A.

2015-01-01

A task force was created in a small, AACSB-accredited business school to develop a more comprehensive set of standards for faculty performance. The task force relied heavily on faculty input to identify and describe key dimensions that capture effective teaching and service performance. The result is a multi-dimensional framework that will be used…

Data-driven freeway performance evaluation framework for project prioritization and decision making.

DOT National Transportation Integrated Search

2017-01-01

This report describes methods that potentially can be incorporated into the performance monitoring and planning processes for freeway performance evaluation and decision making. Reliability analysis was conducted on the selected I-15 corridor by empl...

Data-driven freeway performance evaluation framework for project prioritization and decision making.

DOT National Transportation Integrated Search

2015-03-01

This report describes methods that potentially can be incorporated into the performance monitoring and planning : processes for freeway performance evaluation and decision making. Reliability analysis is conducted on the selected : I-15 corridor by e...

Benchmarking high performance computing architectures with CMS’ skeleton framework

DOE PAGES

Sexton-Kennedy, E.; Gartung, P.; Jones, C. D.

2017-11-23

Here, in 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel’s Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta,more » Mira. Because of this we have revived the 2012 benchmark to test it’s performance and conclusions on these new architectures. This talk will discuss the results of this exercise.« less

Benchmarking high performance computing architectures with CMS’ skeleton framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sexton-Kennedy, E.; Gartung, P.; Jones, C. D.

Here, in 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel’s Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta,more » Mira. Because of this we have revived the 2012 benchmark to test it’s performance and conclusions on these new architectures. This talk will discuss the results of this exercise.« less

Quantitative framework for prospective motion correction evaluation.

PubMed

Pannetier, Nicolas A; Stavrinos, Theano; Ng, Peter; Herbst, Michael; Zaitsev, Maxim; Young, Karl; Matson, Gerald; Schuff, Norbert

2016-02-01

Establishing a framework to evaluate performances of prospective motion correction (PMC) MRI considering motion variability between MRI scans. A framework was developed to obtain quantitative comparisons between different motion correction setups, considering that varying intrinsic motion patterns between acquisitions can induce bias. Intrinsic motion was considered by replaying in a phantom experiment the recorded motion trajectories from subjects. T1-weighted MRI on five volunteers and two different marker fixations (mouth guard and nose bridge fixations) were used to test the framework. Two metrics were investigated to quantify the improvement of the image quality with PMC. Motion patterns vary between subjects as well as between repeated scans within a subject. This variability can be approximated by replaying the motion in a distinct phantom experiment and used as a covariate in models comparing motion corrections. We show that considering the intrinsic motion alters the statistical significance in comparing marker fixations. As an example, two marker fixations, a mouth guard and a nose bridge, were evaluated in terms of their effectiveness for PMC. A mouth guard achieved better PMC performance. Intrinsic motion patterns can bias comparisons between PMC configurations and must be considered for robust evaluations. A framework for evaluating intrinsic motion patterns in PMC is presented. © 2015 Wiley Periodicals, Inc.

Conceptual Modeling Framework for E-Area PA HELP Infiltration Model Simulations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dyer, J. A.

A conceptual modeling framework based on the proposed E-Area Low-Level Waste Facility (LLWF) closure cap design is presented for conducting Hydrologic Evaluation of Landfill Performance (HELP) model simulations of intact and subsided cap infiltration scenarios for the next E-Area Performance Assessment (PA).

A Systems Framework for Assessing Plumbing Products-Related Water Conservation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Williams, Alison; Dunham Whitehead, Camilla; Lutz, James

2011-12-02

Reducing the water use of plumbing products—toilets, urinals, faucets, and showerheads —has been a popular conservation measure. Improved technologies have created opportunities for additional conservation in this area. However, plumbing products do not operate in a vacuum. This paper reviews the literature related to plumbing products to determine a systems framework for evaluating future conservation measures using these products. The main framework comprises the following categories: water use efficiency, product components, product performance, source water, energy, and plumbing/sewer infrastructure. This framework for analysis provides a starting point for professionals considering future water conservation measures to evaluate the need for additionalmore » research, collaboration with other standards or codes committees, and attachment of additional metrics to water use efficiency (such as performance).« less

Evaluating Community Engagement in an Academic Medical Center

PubMed Central

Shone, Laura P.; Dozier, Ann M.; Newton, Gail L.; Green, Theresa; Bennett, Nancy M.

2014-01-01

From the perspective of academic medical centers (AMCs), community engagement is a collaborative process of working toward mutually defined goals to improve the community’s health, and involves partnerships between AMCs, individuals, and entities representing the surrounding community. AMCs increasingly recognize the importance of community engagement, and recent programs such as Prevention Research Centers and Clinical and Translational Science Awards have highlighted community engagement activities. However, there is no standard or accepted metric for evaluating AMCs’ performance and impact of community engagement activities. In this article, the authors present a framework for evaluating AMCs’ community engagement activities. The framework includes broad goals and specific activities within each goal, wherein goals and activities are evaluated using a health services research framework consisting of structure, process, and outcome criteria. To illustrate how to use this community engagement evaluation framework, the authors present specific community engagement goals and activities of the University of Rochester Medical Center to (1) improve the health of the community served by the AMC; (2) increase the AMC’s capacity for community engagement; and (3) increase generalizable knowledge and practices in community engagement and public health. Using a structure-process-outcomes framework, a multidisciplinary team should regularly evaluate an AMC’s community engagement program with the purpose of measurably improving the performance of the AMC and the health of its surrounding community. PMID:24556768

Evaluating the Implementation of Performance Improvement Training: The E[superscript 3] Process for Success

ERIC Educational Resources Information Center

Larbi-Apau, Josephine A.; Moseley, James L.

2008-01-01

This article provides a comprehensive approach to careful review and evaluation of the implementation of performance training intervention. It discusses the E[superscript 3] process for success, a basic framework for evaluating the implementation phase of a training program implemented as a broad-based performance improvement strategy. The intent…

Evaluating the substantive effectiveness of SEA: Towards a better understanding

DOE Office of Scientific and Technical Information (OSTI.GOV)

Doren, D. van; Driessen, P.P.J., E-mail: p.driessen@uu.nl; Schijf, B.

Evaluating the substantive effectiveness of strategic environmental assessment (SEA) is vital in order to know to what extent the tool fulfills its purposes and produces expected results. However, the studies that have evaluated the substantive effectiveness of SEA produce varying outcomes as regards the tool's contribution to decision-making and have used a variety of approaches to appraise its effectiveness. The aim of this article is to discuss the theoretical concept of SEA substantive effectiveness and to present a new approach that can be applied for evaluation studies. The SEA effectiveness evaluation framework that will be presented is composed of conceptsmore » of, and approaches to, SEA effectiveness derived from SEA literature and planning theory. Lessons for evaluation can be learned from planning theory in particular, given its long history of analyzing and understanding how sources of information and decisions affect (subsequent) decision-making. Key concepts of this new approach are 'conformance' and 'performance'. In addition, this article presents a systematic overview of process and context factors that can explain SEA effectiveness, derived from SEA literature. To illustrate the practical value of our framework for the assessment and understanding of substantive effectiveness of SEA, three Dutch SEA case studies are examined. The case studies have confirmed the usefulness of the SEA effectiveness assessment framework. The framework proved helpful in order to describe the cumulative influence of the three SEAs on decision-making and the ultimate plan. - Highlights: Black-Right-Pointing-Pointer A new framework to evaluate the substantive effectiveness of SEA is presented. Black-Right-Pointing-Pointer The framework is based on two key concepts: 'conformance' and 'performance.' Black-Right-Pointing-Pointer The practical applicability of the framework is demonstrated by three Dutch cases. Black-Right-Pointing-Pointer The framework allows for a more systematic understanding of SEA effectiveness. Black-Right-Pointing-Pointer Finally, this paper presents explanations for SEA effectiveness.« less

A Framework for the Development of Automatic DFA Method to Minimize the Number of Components and Assembly Reorientations

NASA Astrophysics Data System (ADS)

Alfadhlani; Samadhi, T. M. A. Ari; Ma’ruf, Anas; Setiasyah Toha, Isa

2018-03-01

Assembly is a part of manufacturing processes that must be considered at the product design stage. Design for Assembly (DFA) is a method to evaluate product design in order to make it simpler, easier and quicker to assemble, so that assembly cost is reduced. This article discusses a framework for developing a computer-based DFA method. The method is expected to aid product designer to extract data, evaluate assembly process, and provide recommendation for the product design improvement. These three things are desirable to be performed without interactive process or user intervention, so product design evaluation process could be done automatically. Input for the proposed framework is a 3D solid engineering drawing. Product design evaluation is performed by: minimizing the number of components; generating assembly sequence alternatives; selecting the best assembly sequence based on the minimum number of assembly reorientations; and providing suggestion for design improvement.

Building capacity for water, sanitation, and hygiene programming: Training evaluation theory applied to CLTS management training in Kenya.

PubMed

Crocker, Jonny; Shields, Katherine F; Venkataramanan, Vidya; Saywell, Darren; Bartram, Jamie

2016-10-01

Training and capacity building are long established critical components of global water, sanitation, and hygiene (WaSH) policies, strategies, and programs. Expanding capacity building support for WaSH in developing countries is one of the targets of the Sustainable Development Goals. There are many training evaluation methods and tools available. However, training evaluations in WaSH have been infrequent, have often not utilized these methods and tools, and have lacked rigor. We developed a conceptual framework for evaluating training in WaSH by reviewing and adapting concepts from literature. Our framework includes three target outcomes: learning, individual performance, and improved programming; and two sets of influences: trainee and context factors. We applied the framework to evaluate a seven-month community-led total sanitation (CLTS) management training program delivered to 42 government officials in Kenya from September 2013 to May 2014. Trainees were given a pre-training questionnaire and were interviewed at two weeks and seven months after initial training. We qualitatively analyzed the data using our conceptual framework. The training program resulted in trainees learning the CLTS process and new skills, and improving their individual performance through application of advocacy, partnership, and supervision soft skills. The link from trainees' performance to improved programming was constrained by resource limitations and pre-existing rigidity of trainees' organizations. Training-over-time enhanced outcomes and enabled trainees to overcome constraints in their work. Training in soft skills is relevant to managing public health programs beyond WaSH. We make recommendations on how training programs can be targeted and adapted to improve outcomes. Our conceptual framework can be used as a tool both for planning and evaluating training programs in WaSH. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Efficient evaluation of wireless real-time control networks.

PubMed

Horvath, Peter; Yampolskiy, Mark; Koutsoukos, Xenofon

2015-02-11

In this paper, we present a system simulation framework for the design and performance evaluation of complex wireless cyber-physical systems. We describe the simulator architecture and the specific developments that are required to simulate cyber-physical systems relying on multi-channel, multihop mesh networks. We introduce realistic and efficient physical layer models and a system simulation methodology, which provides statistically significant performance evaluation results with low computational complexity. The capabilities of the proposed framework are illustrated in the example of WirelessHART, a centralized, real-time, multi-hop mesh network designed for industrial control and monitor applications.

Judgment under uncertainty; a probabilistic evaluation framework for decision-making about sanitation systems in low-income countries.

PubMed

Malekpour, Shirin; Langeveld, Jeroen; Letema, Sammy; Clemens, François; van Lier, Jules B

2013-03-30

This paper introduces the probabilistic evaluation framework, to enable transparent and objective decision-making in technology selection for sanitation solutions in low-income countries. The probabilistic framework recognizes the often poor quality of the available data for evaluations. Within this framework, the evaluations will be done based on the probabilities that the expected outcomes occur in practice, considering the uncertainties in evaluation parameters. Consequently, the outcome of evaluations will not be single point estimates; but there exists a range of possible outcomes. A first trial application of this framework for evaluation of sanitation options in the Nyalenda settlement in Kisumu, Kenya, showed how the range of values that an evaluation parameter may obtain in practice would influence the evaluation outcomes. In addition, as the probabilistic evaluation requires various site-specific data, sensitivity analysis was performed to determine the influence of each data set quality on the evaluation outcomes. Based on that, data collection activities could be (re)directed, in a trade-off between the required investments in those activities and the resolution of the decisions that are to be made. Copyright © 2013 Elsevier Ltd. All rights reserved.

Sensor fusion display evaluation using information integration models in enhanced/synthetic vision applications

NASA Technical Reports Server (NTRS)

Foyle, David C.

1993-01-01

Based on existing integration models in the psychological literature, an evaluation framework is developed to assess sensor fusion displays as might be implemented in an enhanced/synthetic vision system. The proposed evaluation framework for evaluating the operator's ability to use such systems is a normative approach: The pilot's performance with the sensor fusion image is compared to models' predictions based on the pilot's performance when viewing the original component sensor images prior to fusion. This allows for the determination as to when a sensor fusion system leads to: poorer performance than one of the original sensor displays, clearly an undesirable system in which the fused sensor system causes some distortion or interference; better performance than with either single sensor system alone, but at a sub-optimal level compared to model predictions; optimal performance compared to model predictions; or, super-optimal performance, which may occur if the operator were able to use some highly diagnostic 'emergent features' in the sensor fusion display, which were unavailable in the original sensor displays.

Evaluating building performance in healthcare facilities: an organizational perspective.

PubMed

Steinke, Claudia; Webster, Lynn; Fontaine, Marie

2010-01-01

Using the environment as a strategic tool is one of the most cost-effective and enduring approaches for improving public health; however, it is one that requires multiple perspectives. The purpose of this article is to highlight an innovative methodology that has been developed for conducting comprehensive performance evaluations in public sector health facilities in Canada. The building performance evaluation methodology described in this paper is a government initiative. The project team developed a comprehensive building evaluation process for all new capital health projects that would respond to the aforementioned need for stakeholders to be more accountable and to better integrate the larger organizational strategy of facilities. The Balanced Scorecard, which is a multiparadigmatic, performance-based business framework, serves as the underlying theoretical framework for this initiative. It was applied in the development of the conceptual model entitled the Building Performance Evaluation Scorecard, which provides the following benefits: (1) It illustrates a process to link facilities more effectively to the overall mission and goals of an organization; (2) It is both a measurement and a management system that has the ability to link regional facilities to measures of success and larger business goals; (3) It provides a standardized methodology that ensures consistency in assessing building performance; and (4) It is more comprehensive than traditional building evaluations. The methodology presented in this paper is both a measurement and management system that integrates the principles of evidence-based design with the practices of pre- and post-occupancy evaluation. It promotes accountability and continues throughout the life cycle of a project. The advantage of applying this framework is that it engages health organizations in clarifying a vision and strategy for their facilities and helps translate those strategies into action and measurable performance outcomes.

Evaluation of a performance appraisal framework for radiation therapists in planning and simulation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Becker, Jillian, E-mail: jillian.becker@health.qld.gov.au; Bridge, Pete; Brown, Elizabeth

2015-06-15

Constantly evolving technology and techniques within radiation therapy require practitioners to maintain a continuous approach to professional development and training. Systems of performance appraisal and adoption of regular feedback mechanisms are vital to support this development yet frequently lack structure and rely on informal peer support. A Radiation Therapy Performance Appraisal Framework (RT-PAF) for radiation therapists in planning and simulation was developed to define expectations of practice and promote a supportive and objective culture of performance and skills appraisal. Evaluation of the framework was conducted via an anonymous online survey tool. Nine peer reviewers and fourteen recipients provided feedback onmore » its effectiveness and the challenges and limitations of the approach. Findings from the evaluation were positive and suggested that both groups gained benefit from and expressed a strong interest in embedding the approach more routinely. Respondents identified common challenges related to the limited ability to implement suggested development strategies; this was strongly associated with time and rostering issues. This framework successfully defined expectations for practice and provided a fair and objective feedback process that focussed on skills development. It empowered staff to maintain their skills and reach their professional potential. Management support, particularly in regard to provision of protected time was highlighted as critical to the framework's ongoing success. The demonstrated benefits arising in terms of staff satisfaction and development highlight the importance of this commitment to the modern radiation therapy workforce.« less

A Conceptual Framework for Evaluating Higher Education Institutions

ERIC Educational Resources Information Center

Chinta, Ravi; Kebritchi, Mansureh; Ellias, Janelle

2016-01-01

Purpose: Performance evaluation is a topic that has been researched and practiced extensively in business organizations but has received scant attention in higher education institutions. A review of literature revealed that context, input, process, product (CIPP) model is an appropriate performance evaluation model for higher education…

Evaluation of hazard and integrity monitor functions for integrated alerting and notification using a sensor simulation framework

NASA Astrophysics Data System (ADS)

Bezawada, Rajesh; Uijt de Haag, Maarten

2010-04-01

This paper discusses the results of an initial evaluation study of hazard and integrity monitor functions for use with integrated alerting and notification. The Hazard and Integrity Monitor (HIM) (i) allocates information sources within the Integrated Intelligent Flight Deck (IIFD) to required functionality (like conflict detection and avoidance) and determines required performance of these information sources as part of that function; (ii) monitors or evaluates the required performance of the individual information sources and performs consistency checks among various information sources; (iii) integrates the information to establish tracks of potential hazards that can be used for the conflict probes or conflict prediction for various time horizons including the 10, 5, 3, and <3 minutes used in our scenario; (iv) detects and assesses the class of the hazard and provide possible resolutions. The HIM monitors the operation-dependent performance parameters related to the potential hazards in a manner similar to the Required Navigation Performance (RNP). Various HIM concepts have been implemented and evaluated using a previously developed sensor simulator/synthesizer. Within the simulation framework, various inputs to the IIFD and its subsystems are simulated, synthesized from actual collected data, or played back from actual flight test sensor data. The framework and HIM functions are implemented in SimulinkR, a modeling language developed by The MathworksTM. This modeling language allows for test and evaluation of various sensor and communication link configurations as well as the inclusion of feedback from the pilot on the performance of the aircraft.

Evaluating the effectiveness of risk minimisation measures: the application of a conceptual framework to Danish real-world dabigatran data.

PubMed

Nyeland, Martin Erik; Laursen, Mona Vestergaard; Callréus, Torbjörn

2017-06-01

For both marketing authorization holders and regulatory authorities, evaluating the effectiveness of risk minimization measures is now an integral part of pharmacovigilance in the European Union. The overall aim of activities in this area is to assess the performance of risk minimization measures implemented in order to ensure a positive benefit-risk balance in patients treated with a medicinal product. Following a review of the relevant literature, we developed a conceptual framework consisting of four domains (data, knowledge, behaviour and outcomes) intended for the evaluation of risk minimization measures put into practice in the Danish health-care system. For the implementation of the framework, four classes of monitoring variables can be named and defined: patient descriptors, performance-related indicators of knowledge, behaviour and outcomes. We reviewed the features of the framework when applied to historical, real-world data following the introduction of dabigatran in Denmark for the prophylactic treatment of patients with non-valvular atrial fibrillation. The application of the framework provided useful graphical displays and an opportunity for a statistical evaluation (interrupted time series analysis) of a regulatory intervention. © 2017 Commonwealth of Australia. Pharmacoepidemiology & Drug Safety © 2017 John Wiley & Sons, Ltd. © 2017 Commonwealth of Australia. Pharmacoepidemiology & Drug Safety © 2017 John Wiley & Sons, Ltd.

A framework for evaluating and utilizing medical terminology mappings.

PubMed

Hussain, Sajjad; Sun, Hong; Sinaci, Anil; Erturkmen, Gokce Banu Laleci; Mead, Charles; Gray, Alasdair J G; McGuinness, Deborah L; Prud'Hommeaux, Eric; Daniel, Christel; Forsberg, Kerstin

2014-01-01

Use of medical terminologies and mappings across them are considered to be crucial pre-requisites for achieving interoperable eHealth applications. Built upon the outcomes of several research projects, we introduce a framework for evaluating and utilizing terminology mappings that offers a platform for i) performing various mappings strategies, ii) representing terminology mappings together with their provenance information, and iii) enabling terminology reasoning for inferring both new and erroneous mappings. We present the results of the introduced framework from SALUS project where we evaluated the quality of both existing and inferred terminology mappings among standard terminologies.

A Concise and Practical Framework for the Development and Usability Evaluation of Patient Information Websites.

PubMed

Peute, L W; Knijnenburg, S L; Kremer, L C; Jaspers, M W M

2015-01-01

The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS). To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population. The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA. The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content. Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations.

Towards a framework for selection of supervisory control for commercial buildings: HVAC system energy efficiency

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ramachandran, Thiagarajan; Kundu, Soumya; Chen, Yan

This paper develops and utilizes an optimization based framework to investigate the maximal energy efficiency potentially attainable by HVAC system operation in a non-predictive context. Performance is evaluated relative to the existing state of the art set point reset strategies. The expected efficiency increase driven by operation constraints relaxations is evaluated.

Towards a framework for selection of supervisory control for commercial buildings: HVAC system energy efficiency

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ramachandran, Thiagarajan; Kundu, Soumya; Chen, Yan

This paper develops and utilizes an optimization based framework to investigate the maximal energy efficiency potentially attainable by HVAC system operation in a non-predictive context. Performance is evaluated relative to the existing state of the art set-point reset strategies. The expected efficiency increase driven by operation constraints relaxations is evaluated.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Zhang, Yao; Balaprakash, Prasanna; Meng, Jiayuan

We present Raexplore, a performance modeling framework for architecture exploration. Raexplore enables rapid, automated, and systematic search of architecture design space by combining hardware counter-based performance characterization and analytical performance modeling. We demonstrate Raexplore for two recent manycore processors IBM Blue- Gene/Q compute chip and Intel Xeon Phi, targeting a set of scientific applications. Our framework is able to capture complex interactions between architectural components including instruction pipeline, cache, and memory, and to achieve a 3–22% error for same-architecture and cross-architecture performance predictions. Furthermore, we apply our framework to assess the two processors, and discover and evaluate a list ofmore » architectural scaling options for future processor designs.« less

Framework for Teacher Evaluation: Examining the Relationship between Teacher Performance and Student Achievement

ERIC Educational Resources Information Center

Parker, Neelie B.

2017-01-01

This action research study examined the relationship between teacher performance and student achievement in reading language arts and mathematics. The study sought to determine if teacher evaluation methods used to determine teacher performance, had a relationship in improving student achievement. The researcher investigated the topic using…

Non-technical skills evaluation in the critical care air ambulance environment: introduction of an adapted rating instrument--an observational study.

PubMed

Myers, Julia A; Powell, David M C; Psirides, Alex; Hathaway, Karyn; Aldington, Sarah; Haney, Michael F

2016-03-08

In the isolated and dynamic health-care setting of critical care air ambulance transport, the quality of clinical care is strongly influenced by non-technical skills such as anticipating, recognising and understanding, decision making, and teamwork. However there are no published reports identifying or applying a non-technical skills framework specific to an intensive care air ambulance setting. The objective of this study was to adapt and evaluate a non-technical skills rating framework for the air ambulance clinical environment. In the first phase of the project the anaesthetists' non-technical skills (ANTS) framework was adapted to the air ambulance setting, using data collected directly from clinician groups, published literature, and field observation. In the second phase experienced and inexperienced inter-hospital transport clinicians completed a simulated critical care air transport scenario, and their non-technical skills performance was independently rated by two blinded assessors. Observed and self-rated general clinical performance ratings were also collected. Rank-based statistical tests were used to examine differences in the performance of experienced and inexperienced clinicians, and relationships between different assessment approaches and assessors. The framework developed during phase one was referred to as an aeromedical non-technical skills framework, or AeroNOTS. During phase two 16 physicians from speciality training programmes in intensive care, emergency medicine and anaesthesia took part in the clinical simulation study. Clinicians with inter-hospital transport experience performed more highly than those without experience, according to both AeroNOTS non-technical skills ratings (p = 0.001) and general performance ratings (p = 0.003). Self-ratings did not distinguish experienced from inexperienced transport clinicians (p = 0.32) and were not strongly associated with either observed general performance (r(s) = 0.4, p = 0.11) or observed non-technical skills performance (r(s) = 0.4, p = 0.1). This study describes a framework which characterises the non-technical skills required by critical care air ambulance clinicians, and distinguishes higher and lower levels of performance. The AeroNOTS framework could be used to facilitate education and training in non-technical skills for air ambulance clinicians, and further evaluation of this rating system is merited.

Estimation for aerial detection effectiveness with cooperation efficiency factors of early-warning aircraft in early-warning detection SoS under BSC framework

NASA Astrophysics Data System (ADS)

Zhu, Feng; Hu, Xiaofeng; He, Xiaoyuan; Guo, Rui; Li, Kaiming; Yang, Lu

2017-11-01

In the military field, the performance evaluation of early-warning aircraft deployment or construction is always an important problem needing to be explored. As an effective approach of enterprise management and performance evaluation, Balanced Score Card (BSC) attracts more and more attentions and is studied more and more widely all over the world. It can also bring feasible ideas and technical approaches for studying the issue of the performance evaluation of the deployment or construction of early-warning aircraft which is the important component in early-warning detection system of systems (SoS). Therefore, the deep explored researches are carried out based on the previously research works. On the basis of the characteristics of space exploration and aerial detection effectiveness of early-warning detection SoS and the cardinal principle of BSC are analyzed simply, and the performance evaluation framework of the deployment or construction of early-warning aircraft is given, under this framework, aimed at the evaluation issue of aerial detection effectiveness of early-warning detection SoS with the cooperation efficiency factors of the early-warning aircraft and other land based radars, the evaluation indexes are further designed and the relative evaluation model is further established, especially the evaluation radar chart being also drawn to obtain the evaluation results from a direct sight angle. Finally, some practical computer simulations are launched to prove the validity and feasibility of the research thinking and technologic approaches which are proposed in the paper.

An overview of ethical frameworks in public health: can they be supportive in the evaluation of programs to prevent overweight?

PubMed Central

2010-01-01

Background The prevention of overweight sometimes raises complex ethical questions. Ethical public health frameworks may be helpful in evaluating programs or policy for overweight prevention. We give an overview of the purpose, form and contents of such public health frameworks and investigate to which extent they are useful for evaluating programs to prevent overweight and/or obesity. Methods Our search for frameworks consisted of three steps. Firstly, we asked experts in the field of ethics and public health for the frameworks they were aware of. Secondly, we performed a search in Pubmed. Thirdly, we checked literature references in the articles on frameworks we found. In total, we thus found six ethical frameworks. We assessed the area on which the available ethical frameworks focus, the users they target at, the type of policy or intervention they propose to address, and their aim. Further, we looked at their structure and content, that is, tools for guiding the analytic process, the main ethical principles or values, possible criteria for dealing with ethical conflicts, and the concrete policy issues they are applied to. Results All frameworks aim to support public health professionals or policymakers. Most of them provide a set of values or principles that serve as a standard for evaluating policy. Most frameworks articulate both the positive ethical foundations for public health and ethical constraints or concerns. Some frameworks offer analytic tools for guiding the evaluative process. Procedural guidelines and concrete criteria for solving important ethical conflicts in the particular area of the prevention of overweight or obesity are mostly lacking. Conclusions Public health ethical frameworks may be supportive in the evaluation of overweight prevention programs or policy, but seem to lack practical guidance to address ethical conflicts in this particular area. PMID:20969761

Systematic evaluation of deep learning based detection frameworks for aerial imagery

NASA Astrophysics Data System (ADS)

Sommer, Lars; Steinmann, Lucas; Schumann, Arne; Beyerer, Jürgen

2018-04-01

Object detection in aerial imagery is crucial for many applications in the civil and military domain. In recent years, deep learning based object detection frameworks significantly outperformed conventional approaches based on hand-crafted features on several datasets. However, these detection frameworks are generally designed and optimized for common benchmark datasets, which considerably differ from aerial imagery especially in object sizes. As already demonstrated for Faster R-CNN, several adaptations are necessary to account for these differences. In this work, we adapt several state-of-the-art detection frameworks including Faster R-CNN, R-FCN, and Single Shot MultiBox Detector (SSD) to aerial imagery. We discuss adaptations that mainly improve the detection accuracy of all frameworks in detail. As the output of deeper convolutional layers comprise more semantic information, these layers are generally used in detection frameworks as feature map to locate and classify objects. However, the resolution of these feature maps is insufficient for handling small object instances, which results in an inaccurate localization or incorrect classification of small objects. Furthermore, state-of-the-art detection frameworks perform bounding box regression to predict the exact object location. Therefore, so called anchor or default boxes are used as reference. We demonstrate how an appropriate choice of anchor box sizes can considerably improve detection performance. Furthermore, we evaluate the impact of the performed adaptations on two publicly available datasets to account for various ground sampling distances or differing backgrounds. The presented adaptations can be used as guideline for further datasets or detection frameworks.

Performance Assessment of Communicable Disease Surveillance in Disasters: A Systematic Review

PubMed Central

Babaie, Javad; Ardalan, Ali; Vatandoost, Hasan; Goya, Mohammad Mehdi; Akbarisari, Ali

2015-01-01

Background: This study aimed to identify the indices and frameworks that have been used to assess the performance of communicable disease surveillance (CDS) in response to disasters and other emergencies, including infectious disease outbreaks. Method: In this systematic review, PubMed, Google Scholar, Scopus, ScienceDirect, ProQuest databases and grey literature were searched until the end of 2013. All retrieved titles were examined in accordance with inclusion criteria. Abstracts of the relevant titles were reviewed and eligible abstracts were included in a list for data abstraction. Finally, the study variables were extracted. Results: Sixteen articles and one book were found relevant to our study objectives. In these articles, 31 criteria and 35 indicators were used or suggested for the assessment/evaluation of the performance of surveillance systems in disasters. The Centers for Disease Control (CDC) updated guidelines for the evaluation of public health surveillance systems were the most widely used. Conclusion: Despite the importance of performance assessment in improving CDS in response to disasters, there is a lack of clear and accepted frameworks. There is also no agreement on the use of existing criteria and indices. The only relevant framework is the CDC guideline, which is a common framework for assessing public health surveillance systems as a whole. There is an urgent need to develop appropriate frameworks, criteria, and indices for specifically assessing the performance of CDS in response to disasters and other emergencies, including infectious diseases outbreaks. Key words: Disasters, Emergencies, Communicable Diseases, Surveillance System, Performance Assessment PMID:25774323

THE ATMOSPHERIC MODEL EVALUATION TOOL

EPA Science Inventory

This poster describes a model evaluation tool that is currently being developed and applied for meteorological and air quality model evaluation. The poster outlines the framework and provides examples of statistical evaluations that can be performed with the model evaluation tool...

Evaluating large-scale propensity score performance through real-world and synthetic data experiments.

PubMed

Tian, Yuxi; Schuemie, Martijn J; Suchard, Marc A

2018-06-22

Propensity score adjustment is a popular approach for confounding control in observational studies. Reliable frameworks are needed to determine relative propensity score performance in large-scale studies, and to establish optimal propensity score model selection methods. We detail a propensity score evaluation framework that includes synthetic and real-world data experiments. Our synthetic experimental design extends the 'plasmode' framework and simulates survival data under known effect sizes, and our real-world experiments use a set of negative control outcomes with presumed null effect sizes. In reproductions of two published cohort studies, we compare two propensity score estimation methods that contrast in their model selection approach: L1-regularized regression that conducts a penalized likelihood regression, and the 'high-dimensional propensity score' (hdPS) that employs a univariate covariate screen. We evaluate methods on a range of outcome-dependent and outcome-independent metrics. L1-regularization propensity score methods achieve superior model fit, covariate balance and negative control bias reduction compared with the hdPS. Simulation results are mixed and fluctuate with simulation parameters, revealing a limitation of simulation under the proportional hazards framework. Including regularization with the hdPS reduces commonly reported non-convergence issues but has little effect on propensity score performance. L1-regularization incorporates all covariates simultaneously into the propensity score model and offers propensity score performance superior to the hdPS marginal screen.

Fuzzy logic based sensor performance evaluation of vehicle mounted metal detector systems

NASA Astrophysics Data System (ADS)

Abeynayake, Canicious; Tran, Minh D.

2015-05-01

Vehicle Mounted Metal Detector (VMMD) systems are widely used for detection of threat objects in humanitarian demining and military route clearance scenarios. Due to the diverse nature of such operational conditions, operational use of VMMD without a proper understanding of its capability boundaries may lead to heavy causalities. Multi-criteria fitness evaluations are crucial for determining capability boundaries of any sensor-based demining equipment. Evaluation of sensor based military equipment is a multi-disciplinary topic combining the efforts of researchers, operators, managers and commanders having different professional backgrounds and knowledge profiles. Information acquired through field tests usually involves uncertainty, vagueness and imprecision due to variations in test and evaluation conditions during a single test or series of tests. This report presents a fuzzy logic based methodology for experimental data analysis and performance evaluation of VMMD. This data evaluation methodology has been developed to evaluate sensor performance by consolidating expert knowledge with experimental data. A case study is presented by implementing the proposed data analysis framework in a VMMD evaluation scenario. The results of this analysis confirm accuracy, practicability and reliability of the fuzzy logic based sensor performance evaluation framework.

Performability modeling with continuous accomplishment sets

NASA Technical Reports Server (NTRS)

Meyer, J. F.

1979-01-01

A general modeling framework that permits the definition, formulation, and evaluation of performability is described. It is shown that performability relates directly to system effectiveness, and is a proper generalization of both performance and reliability. A hierarchical modeling scheme is used to formulate the capability function used to evaluate performability. The case in which performance variables take values in a continuous accomplishment set is treated explicitly.

Transfer Learning of Classification Rules for Biomarker Discovery and Verification from Molecular Profiling Studies

PubMed Central

Ganchev, Philip; Malehorn, David; Bigbee, William L.; Gopalakrishnan, Vanathi

2013-01-01

We present a novel framework for integrative biomarker discovery from related but separate data sets created in biomarker profiling studies. The framework takes prior knowledge in the form of interpretable, modular rules, and uses them during the learning of rules on a new data set. The framework consists of two methods of transfer of knowledge from source to target data: transfer of whole rules and transfer of rule structures. We evaluated the methods on three pairs of data sets: one genomic and two proteomic. We used standard measures of classification performance and three novel measures of amount of transfer. Preliminary evaluation shows that whole-rule transfer improves classification performance over using the target data alone, especially when there is more source data than target data. It also improves performance over using the union of the data sets. PMID:21571094

An Experimental Design of a Foundational Framework for the Application of Affective Computing to Soaring Flight Simulation and Training

ERIC Educational Resources Information Center

Moon, Shannon

2017-01-01

In the absence of tools for intelligent tutoring systems for soaring flight simulation training, this study evaluated a framework foundation to measure pilot performance, affect, and physiological response to training in real-time. Volunteers were asked to perform a series of flight tasks selected from Federal Aviation Administration Practical…

Evaluating ecommerce websites cognitive efficiency: an integrative framework based on data envelopment analysis.

PubMed

Lo Storto, Corrado

2013-11-01

This paper presents an integrative framework to evaluate ecommerce website efficiency from the user viewpoint using Data Envelopment Analysis (DEA). This framework is inspired by concepts driven from theories of information processing and cognition and considers the website efficiency as a measure of its quality and performance. When the users interact with the website interfaces to perform a task, they are involved in a cognitive effort, sustaining a cognitive cost to search, interpret and process information, and experiencing either a sense of satisfaction or dissatisfaction for that. The amount of ambiguity and uncertainty, and the search (over-)time during navigation that they perceive determine the effort size - and, as a consequence, the cognitive cost amount - they have to bear to perform their task. On the contrary, task performing and result achievement provide the users with cognitive benefits, making interaction with the website potentially attractive, satisfying, and useful. In total, 9 variables are measured, classified in a set of 3 website macro-dimensions (user experience, site navigability and structure). The framework is implemented to compare 52 ecommerce websites that sell products in the information technology and media market. A stepwise regression is performed to assess the influence of cognitive costs and benefits that mostly affect website efficiency. Copyright © 2013 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Deductive Evaluation: Implicit Code Verification With Low User Burden

NASA Technical Reports Server (NTRS)

Di Vito, Ben L.

2016-01-01

We describe a framework for symbolically evaluating C code using a deductive approach that discovers and proves program properties. The framework applies Floyd-Hoare verification principles in its treatment of loops, with a library of iteration schemes serving to derive loop invariants. During evaluation, theorem proving is performed on-the-fly, obviating the generation of verification conditions normally needed to establish loop properties. A PVS-based prototype is presented along with results for sample C functions.

Measuring Up: Are You Looking for a Way to Assess Your Teaching Practice? These Three Frameworks Can Help You Evaluate Your Performance

ERIC Educational Resources Information Center

Carrero, Jacqueline

2015-01-01

The author discusses how teachers can measure their effectiveness and discusses three frameworks they can use to do so: the National Board for Professional Teaching Standards's Core Propositions, Robert J. Marzano's Teacher Evaluation Model, and John Hattie's eight mind frames. She also gives examples from her own experience to show how she…

OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing

NASA Astrophysics Data System (ADS)

Wei, Shoulin; Wang, Feng; Deng, Hui; Liu, Cuiyin; Dai, Wei; Liang, Bo; Mei, Ying; Shi, Congming; Liu, Yingbo; Wu, Jingping

2017-02-01

The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their limitations and programming difficulties. In this paper, we therefore present OpenCluster, an open-source distributed computing framework to support rapidly developing high-performance processing pipelines of astronomical big data. We first detail the OpenCluster design principles and implementations and present the APIs facilitated by the framework. We then demonstrate a case in which OpenCluster is used to resolve complex data processing problems for developing a pipeline for the Mingantu Ultrawide Spectral Radioheliograph. Finally, we present our OpenCluster performance evaluation. Overall, OpenCluster provides not only high fault tolerance and simple programming interfaces, but also a flexible means of scaling up the number of interacting entities. OpenCluster thereby provides an easily integrated distributed computing framework for quickly developing a high-performance data processing system of astronomical telescopes and for significantly reducing software development expenses.

Translation from UML to Markov Model: A Performance Modeling Framework

NASA Astrophysics Data System (ADS)

Khan, Razib Hayat; Heegaard, Poul E.

Performance engineering focuses on the quantitative investigation of the behavior of a system during the early phase of the system development life cycle. Bearing this on mind, we delineate a performance modeling framework of the application for communication system that proposes a translation process from high level UML notation to Continuous Time Markov Chain model (CTMC) and solves the model for relevant performance metrics. The framework utilizes UML collaborations, activity diagrams and deployment diagrams to be used for generating performance model for a communication system. The system dynamics will be captured by UML collaboration and activity diagram as reusable specification building blocks, while deployment diagram highlights the components of the system. The collaboration and activity show how reusable building blocks in the form of collaboration can compose together the service components through input and output pin by highlighting the behavior of the components and later a mapping between collaboration and system component identified by deployment diagram will be delineated. Moreover the UML models are annotated to associate performance related quality of service (QoS) information which is necessary for solving the performance model for relevant performance metrics through our proposed framework. The applicability of our proposed performance modeling framework in performance evaluation is delineated in the context of modeling a communication system.

Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation.

PubMed

Wootton, Richard; Vladzymyrskyy, Anton; Zolfo, Maria; Bonnardot, Laurent

2011-01-01

Telemedicine has been used for many years to support doctors in the developing world. Several networks provide services in different settings and in different ways. However, to draw conclusions about which telemedicine networks are successful requires a method of evaluating them. No general consensus or validated framework exists for this purpose. To define a basic method of performance measurement that can be used to improve and compare teleconsultation networks; to employ the proposed framework in an evaluation of three existing networks; to make recommendations about the future implementation and follow-up of such networks. Analysis based on the experience of three telemedicine networks (in operation for 7-10 years) that provide services to doctors in low-resource settings and which employ the same basic design. Although there are many possible indicators and metrics that might be relevant, five measures for each of the three user groups appear to be sufficient for the proposed framework. In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The proposed performance measurement framework was applied to three mature telemedicine networks. Despite their differences in terms of activity, size and objectives, their performance in certain respects is very similar. For example, the time to first reply from an expert is about 24 hours for each network. Although all three networks had systems in place to collect data from the user perspective, none of them collected information about the coordinator's time required or about ease of system usage. They had only limited information about quality and cost. Measuring the performance of a telemedicine network is essential in understanding whether the network is working as intended and what effect it is having. Based on long-term field experience, the suggested framework is a practical tool that will permit organisations to assess the performance of their own networks and to improve them by comparison with others. All telemedicine systems should provide information about setup and running costs because cost-effectiveness is crucial for sustainability.

Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation

PubMed Central

Wootton, Richard; Vladzymyrskyy, Anton; Zolfo, Maria; Bonnardot, Laurent

2011-01-01

Background Telemedicine has been used for many years to support doctors in the developing world. Several networks provide services in different settings and in different ways. However, to draw conclusions about which telemedicine networks are successful requires a method of evaluating them. No general consensus or validated framework exists for this purpose. Objective To define a basic method of performance measurement that can be used to improve and compare teleconsultation networks; to employ the proposed framework in an evaluation of three existing networks; to make recommendations about the future implementation and follow-up of such networks. Methods Analysis based on the experience of three telemedicine networks (in operation for 7–10 years) that provide services to doctors in low-resource settings and which employ the same basic design. Findings Although there are many possible indicators and metrics that might be relevant, five measures for each of the three user groups appear to be sufficient for the proposed framework. In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The proposed performance measurement framework was applied to three mature telemedicine networks. Despite their differences in terms of activity, size and objectives, their performance in certain respects is very similar. For example, the time to first reply from an expert is about 24 hours for each network. Although all three networks had systems in place to collect data from the user perspective, none of them collected information about the coordinator's time required or about ease of system usage. They had only limited information about quality and cost. Conclusion Measuring the performance of a telemedicine network is essential in understanding whether the network is working as intended and what effect it is having. Based on long-term field experience, the suggested framework is a practical tool that will permit organisations to assess the performance of their own networks and to improve them by comparison with others. All telemedicine systems should provide information about setup and running costs because cost-effectiveness is crucial for sustainability. PMID:22162965

How is genetic testing evaluated? A systematic review of the literature.

PubMed

Pitini, Erica; De Vito, Corrado; Marzuillo, Carolina; D'Andrea, Elvira; Rosso, Annalisa; Federici, Antonio; Di Maria, Emilio; Villari, Paolo

2018-05-01

Given the rapid development of genetic tests, an assessment of their benefits, risks, and limitations is crucial for public health practice. We performed a systematic review aimed at identifying and comparing the existing evaluation frameworks for genetic tests. We searched PUBMED, SCOPUS, ISI Web of Knowledge, Google Scholar, Google, and gray literature sources for any documents describing such frameworks. We identified 29 evaluation frameworks published between 2000 and 2017, mostly based on the ACCE Framework (n = 13 models), or on the HTA process (n = 6), or both (n = 2). Others refer to the Wilson and Jungner screening criteria (n = 3) or to a mixture of different criteria (n = 5). Due to the widespread use of the ACCE Framework, the most frequently used evaluation criteria are analytic and clinical validity, clinical utility and ethical, legal and social implications. Less attention is given to the context of implementation. An economic dimension is always considered, but not in great detail. Consideration of delivery models, organizational aspects, and consumer viewpoint is often lacking. A deeper analysis of such context-related evaluation dimensions may strengthen a comprehensive evaluation of genetic tests and support the decision-making process.

Evaluating Web-Based Learning and Instruction (WBLI): A Case Study and Framework for Evaluation.

ERIC Educational Resources Information Center

Michalski, Greg V.

The purpose of this paper is to suggest an alternative approach to perform relevant and useful evaluations of Web-based learning and instruction (WBLI) that will accommodate performance and keep pace with the growing capabilities of the Internet. Discussion includes the advantages of WBLI, multimedia and streaming use in WBLI, building the…

Teacher Perspectives of the Use of Student Performance Data in Teacher Evaluations

ERIC Educational Resources Information Center

Hopkins, Paul Thomas

2013-01-01

The purpose of this study was to determine how K-12 public school teachers perceive the use of student performance data in teacher evaluations. The proprietary, utility, feasibility, and accuracy standards created by the Joint Committee on Standards for Education Evaluation (JCSEE) served as a framework for the study. An online survey was deployed…

Fuzzy decision-making framework for treatment selection based on the combined QUALIFLEX-TODIM method

NASA Astrophysics Data System (ADS)

Ji, Pu; Zhang, Hong-yu; Wang, Jian-qiang

2017-10-01

Treatment selection is a multi-criteria decision-making problem of significant concern in the medical field. In this study, a fuzzy decision-making framework is established for treatment selection. The framework mitigates information loss by introducing single-valued trapezoidal neutrosophic numbers to denote evaluation information. Treatment selection has multiple criteria that remarkably exceed the alternatives. In consideration of this characteristic, the framework utilises the idea of the qualitative flexible multiple criteria method. Furthermore, it considers the risk-averse behaviour of a decision maker by employing a concordance index based on TODIM (an acronym in Portuguese of interactive and multi-criteria decision-making) method. A sensitivity analysis is performed to illustrate the robustness of the framework. Finally, a comparative analysis is conducted to compare the framework with several extant methods. Results indicate the advantages of the framework and its better performance compared with the extant methods.

A Systematic Approach for Quantitative Analysis of Multidisciplinary Design Optimization Framework

NASA Astrophysics Data System (ADS)

Kim, Sangho; Park, Jungkeun; Lee, Jeong-Oog; Lee, Jae-Woo

An efficient Multidisciplinary Design and Optimization (MDO) framework for an aerospace engineering system should use and integrate distributed resources such as various analysis codes, optimization codes, Computer Aided Design (CAD) tools, Data Base Management Systems (DBMS), etc. in a heterogeneous environment, and need to provide user-friendly graphical user interfaces. In this paper, we propose a systematic approach for determining a reference MDO framework and for evaluating MDO frameworks. The proposed approach incorporates two well-known methods, Analytic Hierarchy Process (AHP) and Quality Function Deployment (QFD), in order to provide a quantitative analysis of the qualitative criteria of MDO frameworks. Identification and hierarchy of the framework requirements and the corresponding solutions for the reference MDO frameworks, the general one and the aircraft oriented one were carefully investigated. The reference frameworks were also quantitatively identified using AHP and QFD. An assessment of three in-house frameworks was then performed. The results produced clear and useful guidelines for improvement of the in-house MDO frameworks and showed the feasibility of the proposed approach for evaluating an MDO framework without a human interference.

Evacuation performance evaluation tool.

PubMed

Farra, Sharon; Miller, Elaine T; Gneuhs, Matthew; Timm, Nathan; Li, Gengxin; Simon, Ashley; Brady, Whittney

2016-01-01

Hospitals conduct evacuation exercises to improve performance during emergency events. An essential aspect in this process is the creation of reliable and valid evaluation tools. The objective of this article is to describe the development and implications of a disaster evacuation performance tool that measures one portion of the very complex process of evacuation. Through the application of the Delphi technique and DeVellis's framework, disaster and neonatal experts provided input in developing this performance evaluation tool. Following development, content validity and reliability of this tool were assessed. Large pediatric hospital and medical center in the Midwest. The tool was pilot tested with an administrative, medical, and nursing leadership group and then implemented with a group of 68 healthcare workers during a disaster exercise of a neonatal intensive care unit (NICU). The tool has demonstrated high content validity with a scale validity index of 0.979 and inter-rater reliability G coefficient (0.984, 95% CI: 0.948-0.9952). The Delphi process based on the conceptual framework of DeVellis yielded a psychometrically sound evacuation performance evaluation tool for a NICU.

FSM-F: Finite State Machine Based Framework for Denial of Service and Intrusion Detection in MANET.

PubMed

N Ahmed, Malik; Abdullah, Abdul Hanan; Kaiwartya, Omprakash

2016-01-01

Due to the continuous advancements in wireless communication in terms of quality of communication and affordability of the technology, the application area of Mobile Adhoc Networks (MANETs) significantly growing particularly in military and disaster management. Considering the sensitivity of the application areas, security in terms of detection of Denial of Service (DoS) and intrusion has become prime concern in research and development in the area. The security systems suggested in the past has state recognition problem where the system is not able to accurately identify the actual state of the network nodes due to the absence of clear definition of states of the nodes. In this context, this paper proposes a framework based on Finite State Machine (FSM) for denial of service and intrusion detection in MANETs. In particular, an Interruption Detection system for Adhoc On-demand Distance Vector (ID-AODV) protocol is presented based on finite state machine. The packet dropping and sequence number attacks are closely investigated and detection systems for both types of attacks are designed. The major functional modules of ID-AODV includes network monitoring system, finite state machine and attack detection model. Simulations are carried out in network simulator NS-2 to evaluate the performance of the proposed framework. A comparative evaluation of the performance is also performed with the state-of-the-art techniques: RIDAN and AODV. The performance evaluations attest the benefits of proposed framework in terms of providing better security for denial of service and intrusion detection attacks.

A Concise and Practical Framework for the Development and Usability Evaluation of Patient Information Websites

PubMed Central

Knijnenburg, S.L.; Kremer, L.C.; Jaspers, M.W.M.

2015-01-01

Summary Background The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS). Objective To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population. Methods The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA. Results The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content. Conclusion Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations. PMID:26171083

Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation.

PubMed

Fleming, Stephen M; Daw, Nathaniel D

2017-01-01

People are often aware of their mistakes, and report levels of confidence in their choices that correlate with objective performance. These metacognitive assessments of decision quality are important for the guidance of behavior, particularly when external feedback is absent or sporadic. However, a computational framework that accounts for both confidence and error detection is lacking. In addition, accounts of dissociations between performance and metacognition have often relied on ad hoc assumptions, precluding a unified account of intact and impaired self-evaluation. Here we present a general Bayesian framework in which self-evaluation is cast as a "second-order" inference on a coupled but distinct decision system, computationally equivalent to inferring the performance of another actor. Second-order computation may ensue whenever there is a separation between internal states supporting decisions and confidence estimates over space and/or time. We contrast second-order computation against simpler first-order models in which the same internal state supports both decisions and confidence estimates. Through simulations we show that second-order computation provides a unified account of different types of self-evaluation often considered in separate literatures, such as confidence and error detection, and generates novel predictions about the contribution of one's own actions to metacognitive judgments. In addition, the model provides insight into why subjects' metacognition may sometimes be better or worse than task performance. We suggest that second-order computation may underpin self-evaluative judgments across a range of domains. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation

PubMed Central

2017-01-01

People are often aware of their mistakes, and report levels of confidence in their choices that correlate with objective performance. These metacognitive assessments of decision quality are important for the guidance of behavior, particularly when external feedback is absent or sporadic. However, a computational framework that accounts for both confidence and error detection is lacking. In addition, accounts of dissociations between performance and metacognition have often relied on ad hoc assumptions, precluding a unified account of intact and impaired self-evaluation. Here we present a general Bayesian framework in which self-evaluation is cast as a “second-order” inference on a coupled but distinct decision system, computationally equivalent to inferring the performance of another actor. Second-order computation may ensue whenever there is a separation between internal states supporting decisions and confidence estimates over space and/or time. We contrast second-order computation against simpler first-order models in which the same internal state supports both decisions and confidence estimates. Through simulations we show that second-order computation provides a unified account of different types of self-evaluation often considered in separate literatures, such as confidence and error detection, and generates novel predictions about the contribution of one’s own actions to metacognitive judgments. In addition, the model provides insight into why subjects’ metacognition may sometimes be better or worse than task performance. We suggest that second-order computation may underpin self-evaluative judgments across a range of domains. PMID:28004960

Clinically led performance management in secondary healthcare: evaluating the attitudes of medical and non-clinical managers.

PubMed

Trebble, Timothy M; Paul, Maureen; Hockey, Peter M; Heyworth, Nicola; Humphrey, Rachael; Powell, Timothy; Clarke, Nicholas

2015-03-01

Improving the quality and activity of clinicians' practice improves patient care. Performance-related human resource management (HRM) is an established approach to improving individual practice but with limited use among clinicians. A framework for performance-related HRM was developed from successful practice in non-healthcare organisations centred on distributive leadership and locally provided, validated and interpreted performance measurement. This study evaluated the response of medical and non-clinical managers to its implementation into a large secondary healthcare organisation. A semistructured qualitative questionnaire was developed from themes identified during framework implementation and included attitudes to previous approaches to measuring doctors' performance, and the structure and response to implementation of the performance-related HRM framework. Responses were analysed through a process of data summarising and categorising. A total of 29, from an invited cohort of 31, medical and non-clinical managers from departmental to executive level were interviewed. Three themes were identified: (1) previous systems of managing clinical performance were considered to be ineffective due to insufficient empowerment of medical managers and poor quality of available performance data; (2) the implemented framework was considered to address these needs and was positively received by medical and non-clinical managers; (3) introduction of performance-related HRM required the involvement of the whole organisation to executive level and inclusion within organisational strategy, structure and training. This study suggests that a performance-related HRM framework may facilitate the management of clinical performance in secondary healthcare, but is dependent on the design and methods of application used. Such approaches contrast with those currently proposed for clinicians in secondary healthcare in the UK and suggest that alternative strategies should be considered. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Teachers' Perceptions of Administrative Autonomy-Supportive versus Controlling Behaviors: Factors That Influence Teacher Intrinsic Motivation and Relationships with Administrators

ERIC Educational Resources Information Center

Scudella, Christine M.

2015-01-01

Federal policy around supervision and evaluation has prompted state action to develop new evaluation systems to increase student achievement. Systems, such as the Framework for Educator Effectiveness implemented in Wisconsin, are tied to teacher performance as well as student achievement. This type of performance-based evaluation system relies on…

Standardized mappings--a framework to combine different semantic mappers into a standardized web-API.

PubMed

Neuhaus, Philipp; Doods, Justin; Dugas, Martin

2015-01-01

Automatic coding of medical terms is an important, but highly complicated and laborious task. To compare and evaluate different strategies a framework with a standardized web-interface was created. Two UMLS mapping strategies are compared to demonstrate the interface. The framework is a Java Spring application running on a Tomcat application server. It accepts different parameters and returns results in JSON format. To demonstrate the framework, a list of medical data items was mapped by two different methods: similarity search in a large table of terminology codes versus search in a manually curated repository. These mappings were reviewed by a specialist. The evaluation shows that the framework is flexible (due to standardized interfaces like HTTP and JSON), performant and reliable. Accuracy of automatically assigned codes is limited (up to 40%). Combining different semantic mappers into a standardized Web-API is feasible. This framework can be easily enhanced due to its modular design.

Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda.

PubMed

Hotchkiss, David R; Aqil, Anwer; Lippeveld, Theo; Mukooyo, Edward

2010-07-03

Sound policy, resource allocation and day-to-day management decisions in the health sector require timely information from routine health information systems (RHIS). In most low- and middle-income countries, the RHIS is viewed as being inadequate in providing quality data and continuous information that can be used to help improve health system performance. In addition, there is limited evidence on the effectiveness of RHIS strengthening interventions in improving data quality and use. The purpose of this study is to evaluate the usefulness of the newly developed Performance of Routine Information System Management (PRISM) framework, which consists of a conceptual framework and associated data collection and analysis tools to assess, design, strengthen and evaluate RHIS. The specific objectives of the study are: a) to assess the reliability and validity of the PRISM instruments and b) to assess the validity of the PRISM conceptual framework. Facility- and worker-level data were collected from 110 health care facilities in twelve districts in Uganda in 2004 and 2007 using records reviews, structured interviews and self-administered questionnaires. The analysis procedures include Cronbach's alpha to assess internal consistency of selected instruments, test-retest analysis to assess the reliability and sensitivity of the instruments, and bivariate and multivariate statistical techniques to assess validity of the PRISM instruments and conceptual framework. Cronbach's alpha analysis suggests high reliability (0.7 or greater) for the indices measuring a promotion of a culture of information, RHIS tasks self-efficacy and motivation. The study results also suggest that a promotion of a culture of information influences RHIS tasks self-efficacy, RHIS tasks competence and motivation, and that self-efficacy and the presence of RHIS staff have a direct influence on the use of RHIS information, a key aspect of RHIS performance. The study results provide some empirical support for the reliability and validity of the PRISM instruments and the validity of the PRISM conceptual framework, suggesting that the PRISM approach can be effectively used by RHIS policy makers and practitioners to assess the RHIS and evaluate RHIS strengthening interventions. However, additional studies with larger sample sizes are needed to further investigate the value of the PRISM instruments in exploring the linkages between RHIS data quality and use, and health systems performance.

Expert consensus on best evaluative practices in community-based rehabilitation.

PubMed

Grandisson, Marie; Thibeault, Rachel; Hébert, Michèle; Cameron, Debra

2016-01-01

The objective of this study was to generate expert consensus on best evaluative practices for community-based rehabilitation (CBR). This consensus includes key features of the evaluation process and methods, and discussion of whether a shared framework should be used to report findings and, if so, which framework should play this role. A Delphi study with two predefined rounds was conducted. Experts in CBR from a wide range of geographical areas and disciplinary backgrounds were recruited to complete the questionnaires. Both quantitative and qualitative analyses were performed to generate the recommendations for best practices in CBR evaluation. A panel of 42 experts reached consensus on 13 recommendations for best evaluative practices in CBR. In regard to the critical qualities of sound CBR evaluation processes, panellists emphasized that these processes should be inclusive, participatory, empowering and respectful of local cultures and languages. The group agreed that evaluators should consider the use of mixed methods and participatory tools, and should combine indicators from a universal list of CBR indicators with locally generated ones. The group also agreed that a common framework should guide CBR evaluations, and that this framework should be a flexible combination between the CBR Matrix and the CBR Principles. An expert panel reached consensus on key features of best evaluative practices in CBR. Knowledge transfer initiatives are now required to develop guidelines, tools and training opportunities to facilitate CBR program evaluations. CBR evaluation processes should strive to be inclusive, participatory, empowering and respectful of local cultures and languages. CBR evaluators should strongly consider using mixed methods, participatory tools, a combination of indicators generated with the local community and with others from a bank of CBR indicators. CBR evaluations should be situated within a shared, but flexible, framework. This shared framework could combine the CBR Matrix and the CBR Principles.

ATIS evaluation framework

DOT National Transportation Integrated Search

2005-04-01

This report documents the results of a series of ITS project evaluations performed for the Washington State Department of Transportation (WSDOT). Five Washington state advanced traveler information system (ATIS) projects selected for federal ITS fund...

NIST biometric evaluations and developments

NASA Astrophysics Data System (ADS)

Garris, Michael D.; Wilson, Charles L.

2005-05-01

This paper presents an R&D framework used by the National Institute of Standards and Technology (NIST) for biometric technology testing and evaluation. The focus of this paper is on fingerprint-based verification and identification. Since 9-11 the NIST Image Group has been mandated by Congress to run a program for biometric technology assessment and biometric systems certification. Four essential areas of activity are discussed: 1) developing test datasets, 2) conducting performance assessment; 3) technology development; and 4) standards participation. A description of activities and accomplishments are provided for each of these areas. In the process, methods of performance testing are described and results from specific biometric technology evaluations are presented. This framework is anticipated to have broad applicability to other technology and application domains.

SpaceCubeX: A Framework for Evaluating Hybrid Multi-Core CPU FPGA DSP Architectures

NASA Technical Reports Server (NTRS)

Schmidt, Andrew G.; Weisz, Gabriel; French, Matthew; Flatley, Thomas; Villalpando, Carlos Y.

2017-01-01

The SpaceCubeX project is motivated by the need for high performance, modular, and scalable on-board processing to help scientists answer critical 21st century questions about global climate change, air quality, ocean health, and ecosystem dynamics, while adding new capabilities such as low-latency data products for extreme event warnings. These goals translate into on-board processing throughput requirements that are on the order of 100-1,000 more than those of previous Earth Science missions for standard processing, compression, storage, and downlink operations. To study possible future architectures to achieve these performance requirements, the SpaceCubeX project provides an evolvable testbed and framework that enables a focused design space exploration of candidate hybrid CPU/FPGA/DSP processing architectures. The framework includes ArchGen, an architecture generator tool populated with candidate architecture components, performance models, and IP cores, that allows an end user to specify the type, number, and connectivity of a hybrid architecture. The framework requires minimal extensions to integrate new processors, such as the anticipated High Performance Spaceflight Computer (HPSC), reducing time to initiate benchmarking by months. To evaluate the framework, we leverage a wide suite of high performance embedded computing benchmarks and Earth science scenarios to ensure robust architecture characterization. We report on our projects Year 1 efforts and demonstrate the capabilities across four simulation testbed models, a baseline SpaceCube 2.0 system, a dual ARM A9 processor system, a hybrid quad ARM A53 and FPGA system, and a hybrid quad ARM A53 and DSP system.

The impact of contracting-out on health system performance: a conceptual framework.

PubMed

Liu, Xingzhu; Hotchkiss, David R; Bose, Sujata

2007-07-01

Despite the increased popularity of contracting-out of health services in developing countries, its effectiveness on overall health system performance is not yet conclusive. Except for substantial evidence of contracting-out's positive effect on access to health services and some evidence on improved equity in access, there is little evidence of contracting-out's impact on quality and efficiency. Most studies on the subject evaluate specific contracting-out projects against narrowly specified project objectives, not against more broadly defined health system goals. For this reason, conclusions of positive effects pertaining to project level may not hold at system level. This paper presents a conceptual framework that is expected to facilitate comprehensive, rigorous, and standardized evaluation of contracting-out at health system level. Specifically, this framework supports: full and standardized description of contracting-out interventions, study of the determinants of effectiveness, examination of provider and purchaser responses, assessment of the impact of contracting-out on all dimensions of health system performance, and cross-project analyses.

Sustainability performance evaluation: Literature review and future directions.

PubMed

Büyüközkan, Gülçin; Karabulut, Yağmur

2018-07-01

Current global economic activities are increasingly being perceived as unsustainable. Despite the high number of publications, sustainability science remains highly dispersed over diverse approaches and topics. This article aims to provide a structured overview of sustainability performance evaluation related publications and to document the current state of literature, categorize publications, analyze and link trends, as well as highlight gaps and provide research recommendations. 128 articles between 2007 and 2018 are identified. The results suggest that sustainability performance evaluation models shall be more balanced, suitable criteria and their interrelations shall be well defined and subjectivity of qualitative criteria inherent to sustainability indicators shall be considered. To address this subjectivity, group decision-making techniques and other analytical methods that can deal with uncertainty, conflicting indicators, and linguistic evaluations can be used in future works. By presenting research gaps, this review stimulates researchers to establish practically applicable sustainability performance evaluation frameworks to help assess and compare the degree of sustainability, leading to more sustainable business practices. The review is unique in defining corporate sustainability performance evaluation for the first time, exploring the gap between sustainability accounting and sustainability assessment, and coming up with a structured overview of innovative research recommendations about integrating analytical assessment methods into conceptual sustainability frameworks. Copyright © 2018 Elsevier Ltd. All rights reserved.

Developing a cultural context for conducting a neuropsychological evaluation with a culturally diverse client: the ECLECTIC framework.

PubMed

Fujii, Daryl E M

2018-02-20

With the increasing diversification of the American population, the discipline of neuropsychology is challenged to develop appropriate tools and conceptual models to meet its evolving client base. Thus far, the focus has been on developing appropriate tests and norms to obtain accurate testing data. By contrast, far less attention has been paid to the contextual impact of culture on an evaluation. This paper attempts to address this shortcoming. This manuscript introduces the ECLECTIC framework for conceptualizing different facets of culture pertinent for understanding a culturally diverse client when conducting a neuropsychological evaluation. Individual components of the framework (E: education and literacy; C: culture and acculturation; L: language; E: economics; C: communication; T: testing situation: comfort and motivation; I: intelligence conceptualization; and C: context of immigration) are introduced and potential biases to fairness in testing are described. In this manner, the framework specifies how individual facets of culture can impact neuropsychological test performance. Clinical implementation of the framework will be illustrated with a case sample. Strengths and weaknesses of the framework are discussed as well as recommendations for implementation.

FRAMEWORK FOR RESPONSIBLE ENVIRONMENTAL DECISION-MAKING (FRED): USING LIFE CYCLE ASSESSMENT TO EVALUATE PREFERABILITY OF PRODUCTS

EPA Science Inventory

Framework for Responsible Environmental Decision-Making (FRED) demonstrates how the life-cycle concept can be used to quantify competing products' environmental performance so that this information may be integrated with considerations of total ownership cost and technical perfor...

The Researching on Evaluation of Automatic Voltage Control Based on Improved Zoning Methodology

NASA Astrophysics Data System (ADS)

Xiao-jun, ZHU; Ang, FU; Guang-de, DONG; Rui-miao, WANG; De-fen, ZHU

2018-03-01

According to the present serious phenomenon of increasing size and structure of power system, hierarchically structured automatic voltage control(AVC) has been the researching spot. In the paper, the reduced control model is built and the adaptive reduced control model is researched to improve the voltage control effect. The theories of HCSD, HCVS, SKC and FCM are introduced and the effect on coordinated voltage regulation caused by different zoning methodologies is also researched. The generic framework for evaluating performance of coordinated voltage regulation is built. Finally, the IEEE-96 stsyem is used to divide the network. The 2383-bus Polish system is built to verify that the selection of a zoning methodology affects not only the coordinated voltage regulation operation, but also its robustness to erroneous data and proposes a comprehensive generic framework for evaluating its performance. The New England 39-bus network is used to verify the adaptive reduced control models’ performance.

An Integrated Framework for Parameter-based Optimization of Scientific Workflows.

PubMed

Kumar, Vijay S; Sadayappan, P; Mehta, Gaurang; Vahi, Karan; Deelman, Ewa; Ratnakar, Varun; Kim, Jihie; Gil, Yolanda; Hall, Mary; Kurc, Tahsin; Saltz, Joel

2009-01-01

Data analysis processes in scientific applications can be expressed as coarse-grain workflows of complex data processing operations with data flow dependencies between them. Performance optimization of these workflows can be viewed as a search for a set of optimal values in a multi-dimensional parameter space. While some performance parameters such as grouping of workflow components and their mapping to machines do not a ect the accuracy of the output, others may dictate trading the output quality of individual components (and of the whole workflow) for performance. This paper describes an integrated framework which is capable of supporting performance optimizations along multiple dimensions of the parameter space. Using two real-world applications in the spatial data analysis domain, we present an experimental evaluation of the proposed framework.

Evaluating the Effects of School Health Interventions on School Performance. Design Report.

ERIC Educational Resources Information Center

Devaney, Barbara; And Others

This evaluation design report presents a general framework for assessing the effects of school health interventions on students' school performance in order to guide efforts to develop strong empirical evidence. The report begins with an overview of eight general types of school health interventions: school health education, health services,…

An Agency Theory Perspective on Student Performance Evaluation

ERIC Educational Resources Information Center

Smith, Michael E.; Zsidisin, George A.; Adams, Laural L.

2005-01-01

The emphasis in recent research on the responsibility of college and university business instructors to prepare students for future employment underscores a need to refine the evaluation of student performance. In this article, an agency theory framework is used to understand the trade-offs that may be involved in the selection of various…

Topologically Guided, Automated Construction of Metal–Organic Frameworks and Their Evaluation for Energy-Related Applications

DOE Office of Scientific and Technical Information (OSTI.GOV)

Colón, Yamil J.; Gómez-Gualdrón, Diego A.; Snurr, Randall Q.

Metal-organic frameworks (MOFs) are promising materials for a range of energy and environmental applications. Here we describe in detail a computational algorithm and code to generate MOFs based on edge-transitive topological nets for subsequent evaluation via molecular simulation. This algorithm has been previously used by us to construct and evaluate 13 512 MOFs of 41 different topologies for cryo-adsorbed hydrogen storage. Grand canonical Monte Carlo simulations are used here to evaluate the 13 512 structures for the storage of gaseous fuels such as hydrogen and methane and nondistillative separation of xenon/krypton mixtures at various operating conditions. MOF performance for bothmore » gaseous fuel storage and xenon/krypton separation is influenced by topology. Simulation data suggest that gaseous fuel storage performance is topology-dependent due to MOF properties such as void fraction and surface area combining differently in different topologies, whereas xenon/krypton separation performance is topology-dependent due to how topology constrains the pore size distribution.« less

An experimental study on CHVE's performance evaluation.

PubMed

Paiva, Paulo V F; Machado, Liliane S; Oliveira, Jauvane C

2012-01-01

Virtual reality-based training simulators, with collaborative capabilities, are known to improve the way users interact with one another while learning or improving skills on a given medical procedure. Performance evaluation of Collaborative Haptic Virtual Environments (CHVE) allows us to understand how such systems can work in the Internet, as well as the requirements for multisensorial and real-time data. This work discloses new performance evaluation results for the collaborative module of the CyberMed VR framework.

Adapting Pipeline Architectures to Track Developing Aftershock Sequences and Recurrent Explosions

DTIC Science & Technology

2014-02-14

Sumatra earthquake was used to study the performance of subspace detectors to detect and classify events from within a very large (Area = ~250,000 km2... detectors to identify and organize repeating waveforms discovered in multichannel seismic data streams. The framework has been tested and evaluated on...a variety of different test cases from mining blasts in Central Asia to moderate and large earthquake aftershock sequences. The framework performs

A structured framework improves clinical patient assessment and nontechnical skills of early career emergency nurses: a pre-post study using full immersion simulation.

PubMed

Munroe, Belinda; Curtis, Kate; Murphy, Margaret; Strachan, Luke; Considine, Julie; Hardy, Jennifer; Wilson, Mark; Ruperto, Kate; Fethney, Judith; Buckley, Thomas

2016-08-01

The aim of this study was to evaluate the effect of the new evidence-informed nursing assessment framework HIRAID (History, Identify Red flags, Assessment, Interventions, Diagnostics, reassessment and communication) on the quality of patient assessment and fundamental nontechnical skills including communication, decision making, task management and situational awareness. Assessment is a core component of nursing practice and underpins clinical decisions and the safe delivery of patient care. Yet there is no universal or validated system used to teach emergency nurses how to comprehensively assess and care for patients. A pre-post design was used. The performance of thirty eight emergency nurses from five Australian hospitals was evaluated before and after undertaking education in the application of the HIRAID assessment framework. Video recordings of participant performance in immersive simulations of common presentations to the emergency department were evaluated, as well as participant documentation during the simulations. Paired parametric and nonparametric tests were used to compare changes from pre to postintervention. From pre to postintervention, participant performance increases were observed in the percentage of patient history elements collected, critical indicators of urgency collected and reported to medical officers, and patient reassessments performed. Participants also demonstrated improvement in each of the four nontechnical skills categories: communication, decision making, task management and situational awareness. The HIRAID assessment framework improves clinical patient assessments performed by emergency nurses and has the potential to enhance patient care. HIRAID should be considered for integration into clinical practice to provide nurses with a systematic approach to patient assessment and potentially improve the delivery of safe patient care. © 2016 John Wiley & Sons Ltd.

Evaluating the use of key performance indicators to evidence the patient experience.

PubMed

McCance, Tanya; Hastings, Jack; Dowler, Hilda

2015-11-01

To test eight person-centred key performance indicators and the feasibility of an appropriate measurement framework as an approach to evidencing the patient experience. The value of measuring the quality of patient care is undisputed in the international literature, however, the type of measures that can be used to generate data that is meaningful for practice continues to be debated. This paper offers a different perspective to the 'measurement' of the nursing and midwifery contribution to the patient experience. Fourth generation evaluation was the methodological approach used to evaluate the implementation of the key performance indicators and measurement framework across three participating organisations involving nine practice settings. Data were collected by repeated use of claims, concerns and issues with staff working across nine participating sites (n = 18) and the senior executives from the three partner organisations (n = 12). Data were collected during the facilitated sessions with stakeholders and analysed in conjunction with the data generated from the measurement framework. The data reveal the inherent value placed on the evidence generated from the implementation of the key performance indicators as reflected in the following themes: measuring what matters; evidencing the patient experience; engaging staff; a focus for improving practice; and articulating and demonstrating the positive contribution of nursing and midwifery. The implementation of the key performance indicators and the measurement framework has been effective in generating evidence that demonstrates the patient experience. The nature of the data generated not only privileges the patient voice but also offers feedback to nurses and midwives that can inform the development of person-centred cultures. The use of these indicators will produce evidence of patient experience that can be used by nurse and midwives to celebrate and further inform person-centred practice. © 2015 John Wiley & Sons Ltd.

Orchestration in Learning Technology Research: Evaluation of a Conceptual Framework

ERIC Educational Resources Information Center

Prieto, Luis P.; Dimitriadis, Yannis; Asensio-Pérez, Juan I.; Looi, Chee-Kit

2015-01-01

The term "orchestrating learning" is being used increasingly often, referring to the coordination activities performed while applying learning technologies to authentic settings. However, there is little consensus about how this notion should be conceptualised, and what aspects it entails. In this paper, a conceptual framework for…

A quantitative evaluation of a qualitative risk assessment framework: Examining the assumptions and predictions of the Productivity Susceptibility Analysis (PSA)

PubMed Central

2018-01-01

Qualitative risk assessment frameworks, such as the Productivity Susceptibility Analysis (PSA), have been developed to rapidly evaluate the risks of fishing to marine populations and prioritize management and research among species. Despite being applied to over 1,000 fish populations, and an ongoing debate about the most appropriate method to convert biological and fishery characteristics into an overall measure of risk, the assumptions and predictive capacity of these approaches have not been evaluated. Several interpretations of the PSA were mapped to a conventional age-structured fisheries dynamics model to evaluate the performance of the approach under a range of assumptions regarding exploitation rates and measures of biological risk. The results demonstrate that the underlying assumptions of these qualitative risk-based approaches are inappropriate, and the expected performance is poor for a wide range of conditions. The information required to score a fishery using a PSA-type approach is comparable to that required to populate an operating model and evaluating the population dynamics within a simulation framework. In addition to providing a more credible characterization of complex system dynamics, the operating model approach is transparent, reproducible and can evaluate alternative management strategies over a range of plausible hypotheses for the system. PMID:29856869

FSM-F: Finite State Machine Based Framework for Denial of Service and Intrusion Detection in MANET

PubMed Central

N. Ahmed, Malik; Abdullah, Abdul Hanan; Kaiwartya, Omprakash

2016-01-01

Due to the continuous advancements in wireless communication in terms of quality of communication and affordability of the technology, the application area of Mobile Adhoc Networks (MANETs) significantly growing particularly in military and disaster management. Considering the sensitivity of the application areas, security in terms of detection of Denial of Service (DoS) and intrusion has become prime concern in research and development in the area. The security systems suggested in the past has state recognition problem where the system is not able to accurately identify the actual state of the network nodes due to the absence of clear definition of states of the nodes. In this context, this paper proposes a framework based on Finite State Machine (FSM) for denial of service and intrusion detection in MANETs. In particular, an Interruption Detection system for Adhoc On-demand Distance Vector (ID-AODV) protocol is presented based on finite state machine. The packet dropping and sequence number attacks are closely investigated and detection systems for both types of attacks are designed. The major functional modules of ID-AODV includes network monitoring system, finite state machine and attack detection model. Simulations are carried out in network simulator NS-2 to evaluate the performance of the proposed framework. A comparative evaluation of the performance is also performed with the state-of-the-art techniques: RIDAN and AODV. The performance evaluations attest the benefits of proposed framework in terms of providing better security for denial of service and intrusion detection attacks. PMID:27285146

Systematic evaluation of atmospheric chemistry-transport model CHIMERE

NASA Astrophysics Data System (ADS)

Khvorostyanov, Dmitry; Menut, Laurent; Mailler, Sylvain; Siour, Guillaume; Couvidat, Florian; Bessagnet, Bertrand; Turquety, Solene

2017-04-01

Regional-scale atmospheric chemistry-transport models (CTM) are used to develop air quality regulatory measures, to support environmentally sensitive decisions in the industry, and to address variety of scientific questions involving the atmospheric composition. Model performance evaluation with measurement data is critical to understand their limits and the degree of confidence in model results. CHIMERE CTM (http://www.lmd.polytechnique.fr/chimere/) is a French national tool for operational forecast and decision support and is widely used in the international research community in various areas of atmospheric chemistry and physics, climate, and environment (http://www.lmd.polytechnique.fr/chimere/CW-articles.php). This work presents the model evaluation framework applied systematically to the new CHIMERE CTM versions in the course of the continuous model development. The framework uses three of the four CTM evaluation types identified by the Environmental Protection Agency (EPA) and the American Meteorological Society (AMS): operational, diagnostic, and dynamic. It allows to compare the overall model performance in subsequent model versions (operational evaluation), identify specific processes and/or model inputs that could be improved (diagnostic evaluation), and test the model sensitivity to the changes in air quality, such as emission reductions and meteorological events (dynamic evaluation). The observation datasets currently used for the evaluation are: EMEP (surface concentrations), AERONET (optical depths), and WOUDC (ozone sounding profiles). The framework is implemented as an automated processing chain and allows interactive exploration of the results via a web interface.

CMAQ Model Evaluation Framework

EPA Pesticide Factsheets

CMAQ is tested to establish the modeling system’s credibility in predicting pollutants such as ozone and particulate matter. Evaluation of CMAQ has been designed to assess the model’s performance for specific time periods and for specific uses.

Evaluation readiness: improved evaluation planning using a data inventory framework.

PubMed

Cohen, A B; Hall, K C; Cohodes, D R

1985-01-01

Factors intrinsic to many programs, such as ambiguously stated objectives, inadequately defined performance measures, and incomplete or unreliable databases, often conspire to limit the evaluability of these programs. Current evaluation planning approaches are somewhat constrained in their ability to overcome these obstacles and to achieve full preparedness for evaluation. In this paper, the concept of evaluation readiness is introduced as a complement to other evaluation planning approaches, most notably that of evaluability assessment. The basic products of evaluation readiness--the formal program definition and the data inventory framework--are described, along with a guide for assuring more timely and appropriate evaluation response capability to support the decision making needs of program managers. The utility of evaluation readiness for program planning, as well as for effective management, is also discussed.

Evaluating the transport layer of the ALFA framework for the Intel® Xeon Phi™ Coprocessor

NASA Astrophysics Data System (ADS)

Santogidis, Aram; Hirstius, Andreas; Lalis, Spyros

2015-12-01

The ALFA framework supports the software development of major High Energy Physics experiments. As part of our research effort to optimize the transport layer of ALFA, we focus on profiling its data transfer performance for inter-node communication on the Intel Xeon Phi Coprocessor. In this article we present the collected performance measurements with the related analysis of the results. The optimization opportunities that are discovered, help us to formulate the future plans of enabling high performance data transfer for ALFA on the Intel Xeon Phi architecture.

Units of Distinction: Creating a Blueprint for Recognition of High-Performing Medical-Surgical Nursing Units.

PubMed

Jeffery, Alvin D; Mosier, Sammie; Baker, Allison; Korwek, Kimberly; Borum, Cindy; Englebright, Jane

2018-02-01

Hospital medical-surgical (M/S) nursing units are responsible for up to 28 million encounters annually, yet receive little attention from professional organizations and national initiatives targeted to improve quality and performance. We sought to develop a framework recognizing high-performing units within our large hospital system. This was a retrospective data analysis of M/S units throughout a 168-hospital system. Measures represented patient experience, employee engagement, staff scheduling, nursing-sensitive patient outcomes, professional practices, and clinical process measures. Four hundred ninety units from 129 hospitals contributed information to test the framework. A manual scoring system identified the top 5% and recognized them as a "Unit of Distinction." Secondary analyses with machine learning provided validation of the proposed framework. Similar to external recognition programs, this framework and process provide a holistic evaluation useful for meaningful recognition and lay the groundwork for benchmarking in improvement efforts.

Matching methods evaluation framework for stereoscopic breast x-ray images.

PubMed

Rousson, Johanna; Naudin, Mathieu; Marchessoux, Cédric

2016-01-01

Three-dimensional (3-D) imaging has been intensively studied in the past few decades. Depth information is an important added value of 3-D systems over two-dimensional systems. Special focuses were devoted to the development of stereo matching methods for the generation of disparity maps (i.e., depth information within a 3-D scene). Dedicated frameworks were designed to evaluate and rank the performance of different stereo matching methods but never considering x-ray medical images. Yet, 3-D x-ray acquisition systems and 3-D medical displays have already been introduced into the diagnostic market. To access the depth information within x-ray stereoscopic images, computing accurate disparity maps is essential. We aimed at developing a framework dedicated to x-ray stereoscopic breast images used to evaluate and rank several stereo matching methods. A multiresolution pyramid optimization approach was integrated to the framework to increase the accuracy and the efficiency of the stereo matching techniques. Finally, a metric was designed to score the results of the stereo matching compared with the ground truth. Eight methods were evaluated and four of them [locally scaled sum of absolute differences (LSAD), zero mean sum of absolute differences, zero mean sum of squared differences, and locally scaled mean sum of squared differences] appeared to perform equally good with an average error score of 0.04 (0 is the perfect matching). LSAD was selected for generating the disparity maps.

Towards a balanced performance measurement system in a public health care organization.

PubMed

Yuen, Peter P; Ng, Artie W

2012-01-01

This article attempts to devise an integrated performance measurement framework to assess the Hong Kong Hospital Authority (HA) management system by harnessing previous performance measurement systems. An integrated evaluative framework based on the balanced score card (BSC) was developed and applied using the case study method and longitudinal data to evaluate the HA's performance management system. The authors unveil evolving HA performance indicators (P1). Despite the HA staffs explicit quality emphasis, cost control remains the primary focus in their performance measurements. RESEARCH LHNITATIONS/IMPLICATIONS: Data used in this study are from secondary sources, disclosed mostly by HA staff. This study shows public sector staff often attach too much importance to cost control and easily measurable activities at the expense of quality and other less easily measurable attributes'. A balanced performance measurement system, linked to health targets, with a complementary budgeting process that supports pertinent resource allocation is yet to be implemented in Hong Kong's public hospitals.

A unified framework for evaluating the risk of re-identification of text de-identification tools.

PubMed

Scaiano, Martin; Middleton, Grant; Arbuckle, Luk; Kolhatkar, Varada; Peyton, Liam; Dowling, Moira; Gipson, Debbie S; El Emam, Khaled

2016-10-01

It has become regular practice to de-identify unstructured medical text for use in research using automatic methods, the goal of which is to remove patient identifying information to minimize re-identification risk. The metrics commonly used to determine if these systems are performing well do not accurately reflect the risk of a patient being re-identified. We therefore developed a framework for measuring the risk of re-identification associated with textual data releases. We apply the proposed evaluation framework to a data set from the University of Michigan Medical School. Our risk assessment results are then compared with those that would be obtained using a typical contemporary micro-average evaluation of recall in order to illustrate the difference between the proposed evaluation framework and the current baseline method. We demonstrate how this framework compares against common measures of the re-identification risk associated with an automated text de-identification process. For the probability of re-identification using our evaluation framework we obtained a mean value for direct identifiers of 0.0074 and a mean value for quasi-identifiers of 0.0022. The 95% confidence interval for these estimates were below the relevant thresholds. The threshold for direct identifier risk was based on previously used approaches in the literature. The threshold for quasi-identifiers was determined based on the context of the data release following commonly used de-identification criteria for structured data. Our framework attempts to correct for poorly distributed evaluation corpora, accounts for the data release context, and avoids the often optimistic assumptions that are made using the more traditional evaluation approach. It therefore provides a more realistic estimate of the true probability of re-identification. This framework should be used as a basis for computing re-identification risk in order to more realistically evaluate future text de-identification tools. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework.

PubMed

Wolterink, Jelmer M; Leiner, Tim; de Vos, Bob D; Coatrieux, Jean-Louis; Kelm, B Michael; Kondo, Satoshi; Salgado, Rodrigo A; Shahzad, Rahil; Shu, Huazhong; Snoeren, Miranda; Takx, Richard A P; van Vliet, Lucas J; van Walsum, Theo; Willems, Tineke P; Yang, Guanyu; Zheng, Yefeng; Viergever, Max A; Išgum, Ivana

2016-05-01

The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiac CT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiac CT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiac CT using a publicly available standardized framework. Cardiac CT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CT scanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient. Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen's kappa for patient CVD risk categorization by the evaluated methods ranged from 0.80 to 1.00. A publicly available standardized framework for the evaluation of (semi)automatic methods for CAC identification in cardiac CT is described. An evaluation of five (semi)automatic methods within this framework shows that automatic per patient CVD risk categorization is feasible. CAC lesions at ambiguous locations such as the coronary ostia remain challenging, but their detection had limited impact on CVD risk determination.

Impacts of Performance Pay under the Teacher Incentive Fund: Study Design Report

ERIC Educational Resources Information Center

Glazerman, Steven; Chiang, Hanley; Wellington, Alison; Constantine, Jill; Player, Dan

2011-01-01

The body of research on the design, implementation, and effects of performance-based compensation systems has influenced the design and evaluation of the 2010 Teacher Incentive Fund (TIF) grants. In the sections presented here, the authors describe the key components of 2010 TIF grants and the conceptual framework for the evaluation. The remainder…

The LSST metrics analysis framework (MAF)

NASA Astrophysics Data System (ADS)

Jones, R. L.; Yoachim, Peter; Chandrasekharan, Srinivasan; Connolly, Andrew J.; Cook, Kem H.; Ivezic, Željko; Krughoff, K. S.; Petry, Catherine; Ridgway, Stephen T.

2014-07-01

We describe the Metrics Analysis Framework (MAF), an open-source python framework developed to provide a user-friendly, customizable, easily-extensible set of tools for analyzing data sets. MAF is part of the Large Synoptic Survey Telescope (LSST) Simulations effort. Its initial goal is to provide a tool to evaluate LSST Operations Simulation (OpSim) simulated surveys to help understand the effects of telescope scheduling on survey performance, however MAF can be applied to a much wider range of datasets. The building blocks of the framework are Metrics (algorithms to analyze a given quantity of data), Slicers (subdividing the overall data set into smaller data slices as relevant for each Metric), and Database classes (to access the dataset and read data into memory). We describe how these building blocks work together, and provide an example of using MAF to evaluate different dithering strategies. We also outline how users can write their own custom Metrics and use these within the framework.

Capitalizing on Curriculum-Based Measurement for Reading: Collaboration within a Response to Instruction Framework

ERIC Educational Resources Information Center

Weiss, Stacy L.; Friesen, Amber

2014-01-01

Response to Instruction (RTI) frameworks provide a structure for assessing student progress and evaluating the effectiveness of reading interventions. Schools frequently use RTI to support students who are struggling with learning to read while utilizing curriculum-based measurement (CBM) to monitor performance and guide instructional decisions…

A novel framework for assessing metadata quality in epidemiological and public health research settings

PubMed Central

McMahon, Christiana; Denaxas, Spiros

2016-01-01

Metadata are critical in epidemiological and public health research. However, a lack of biomedical metadata quality frameworks and limited awareness of the implications of poor quality metadata renders data analyses problematic. In this study, we created and evaluated a novel framework to assess metadata quality of epidemiological and public health research datasets. We performed a literature review and surveyed stakeholders to enhance our understanding of biomedical metadata quality assessment. The review identified 11 studies and nine quality dimensions; none of which were specifically aimed at biomedical metadata. 96 individuals completed the survey; of those who submitted data, most only assessed metadata quality sometimes, and eight did not at all. Our framework has four sections: a) general information; b) tools and technologies; c) usability; and d) management and curation. We evaluated the framework using three test cases and sought expert feedback. The framework can assess biomedical metadata quality systematically and robustly. PMID:27570670

A novel framework for assessing metadata quality in epidemiological and public health research settings.

PubMed

McMahon, Christiana; Denaxas, Spiros

2016-01-01

Metadata are critical in epidemiological and public health research. However, a lack of biomedical metadata quality frameworks and limited awareness of the implications of poor quality metadata renders data analyses problematic. In this study, we created and evaluated a novel framework to assess metadata quality of epidemiological and public health research datasets. We performed a literature review and surveyed stakeholders to enhance our understanding of biomedical metadata quality assessment. The review identified 11 studies and nine quality dimensions; none of which were specifically aimed at biomedical metadata. 96 individuals completed the survey; of those who submitted data, most only assessed metadata quality sometimes, and eight did not at all. Our framework has four sections: a) general information; b) tools and technologies; c) usability; and d) management and curation. We evaluated the framework using three test cases and sought expert feedback. The framework can assess biomedical metadata quality systematically and robustly.

Measuring social accountability in health professional education: development and international pilot testing of an evaluation framework.

PubMed

Larkins, Sarah L; Preston, Robyn; Matte, Marie C; Lindemann, Iris C; Samson, Rex; Tandinco, Filedito D; Buso, David; Ross, Simone J; Pálsdóttir, Björg; Neusy, André-Jacques

2013-01-01

Health professional schools are responsible for producing graduates with competencies and attitudes to address health inequities and respond to priority health needs. Health professional schools striving towards social accountability founded the Training for Health Equity Network (THEnet). This article describes the development of THEnet evaluation framework for socially accountable health professional education, presents the framework to be used as a tool by other schools and discusses the findings of pilot implementation at five schools. The framework was designed collaboratively and built on Boelen and Woollard's conceptualization, production and usability model. It includes key components, linked to aspirational statements, indicators and suggested measurement tools. Five schools completed pilot implementation, involving workshops, document/data review and focus group discussions with faculty, students and community members. Three sections of the framework consider: How does our school work?; What do we do? and What difference do we make? Pilot testing proved that the evaluation framework was acceptable and feasible across contexts and produced findings useful at school level and to compare schools. The framework is designed as a formative exercise to help schools take a critical look at their performance and progress towards social accountability. Initiatives to implement the framework more widely are underway. The framework effectively aids in identifying strengths, weaknesses and gaps, with a view to schools striving for continuous self-improvement. THEnet evaluation framework is applicable and useful across contexts. It is possible and desirable to assess progress towards social accountability in health professional schools and this is an important step in producing health professionals with knowledge, attitudes, and skills to meet the challenges of priority health needs of underserved populations.

Perceived Utility of the RE-AIM Framework for Health Promotion/Disease Prevention Initiatives for Older Adults: A Case Study from the U.S. Evidence-Based Disease Prevention Initiative.

PubMed

Ory, Marcia G; Altpeter, Mary; Belza, Basia; Helduser, Janet; Zhang, Chen; Smith, Matthew Lee

2014-01-01

Dissemination and implementation (D&I) frameworks are increasingly being promoted in public health research. However, less is known about their uptake in the field, especially for diverse sets of programs. Limited questionnaires exist to assess the ways that frameworks can be utilized in program planning and evaluation. We present a case study from the United States that describes the implementation of the RE-AIM framework by state aging services providers and public health partners and a questionnaire that can be used to assess the utility of such frameworks in practice. An online questionnaire was developed to capture community perspectives about the utility of the RE-AIM framework. Distributed to project leads in 27 funded states in an evidence-based disease prevention initiative for older adults, 40 key stakeholders responded representing a 100% state-participation rate among the 27 funded states. Findings suggest that there is perceived utility in using the RE-AIM framework when evaluating grand-scale initiatives for older adults. The RE-AIM framework was seen as useful for planning, implementation, and evaluation with relevance for evaluators, providers, community leaders, and policy makers. Yet, the uptake was not universal, and some respondents reported difficulties in use, especially adopting the framework as a whole. This questionnaire can serve as the basis to assess ways the RE-AIM framework can be utilized by practitioners in state-wide D&I efforts. Maximal benefit can be derived from examining the assessment of RE-AIM-related knowledge and confidence as part of a continual quality assurance process. We recommend such an assessment be performed before the implementation of new funding initiatives and throughout their course to assess RE-AIM uptake and to identify areas for technical assistance.

A Conceptual Framework to Measure Systems’ Performance during Emergency Preparedness Exercises

PubMed Central

Savoia, Elena; Agboola, Foluso; Biddinger, Paul D.

2014-01-01

Large-scale public health emergencies require a sophisticated, coordinated response involving multiple entities to protect health and minimize suffering. However, the rarity of such emergencies presents a barrier to gathering observational data about the effectiveness of the public health response before such events occur. For this reason, public health practitioners increasingly have relied on simulated emergencies, known as “exercises” as proxies to test their emergency capabilities. However, the formal evaluation of performance in these exercises, historically has been inconsistent, and there is little research to describe how data acquired from simulated emergencies actually support conclusions about the quality of the public health emergency response system. Over the past six years, we have designed and evaluated more than seventy public health emergency exercises, collaborating with public health agencies, hospitals and others to test a wide variety of systems and their capabilities. Using the data and experience that we gathered, we have developed a conceptual framework that describes the essential elements necessary to consider when applying performance measurement science to public health emergency exercises. We suggest that this framework may assist practitioners and researchers who wish to better measure performance in exercises and to improve public health emergency preparedness. PMID:25233015

Quality evaluation in health care services based on customer-provider relationships.

PubMed

Eiriz, Vasco; Figueiredo, José António

2005-01-01

To develop a framework for evaluating the quality of Portuguese health care organisations based on the relationship between customers and providers, to define key variables related to the quality of health care services based on a review of the available literature, and to establish a conceptual framework in order to test the framework and variables empirically. Systematic review of the literature. Health care services quality should not be evaluated exclusively by customers. Given the complexity, ambiguity and heterogeneity of health care services, the authors develop a framework for health care evaluation based on the relationship between customers (patients, their relatives and citizens) and providers (managers, doctors, other technical staff and non-technical staff), and considering four quality items (customer service orientation, financial performance, logistical functionality and level of staff competence). This article identifies important changes in the Portuguese health care industry, such as the ownership of health care providers. At the same time, customers are changing their attitudes towards health care, becoming much more concerned and demanding of health services. These changes are forcing Portuguese private and public health care organisations to develop more marketing-oriented services. This article recognises the importance of quality evaluation of health care services as a means of increasing customer satisfaction and organisational efficiency, and develops a framework for health care evaluation based on the relationship between customers and providers.

Why Aren't Evaluations Working and What to Do About It: A Framework for Negotiating Meaningful Evaluation in Nonprofits

ERIC Educational Resources Information Center

Liket, Kellie C.; Rey-Garcia, Marta; Maas, Karen E. H.

2014-01-01

Nonprofit organizations are under great pressure to use evaluations to show that their programs "work" and that they are "effective." However, empirical evidence indicates that nonprofits struggle to perform useful evaluations, especially when conducted under accountability pressures. An increasing body of evidence highlights…

Development and Implementation of a Telecommuting Evaluation Framework, and Modeling the Executive Telecommuting Adoption Process

NASA Astrophysics Data System (ADS)

Vora, V. P.; Mahmassani, H. S.

2002-02-01

This work proposes and implements a comprehensive evaluation framework to document the telecommuter, organizational, and societal impacts of telecommuting through telecommuting programs. Evaluation processes and materials within the outlined framework are also proposed and implemented. As the first component of the evaluation process, the executive survey is administered within a public sector agency. The survey data is examined through exploratory analysis and is compared to a previous survey of private sector executives. The ordinal probit, dynamic probit, and dynamic generalized ordinal probit (DGOP) models of telecommuting adoption are calibrated to identify factors which significantly influence executive adoption preferences and to test the robustness of such factors. The public sector DGOP model of executive willingness to support telecommuting under different program scenarios is compared with an equivalent private sector DGOP model. Through the telecommuting program, a case study of telecommuting travel impacts is performed to further substantiate research.

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation

PubMed Central

Barasa, Edwine W.; Molyneux, Sassy; English, Mike; Cleary, Susan

2015-01-01

Background: Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices. Methods: We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken. Results: Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values. Conclusion: Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought. PMID:26673332

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation.

PubMed

Barasa, Edwine W; Molyneux, Sassy; English, Mike; Cleary, Susan

2015-09-16

Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices. We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken. Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values. Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought. © 2015 by Kerman University of Medical Sciences.

Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation.

PubMed

Witter, Sophie; Toonen, Jurrien; Meessen, Bruno; Kagubare, Jean; Fritsche, György; Vaughan, Kelsey

2013-09-29

Performance-based financing is increasingly being applied in a variety of contexts, with the expectation that it can improve the performance of health systems. However, while there is a growing literature on implementation issues and effects on outputs, there has been relatively little focus on interactions between PBF and health systems and how these should be studied. This paper aims to contribute to filling that gap by developing a framework for assessing the interactions between PBF and health systems, focusing on low and middle income countries. In doing so, it elaborates a general framework for monitoring and evaluating health system reforms in general. This paper is based on an exploratory literature review and on the work of a group of academics and PBF practitioners. The group developed ideas for the monitoring and evaluation framework through exchange of emails and working documents. Ideas were further refined through discussion at the Health Systems Research symposium in Beijing in October 2012, through comments from members of the online PBF Community of Practice and Beijing participants, and through discussion with PBF experts in Bergen in June 2013. The paper starts with a discussion of definitions, to clarify the core concept of PBF and how the different terms are used. It then develops a framework for monitoring its interactions with the health system, structured around five domains of context, the development process, design, implementation and effects. Some of the key questions for monitoring and evaluation are highlighted, and a systematic approach to monitoring effects proposed, structured according to the health system pillars, but also according to inputs, processes and outputs. The paper lays out a broad framework within which indicators can be prioritised for monitoring and evaluation of PBF or other health system reforms. It highlights the dynamic linkages between the domains and the different pillars. All of these are also framed within inter-sectoral and wider societal contexts. It highlights the importance of differentiating short term and long term effects, and also effects (intended and unintended) at different levels of the health system, and for different sectors and areas of the country. Outstanding work will include using and refining the framework and agreeing on the most important hypotheses to test using it, in relation to PBF but also other purchasing and provider payment reforms, as well as appropriate research methods to use for this task.

A Framework to Develop Persuasive Smart Environments

NASA Astrophysics Data System (ADS)

Lobo, Pedro; Romão, Teresa; Dias, A. Eduardo; Danado, José Carlos

This paper presents a framework for the creation of context-sensitive persuasive applications. The framework allows the authoring of new persuasive smart environments producing the appropriate feedback to the users based on different sensors spread throughout the environment to capture contextual information. Using this framework, we created an application, Smart Bins, aimed at promoting users' behavioural changes regarding the recycling of waste materials. Furthermore, to evaluate the usability of our authoring tool, we performed user tests to analyze if developers could successfully create the Smart Bins application using the framework. A description of the Smart Bins application, as well as the results of the user tests, are also presented in this paper.

A general framework for multicharacter segmentation and its application in recognizing multilingual Asian documents

NASA Astrophysics Data System (ADS)

Wen, Di; Ding, Xiaoqing

2003-12-01

In this paper we propose a general framework for character segmentation in complex multilingual documents, which is an endeavor to combine the traditionally separated segmentation and recognition processes into a cooperative system. The framework contains three basic steps: Dissection, Local Optimization and Global Optimization, which are designed to fuse various properties of the segmentation hypotheses hierarchically into a composite evaluation to decide the final recognition results. Experimental results show that this framework is general enough to be applied in variety of documents. A sample system based on this framework to recognize Chinese, Japanese and Korean documents and experimental performance is reported finally.

A framework to assess management performance in district health systems: a qualitative and quantitative case study in Iran.

PubMed

Tabrizi, Jafar Sadegh; Gholipour, Kamal; Iezadi, Shabnam; Farahbakhsh, Mostafa; Ghiasi, Akbar

2018-01-01

The aim was to design a district health management performance framework for Iran's healthcare system. The mixed-method study was conducted between September 2015 and May 2016 in Tabriz, Iran. In this study, the indicators of district health management performance were obtained by analyzing the 45 semi-structured surveys of experts in the public health system. Content validity of performance indicators which were generated in qualitative part were reviewed and confirmed based on content validity index (CVI). Also content validity ratio (CVR) was calculated using data acquired from a survey of 21 experts in quantitative part. The result of this study indicated that, initially, 81 indicators were considered in framework of district health management performance and, at the end, 53 indicators were validated and confirmed. These indicators were classified in 11 categories which include: human resources and organizational creativity, management and leadership, rules and ethics, planning and evaluation, district managing, health resources management and economics, community participation, quality improvement, research in health system, health information management, epidemiology and situation analysis. The designed framework model can be used to assess the district health management and facilitates performance improvement at the district level.

Storying Teacher Education Policy: Critical Counternarratives of Curricular, Pedagogical, and Activist Responses to State-Mandated Teacher Performance Assessments

ERIC Educational Resources Information Center

Henning, Nick; Dover, Alison; Dotson, Erica K.; Agarwal-Rangath, Ruchi

2018-01-01

The rise of high-stakes, standardized, teacher performance assessments (TPAs) is central to the industry being created out of the regulation, policing, and evaluation of university-based teacher education In addition to reinforcing a narrow and counter-critical framework, TPAs can shift responsibility for the evaluation of teacher candidates from…

Teachers as Strategic Classroom Leaders: The Relationship of Their Cognitive and Behavioral Agility to Student Outcomes and Performance Evaluations

ERIC Educational Resources Information Center

Warkentien, Michael

2016-01-01

The purpose of this non-experimental study was to determine whether teacher cognitive and behavioral agility relates to student achievement as measured by their value-added model (VAM) score and their performance evaluation measured through the Marzano instructional practice (IP) framework, and whether that relationship is moderated by contextual…

A strategy for monitoring and evaluating massive open online courses.

PubMed

Chapman, S A; Goodman, S; Jawitz, J; Deacon, A

2016-08-01

We argue that the complex, innovative and adaptive nature of Massive Open Online Course (MOOC) initiatives poses particular challenges to monitoring and evaluation, in that any evaluation strategy will need to follow a systems approach. This article aims to guide organizations implementing MOOCs through a series of steps to assist them in developing a strategy to monitor, improve, and judge the merit of their initiatives. We describe how we operationalise our strategy by first defining the different layers of interacting agents in a given MOOC system. We then tailor our approach to these different layers. Specifically, a two-pronged approach was developed, where we suggest that individual projects be assessed through performance monitoring; assessment criteria for which would be defined at the outset to include coverage, participation, quality and student achievement. In contrast, the success of an overall initiative should be considered within a more adaptive, emergent evaluation inquiry framework. We present the inquiry framework we developed for MOOC initiatives, and show how this framework might be used to develop evaluation questions and an assessment methodology. We also define the more fixed indicators and measures for project performance monitoring. Our strategy is described as it was developed to inform the evaluation of a MOOC initiative at the University of Cape Town (UCT), South Africa. Copyright © 2016 Elsevier Ltd. All rights reserved.

Driving Under the Influence (of Language).

PubMed

Barrett, Daniel Paul; Bronikowski, Scott Alan; Yu, Haonan; Siskind, Jeffrey Mark

2017-06-09

We present a unified framework which supports grounding natural-language semantics in robotic driving. This framework supports acquisition (learning grounded meanings of nouns and prepositions from human sentential annotation of robotic driving paths), generation (using such acquired meanings to generate sentential description of new robotic driving paths), and comprehension (using such acquired meanings to support automated driving to accomplish navigational goals specified in natural language). We evaluate the performance of these three tasks by having independent human judges rate the semantic fidelity of the sentences associated with paths. Overall, machine performance is 74.9%, while the performance of human annotators is 83.8%.

Beyond the plot: technology extrapolation domains for scaling out agronomic science

NASA Astrophysics Data System (ADS)

Rattalino Edreira, Juan I.; Cassman, Kenneth G.; Hochman, Zvi; van Ittersum, Martin K.; van Bussel, Lenny; Claessens, Lieven; Grassini, Patricio

2018-05-01

Ensuring an adequate food supply in systems that protect environmental quality and conserve natural resources requires productive and resource-efficient cropping systems on existing farmland. Meeting this challenge will be difficult without a robust spatial framework that facilitates rapid evaluation and scaling-out of currently available and emerging technologies. Here we develop a global spatial framework to delineate ‘technology extrapolation domains’ based on key climate and soil factors that govern crop yields and yield stability in rainfed crop production. The proposed framework adequately represents the spatial pattern of crop yields and stability when evaluated over the data-rich US Corn Belt. It also facilitates evaluation of cropping system performance across continents, which can improve efficiency of agricultural research that seeks to intensify production on existing farmland. Populating this biophysical spatial framework with appropriate socio-economic attributes provides the potential to amplify the return on investments in agricultural research and development by improving the effectiveness of research prioritization and impact assessment.

An Odds Ratio Approach for Detecting DDF under the Nested Logit Modeling Framework

ERIC Educational Resources Information Center

Terzi, Ragip; Suh, Youngsuk

2015-01-01

An odds ratio approach (ORA) under the framework of a nested logit model was proposed for evaluating differential distractor functioning (DDF) in multiple-choice items and was compared with an existing ORA developed under the nominal response model. The performances of the two ORAs for detecting DDF were investigated through an extensive…

Building an automated SOAP classifier for emergency department reports.

PubMed

Mowery, Danielle; Wiebe, Janyce; Visweswaran, Shyam; Harkema, Henk; Chapman, Wendy W

2012-02-01

Information extraction applications that extract structured event and entity information from unstructured text can leverage knowledge of clinical report structure to improve performance. The Subjective, Objective, Assessment, Plan (SOAP) framework, used to structure progress notes to facilitate problem-specific, clinical decision making by physicians, is one example of a well-known, canonical structure in the medical domain. Although its applicability to structuring data is understood, its contribution to information extraction tasks has not yet been determined. The first step to evaluating the SOAP framework's usefulness for clinical information extraction is to apply the model to clinical narratives and develop an automated SOAP classifier that classifies sentences from clinical reports. In this quantitative study, we applied the SOAP framework to sentences from emergency department reports, and trained and evaluated SOAP classifiers built with various linguistic features. We found the SOAP framework can be applied manually to emergency department reports with high agreement (Cohen's kappa coefficients over 0.70). Using a variety of features, we found classifiers for each SOAP class can be created with moderate to outstanding performance with F(1) scores of 93.9 (subjective), 94.5 (objective), 75.7 (assessment), and 77.0 (plan). We look forward to expanding the framework and applying the SOAP classification to clinical information extraction tasks. Copyright © 2011. Published by Elsevier Inc.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Spogen, L.R.; Cleland, L.L.

An approach to the development of performance based regulations (PBR's) is described. Initially, a framework is constructed that consists of a function hierarchy and associated measures. The function at the top of the hierarchy is described in terms of societal objectives. Decomposition of this function into subordinate functions and their subsequent decompositions yield the function hierarchy. ''Bottom'' functions describe the roles of system components. When measures are identified for the performance of each function and means of aggregating performances to higher levels are established, the framework may be employed for developing PBR's. Consideration of system flexibility and performance uncertainty guidemore » in determining the hierarchical level at which regulations are formulated. Ease of testing compliance is also a factor. To show the viability of the approach, the framework developed by Lawrence Livermore Laboratory for the Nuclear Regulatory Commission for evaluation of material control systems at fixed facilities is presented.« less

Real-Time Data Capture and Management Evaluation and Performance Measures : Evaluation Framework

DOT National Transportation Integrated Search

2011-09-01

Through connected vehicle research, the U.S. DOT Intelligent Transportation Systems Joint Program Office (ITS JPO) is leading an effort to assess the potential for systematic and dynamic data capture from vehicles, travelers and the transportation sy...

Evaluation of stormwater harvesting sites using multi criteria decision methodology

NASA Astrophysics Data System (ADS)

Inamdar, P. M.; Sharma, A. K.; Cook, Stephen; Perera, B. J. C.

2018-07-01

Selection of suitable urban stormwater harvesting sites and associated project planning are often complex due to spatial, temporal, economic, environmental and social factors, and related various other variables. This paper is aimed at developing a comprehensive methodology framework for evaluating of stormwater harvesting sites in urban areas using Multi Criteria Decision Analysis (MCDA). At the first phase, framework selects potential stormwater harvesting (SWH) sites using spatial characteristics in a GIS environment. In second phase, MCDA methodology is used for evaluating and ranking of SWH sites in multi-objective and multi-stakeholder environment. The paper briefly describes first phase of framework and focuses chiefly on the second phase of framework. The application of the methodology is also demonstrated over a case study comprising of the local government area, City of Melbourne (CoM), Australia for the benefit of wider water professionals engaged in this area. Nine performance measures (PMs) were identified to characterise the objectives and system performance related to the eight alternative SWH sites for the demonstration of the application of developed methodology. To reflect the stakeholder interests in the current study, four stakeholder participant groups were identified, namely, water authorities (WA), academics (AC), consultants (CS), and councils (CL). The decision analysis methodology broadly consisted of deriving PROMETHEE II rankings of eight alternative SWH sites in the CoM case study, under two distinct group decision making scenarios. The major innovation of this work is the development and application of comprehensive methodology framework that assists in the selection of potential sites for SWH, and facilitates the ranking in multi-objective and multi-stakeholder environment. It is expected that the proposed methodology will assist the water professionals and managers with better knowledge that will reduce the subjectivity in the selection and evaluation of SWH sites.

Defect Detection and Segmentation Framework for Remote Field Eddy Current Sensor Data

PubMed Central

2017-01-01

Remote-Field Eddy-Current (RFEC) technology is often used as a Non-Destructive Evaluation (NDE) method to prevent water pipe failures. By analyzing the RFEC data, it is possible to quantify the corrosion present in pipes. Quantifying the corrosion involves detecting defects and extracting their depth and shape. For large sections of pipelines, this can be extremely time-consuming if performed manually. Automated approaches are therefore well motivated. In this article, we propose an automated framework to locate and segment defects in individual pipe segments, starting from raw RFEC measurements taken over large pipelines. The framework relies on a novel feature to robustly detect these defects and a segmentation algorithm applied to the deconvolved RFEC signal. The framework is evaluated using both simulated and real datasets, demonstrating its ability to efficiently segment the shape of corrosion defects. PMID:28984823

Defect Detection and Segmentation Framework for Remote Field Eddy Current Sensor Data.

PubMed

Falque, Raphael; Vidal-Calleja, Teresa; Miro, Jaime Valls

2017-10-06

Remote-Field Eddy-Current (RFEC) technology is often used as a Non-Destructive Evaluation (NDE) method to prevent water pipe failures. By analyzing the RFEC data, it is possible to quantify the corrosion present in pipes. Quantifying the corrosion involves detecting defects and extracting their depth and shape. For large sections of pipelines, this can be extremely time-consuming if performed manually. Automated approaches are therefore well motivated. In this article, we propose an automated framework to locate and segment defects in individual pipe segments, starting from raw RFEC measurements taken over large pipelines. The framework relies on a novel feature to robustly detect these defects and a segmentation algorithm applied to the deconvolved RFEC signal. The framework is evaluated using both simulated and real datasets, demonstrating its ability to efficiently segment the shape of corrosion defects.

Total Performance Development Systems: Blueprints for Building and Sustaining Organizational Excellence

ERIC Educational Resources Information Center

Schanie, Charles F.; Kemper, James E.

2008-01-01

Most "performance evaluation" or "performance development" programs in higher education today are little more than metrically-weak, bureaucratic programs aimed simply at establishing a legally-grounded employee record and a rough basis for merit increases. This article outlines the rationale and procedural framework for a talent development and…

Project SIDE. Performance Report. 1992-1993.

ERIC Educational Resources Information Center

Alabama Southern Community Coll., Monroeville.

This document consists of the performance report and evaluation report of the second funding of Project SIDE, a workplace literacy project at Alabama Southern Community College (ASCC). The performance report contains the following: a list of the industrial partners; basic skills framework; chronological listing by company of courses offered;…

5 CFR 430.303 - Definitions.

Code of Federal Regulations, 2013 CFR

2013-01-01

..., including customers and employees. Critical element means a key component of an executive's work that.... Performance management system means the framework of policies and practices that an agency establishes under..., developing, evaluating, and rewarding both individual and organizational performance and for using resulting...

5 CFR 430.303 - Definitions.

Code of Federal Regulations, 2014 CFR

2014-01-01

..., including customers and employees. Critical element means a key component of an executive's work that.... Performance management system means the framework of policies and practices that an agency establishes under..., developing, evaluating, and rewarding both individual and organizational performance and for using resulting...

5 CFR 430.303 - Definitions.

Code of Federal Regulations, 2012 CFR

2012-01-01

..., including customers and employees. Critical element means a key component of an executive's work that.... Performance management system means the framework of policies and practices that an agency establishes under..., developing, evaluating, and rewarding both individual and organizational performance and for using resulting...

A high performance data parallel tensor contraction framework: Application to coupled electro-mechanics

NASA Astrophysics Data System (ADS)

Poya, Roman; Gil, Antonio J.; Ortigosa, Rogelio

2017-07-01

The paper presents aspects of implementation of a new high performance tensor contraction framework for the numerical analysis of coupled and multi-physics problems on streaming architectures. In addition to explicit SIMD instructions and smart expression templates, the framework introduces domain specific constructs for the tensor cross product and its associated algebra recently rediscovered by Bonet et al. (2015, 2016) in the context of solid mechanics. The two key ingredients of the presented expression template engine are as follows. First, the capability to mathematically transform complex chains of operations to simpler equivalent expressions, while potentially avoiding routes with higher levels of computational complexity and, second, to perform a compile time depth-first or breadth-first search to find the optimal contraction indices of a large tensor network in order to minimise the number of floating point operations. For optimisations of tensor contraction such as loop transformation, loop fusion and data locality optimisations, the framework relies heavily on compile time technologies rather than source-to-source translation or JIT techniques. Every aspect of the framework is examined through relevant performance benchmarks, including the impact of data parallelism on the performance of isomorphic and nonisomorphic tensor products, the FLOP and memory I/O optimality in the evaluation of tensor networks, the compilation cost and memory footprint of the framework and the performance of tensor cross product kernels. The framework is then applied to finite element analysis of coupled electro-mechanical problems to assess the speed-ups achieved in kernel-based numerical integration of complex electroelastic energy functionals. In this context, domain-aware expression templates combined with SIMD instructions are shown to provide a significant speed-up over the classical low-level style programming techniques.

A validation framework for brain tumor segmentation.

PubMed

Archip, Neculai; Jolesz, Ferenc A; Warfield, Simon K

2007-10-01

We introduce a validation framework for the segmentation of brain tumors from magnetic resonance (MR) images. A novel unsupervised semiautomatic brain tumor segmentation algorithm is also presented. The proposed framework consists of 1) T1-weighted MR images of patients with brain tumors, 2) segmentation of brain tumors performed by four independent experts, 3) segmentation of brain tumors generated by a semiautomatic algorithm, and 4) a software tool that estimates the performance of segmentation algorithms. We demonstrate the validation of the novel segmentation algorithm within the proposed framework. We show its performance and compare it with existent segmentation. The image datasets and software are available at http://www.brain-tumor-repository.org/. We present an Internet resource that provides access to MR brain tumor image data and segmentation that can be openly used by the research community. Its purpose is to encourage the development and evaluation of segmentation methods by providing raw test and image data, human expert segmentation results, and methods for comparing segmentation results.

Video scrambling for privacy protection in video surveillance: recent results and validation framework

NASA Astrophysics Data System (ADS)

Dufaux, Frederic

2011-06-01

The issue of privacy in video surveillance has drawn a lot of interest lately. However, thorough performance analysis and validation is still lacking, especially regarding the fulfillment of privacy-related requirements. In this paper, we first review recent Privacy Enabling Technologies (PET). Next, we discuss pertinent evaluation criteria for effective privacy protection. We then put forward a framework to assess the capacity of PET solutions to hide distinguishing facial information and to conceal identity. We conduct comprehensive and rigorous experiments to evaluate the performance of face recognition algorithms applied to images altered by PET. Results show the ineffectiveness of naïve PET such as pixelization and blur. Conversely, they demonstrate the effectiveness of more sophisticated scrambling techniques to foil face recognition.

Environmental performance policy indicators for the public sector: the case of the defence sector.

PubMed

Ramos, Tomás B; Alves, Inês; Subtil, Rui; Joanaz de Melo, João

2007-03-01

The development of environmental performance policy indicators for public services, and in particular for the defence sector, is an emerging issue. Despite a number of recent initiatives there has been little work done in this area, since the other sectors usually focused on are agriculture, transport, industry, tourism and energy. This type of tool can be an important component for environmental performance evaluation at policy level, when integrated in the general performance assessment system of public missions and activities. The main objective of this research was to develop environmental performance policy indicators for the public sector, specifically applied to the defence sector. Previous research included an assessment of the environmental profile, through the evaluation of how environmental management practices have been adopted in this sector and an assessment of environmental aspects and impacts. This paper builds upon that previous research, developing an indicator framework--SEPI--supported by the selection and construction of environmental performance indicators. Another aim is to discuss how the current environmental indicator framework can be integrated into overall performance management. The Portuguese defence sector is presented and the usefulness of this methodology demonstrated. Feasibility and relevancy criteria are applied to evaluate the set of indicators proposed, allowing indicators to be scored and indicators for the policy level to be obtained.

Implementation of a Systematic Accountability Framework in 2014 to Improve the Performance of the Nigerian Polio Program

PubMed Central

Tegegne, Sisay G.; MKanda, Pascal; Yehualashet, Yared G.; Erbeto, Tesfaye B.; Touray, Kebba; Nsubuga, Peter; Banda, Richard; Vaz, Rui G.

2016-01-01

Background. An accountability framework is a central feature of managing human and financial resources. One of its primary goals is to improve program performance through close monitoring of selected priority activities. The principal objective of this study was to determine the contribution of a systematic accountability framework to improving the performance of the World Health Organization (WHO)–Nigeria polio program staff, as well as the program itself. Methods. The effect of implementation of the accountability framework was evaluated using data on administrative actions and select process indicators associated with acute flaccid paralysis (AFP) surveillance, routine immunization, and polio supplemental immunization activities. Data were collected in 2014 during supportive supervision, using Magpi software (a company that provides service to collect data using mobile phones). A total of 2500 staff were studied. Results. Data on administrative actions and process indicators from quarters 2–4 in 2014 were compared. With respect to administrative actions, 1631 personnel (74%) received positive feedback (written or verbal commendation) in quarter 4 through the accountability framework, compared with 1569 (73%) and 1152 (61%) during quarters 3 and 2, respectively. These findings accorded with data on process indicators associated with AFP surveillance and routine immunization, showing statistically significant improvements in staff performance at the end of quarter 4, compared with other quarters. Conclusions. Improvements in staff performance and process indicators were observed for the WHO-Nigeria polio program after implementation of a systematic accountability framework. PMID:26823334

Filtering Based Adaptive Visual Odometry Sensor Framework Robust to Blurred Images

PubMed Central

Zhao, Haiying; Liu, Yong; Xie, Xiaojia; Liao, Yiyi; Liu, Xixi

2016-01-01

Visual odometry (VO) estimation from blurred image is a challenging problem in practical robot applications, and the blurred images will severely reduce the estimation accuracy of the VO. In this paper, we address the problem of visual odometry estimation from blurred images, and present an adaptive visual odometry estimation framework robust to blurred images. Our approach employs an objective measure of images, named small image gradient distribution (SIGD), to evaluate the blurring degree of the image, then an adaptive blurred image classification algorithm is proposed to recognize the blurred images, finally we propose an anti-blurred key-frame selection algorithm to enable the VO robust to blurred images. We also carried out varied comparable experiments to evaluate the performance of the VO algorithms with our anti-blur framework under varied blurred images, and the experimental results show that our approach can achieve superior performance comparing to the state-of-the-art methods under the condition with blurred images while not increasing too much computation cost to the original VO algorithms. PMID:27399704

Identifying and Evaluating External Validity Evidence for Passing Scores

ERIC Educational Resources Information Center

Davis-Becker, Susan L.; Buckendahl, Chad W.

2013-01-01

A critical component of the standard setting process is collecting evidence to evaluate the recommended cut scores and their use for making decisions and classifying students based on test performance. Kane (1994, 2001) proposed a framework by which practitioners can identify and evaluate evidence of the results of the standard setting from (1)…

Science and Art of Setting Performance Standards and Cutoff Scores in Kinesiology

ERIC Educational Resources Information Center

Zhu, Weimo

2013-01-01

Setting standards and cutoff scores is essential to any measurement and evaluation practice. Two evaluation frameworks, norm-referenced (NR) and criterion-referenced (CR), have often been used for setting standards. Although setting fitness standards based on the NR evaluation is relatively easy as long as a nationally representative sample can be…

Evaluating and Improving Cloud Processes in the Multi-Scale Modeling Framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ackerman, Thomas P.

2015-03-01

The research performed under this grant was intended to improve the embedded cloud model in the Multi-scale Modeling Framework (MMF) for convective clouds by using a 2-moment microphysics scheme rather than the single moment scheme used in all the MMF runs to date. The technical report and associated documents describe the results of testing the cloud resolving model with fixed boundary conditions and evaluation of model results with data. The overarching conclusion is that such model evaluations are problematic because errors in the forcing fields control the results so strongly that variations in parameterization values cannot be usefully constrained

A framework for evaluating complex networks measurements

NASA Astrophysics Data System (ADS)

Comin, Cesar H.; Silva, Filipi N.; Costa, Luciano da F.

2015-06-01

A good deal of current research in complex networks involves the characterization and/or classification of the topological properties of given structures, which has motivated several respective measurements. This letter proposes a framework for evaluating the quality of complex-network measurements in terms of their effective resolution, degree of degeneracy and discriminability. The potential of the suggested approach is illustrated with respect to comparing the characterization of several model and real-world networks by using concentric and symmetry measurements. The results indicate a markedly superior performance for the latter type of mapping.

Using frameworks to diagram value in complex policy and environmental interventions to prevent childhood obesity.

PubMed

Swank, Melissa Farrell; Brennan, Laura K; Gentry, Daniel; Kemner, Allison L

2015-01-01

To date, few tools assist policy makers and practitioners in understanding and conveying the implementation costs, potential impacts, and value of policy and environmental changes to address healthy eating, active living, and childhood obesity. For the Evaluation of Healthy Kids, Healthy Communities (HKHC), evaluators considered inputs (resources and investments) that generate costs and savings as well as benefits and harms related to social, economic, environmental, and health-related outcomes in their assessment of 49 HKHC community partnerships funded from 2009 to 2014. Using data collected through individual and group interviews and an online performance monitoring system, evaluators created a socioecological framework to assess investments, resources, costs, savings, benefits, and harms at the individual, organizational, community, and societal levels. Evaluators customized frameworks for 6 focal strategies: active transportation, parks and play spaces, child care physical activity standards, corner stores, farmers' markets, and child care nutrition standards. To illustrate the Value Frameworks, this brief highlights the 38 HKHC communities implementing at least 1 active transportation strategy. Evaluators populated this conceptual Value Framework with themes from the strategy-specific inputs and outputs. The range of factors corresponding to the implementation and impact of the HKHC community partnerships are highlighted along with the inputs and outputs. The Value Frameworks helped evaluators identify gaps in current analysis models (ie, benefit-cost analysis, cost-effectiveness analysis) as well as paint a more complete picture of value for potential obesity prevention strategies. These frameworks provide a comprehensive understanding of investments needed, proposed costs and savings, and potential benefits and harms associated with economic, social, environmental, and health outcomes. This framing also allowed evaluators to demonstrate the interdependence of each socioecological level on the others in these multicomponent interventions. This model can be used by practitioners and community leaders to assess realistic and sustainable strategies to combat childhood obesity.

A Planning and Evaluation Six-Pack for Sustainable Organizations: The Six-P Framework

ERIC Educational Resources Information Center

Marker, Anthony; Johnsen, Elizabeth; Caswell, Christina

2009-01-01

As performance improvement (PI) practitioners, we seek not only to solve organizational problems but also to add value. For some time, we have focused on financial value. However, we are beginning to be held accountable also for the impact of our interventions on society and the environment. The Six-P framework--proposed here--can help PI…

A Historical Perspective of Testing and Assessment Including the Impact of Summative and Formative Assessment on Student Achievement

ERIC Educational Resources Information Center

Brink, Carole Sanger

2011-01-01

In 2007, Georgia developed a comprehensive framework to define what students need to know. One component of this framework emphasizes the use of both formative and summative assessments as part of an integral and specific component of the teachers. performance evaluation. Georgia administers the Criterion-Referenced Competency Test (CRCT) to every…

Middle Managers' Experience of Policy Implementation and Mediation in the Context of the Scottish Quality Enhancement Framework

ERIC Educational Resources Information Center

Saunders, Murray; Sin, Cristina

2015-01-01

This paper analyses how middle managers perform and experience their role in enacting policy in Scottish higher education institutions. The policy focus is the quality enhancement framework (QEF) for learning and teaching in higher education, which was launched in 2003. The data-set was collected between 2008 and 2010, during the evaluation of the…

Rapid Prototyping of an Aircraft Model in an Object-Oriented Simulation

NASA Technical Reports Server (NTRS)

Kenney, P. Sean

2003-01-01

A team was created to participate in the Mars Scout Opportunity. Trade studies determined that an aircraft provided the best opportunity to complete the science objectives of the team. A high fidelity six degree of freedom flight simulation was required to provide credible evidence that the aircraft design fulfilled mission objectives and to support the aircraft design process by providing performance evaluations. The team created the simulation using the Langley Standard Real-Time Simulation in C++ (LaSRS++) application framework. A rapid prototyping approach was necessary because the team had only three months to both develop the aircraft simulation model and evaluate aircraft performance as the design and mission parameters matured. The design of LaSRS++ enabled rapid-prototyping in several ways. First, the framework allowed component models to be designed, implemented, unit-tested, and integrated quickly. Next, the framework provides a highly reusable infrastructure that allowed developers to maximize code reuse while concentrating on aircraft and mission specific features. Finally, the framework reduces risk by providing reusable components that allow developers to build a quality product with a compressed testing cycle that relies heavily on unit testing of new components.

A research framework for pharmacovigilance in health social media: Identification and evaluation of patient adverse drug event reports.

PubMed

Liu, Xiao; Chen, Hsinchun

2015-12-01

Social media offer insights of patients' medical problems such as drug side effects and treatment failures. Patient reports of adverse drug events from social media have great potential to improve current practice of pharmacovigilance. However, extracting patient adverse drug event reports from social media continues to be an important challenge for health informatics research. In this study, we develop a research framework with advanced natural language processing techniques for integrated and high-performance patient reported adverse drug event extraction. The framework consists of medical entity extraction for recognizing patient discussions of drug and events, adverse drug event extraction with shortest dependency path kernel based statistical learning method and semantic filtering with information from medical knowledge bases, and report source classification to tease out noise. To evaluate the proposed framework, a series of experiments were conducted on a test bed encompassing about postings from major diabetes and heart disease forums in the United States. The results reveal that each component of the framework significantly contributes to its overall effectiveness. Our framework significantly outperforms prior work. Published by Elsevier Inc.

Understanding urban water performance at the city-region scale using an urban water metabolism evaluation framework.

PubMed

Renouf, Marguerite A; Kenway, Steven J; Lam, Ka Leung; Weber, Tony; Roux, Estelle; Serrao-Neumann, Silvia; Choy, Darryl Low; Morgan, Edward A

2018-06-15

Water sensitive interventions are being promoted to reduce the adverse impacts of urban development on natural water cycles. However it is currently difficult to know the best strategy for their implementation because current and desired urban water performance is not well quantified. This is particularly at the city-region scale, which is important for strategic urban planning. This work aimed to fill this gap by quantifying the water performance of urban systems within city-regions using 'urban water metabolism' evaluation, to inform decisions about water sensitive interventions. To do this we adapted an existing evaluation framework with new methods. In particular, we used land use data for defining system boundaries, and for estimating natural hydrological flows. The criteria for gauging the water performance were water efficiency (in terms of water extracted externally) and hydrological performance (how much natural hydrological flows have changed relative to a nominated pre-urbanised state). We compared these performance criteria for urban systems within three Australian city-regions (South East Queensland, Melbourne and Perth metropolitan areas), under current conditions, and after implementation of example water sensitive interventions (demand management, rainwater/stormwater harvesting, wastewater recycling and increasing perviousness). The respective water efficiencies were found to be 79, 90 and 133 kL/capita/yr. In relation to hydrological performance, stormwater runoff relative to pre-urbanised flows was of most note, estimated to be 2-, 6- and 3- fold, respectively. The estimated performance benefits from water sensitive interventions suggested different priorities for each region, and that combined implementation of a range of interventions may be necessary to make substantive gains in performance. We concluded that the framework is suited to initial screening of the type and scale of water sensitive interventions needed to achieve desired water performance objectives. Copyright © 2018 Elsevier Ltd. All rights reserved.

Development of Type 2 Diabetes Mellitus Phenotyping Framework Using Expert Knowledge and Machine Learning Approach.

PubMed

Kagawa, Rina; Kawazoe, Yoshimasa; Ida, Yusuke; Shinohara, Emiko; Tanaka, Katsuya; Imai, Takeshi; Ohe, Kazuhiko

2017-07-01

Phenotyping is an automated technique that can be used to distinguish patients based on electronic health records. To improve the quality of medical care and advance type 2 diabetes mellitus (T2DM) research, the demand for T2DM phenotyping has been increasing. Some existing phenotyping algorithms are not sufficiently accurate for screening or identifying clinical research subjects. We propose a practical phenotyping framework using both expert knowledge and a machine learning approach to develop 2 phenotyping algorithms: one is for screening; the other is for identifying research subjects. We employ expert knowledge as rules to exclude obvious control patients and machine learning to increase accuracy for complicated patients. We developed phenotyping algorithms on the basis of our framework and performed binary classification to determine whether a patient has T2DM. To facilitate development of practical phenotyping algorithms, this study introduces new evaluation metrics: area under the precision-sensitivity curve (AUPS) with a high sensitivity and AUPS with a high positive predictive value. The proposed phenotyping algorithms based on our framework show higher performance than baseline algorithms. Our proposed framework can be used to develop 2 types of phenotyping algorithms depending on the tuning approach: one for screening, the other for identifying research subjects. We develop a novel phenotyping framework that can be easily implemented on the basis of proper evaluation metrics, which are in accordance with users' objectives. The phenotyping algorithms based on our framework are useful for extraction of T2DM patients in retrospective studies.

How Do Surgery Students Use Written Language to Say What They See? A Framework to Understand Medical Students' Written Evaluations of Their Teachers.

PubMed

Lim, David W; White, Jonathan S

2015-11-01

There remains debate regarding the value of the written comments that medical students are traditionally asked to provide to evaluate the teaching they receive. The purpose of this study was to examine written teaching evaluations to understand how medical students conceptualize teachers' behaviors and performance. All written comments collected from medical students about teachers in the two surgery clerkships at the University of Alberta in 2009-2010 and 2010-2011 were collated and anonymized. A grounded theory approach was used for analysis, with iterative reading and open coding to identify recurring themes. A framework capturing variations observed in the data was generated until data saturation was achieved. Domains and subdomains were named using an in situ coding approach. The conceptual framework contained three main domains: "Physician as Teacher," "Physician as Person," and "Physician as Physician." Under "Physician as Teacher," students commented on specific acts of teaching and subjective perceptions of an educator's teaching values. Under the "Physician as Physician" domain, students commented on elements of their educator's physicianship, including communication and collaborative skills, medical expertise, professionalism, and role modeling. Under "Physician as Person," students commented on how both positive and negative personality traits impacted their learning. This framework describes how medical students perceive their teachers and how they use written language to attach meaning to the behaviors they observe. Such a framework can be used to help students provide more constructive feedback to teachers and to assist in faculty development efforts aimed at improving teaching performance.

A Comparison Framework for Reactor Anti-Neutrino Detectors in Near-Field Nuclear Safeguards Applications

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mendenhall, M.; Bowden, N.; Brodsky, J.

Electron anti-neutrino ( e) detectors can support nuclear safeguards, from reactor monitoring to spent fuel characterization. In recent years, the scientific community has developed multiple detector concepts, many of which have been prototyped or deployed for specific measurements by their respective collaborations. However, the diversity of technical approaches, deployment conditions, and analysis techniques complicates direct performance comparison between designs. We have begun development of a simulation framework to compare and evaluate existing and proposed detector designs for nonproliferation applications in a uniform manner. This report demonstrates the intent and capabilities of the framework by evaluating four detector design concepts, calculatingmore » generic reactor antineutrino counting sensitivity, and capabilities in a plutonium disposition application example.« less

The outcome competency framework for practitioners in infection prevention and control: use of the outcome logic model for evaluation.

PubMed

Burnett, E; Curran, E; Loveday, H P; Kiernan, M A; Tannahill, M

2014-01-01

Healthcare is delivered in a dynamic environment with frequent changes in populations, methods, equipment and settings. Infection prevention and control practitioners (IPCPs) must ensure that they are competent in addressing the challenges they face and are equipped to develop infection prevention and control (IPC) services in line with a changing world of healthcare provision. A multifaceted Framework was developed to assist IPCPs to enhance competence at an individual, team and organisational level to enable quality performance and improved quality of care. However, if these aspirations are to be met, it is vital that competency frameworks are fit for purpose or they risk being ignored. The aim of this unique study was to evaluate short and medium term outcomes as set out in the Outcome Logic Model to assist with the evaluation of the impact and success of the Framework. This study found that while the Framework is being used effectively in some areas, it is not being used as much or in the ways that were anticipated. The findings will enable future work on revision, communication and dissemination, and will provide intelligence to those initiating education and training in the utilisation of the competences.

The outcome competency framework for practitioners in infection prevention and control: use of the outcome logic model for evaluation

PubMed Central

Curran, E; Loveday, HP; Kiernan, MA; Tannahill, M

2013-01-01

Healthcare is delivered in a dynamic environment with frequent changes in populations, methods, equipment and settings. Infection prevention and control practitioners (IPCPs) must ensure that they are competent in addressing the challenges they face and are equipped to develop infection prevention and control (IPC) services in line with a changing world of healthcare provision. A multifaceted Framework was developed to assist IPCPs to enhance competence at an individual, team and organisational level to enable quality performance and improved quality of care. However, if these aspirations are to be met, it is vital that competency frameworks are fit for purpose or they risk being ignored. The aim of this unique study was to evaluate short and medium term outcomes as set out in the Outcome Logic Model to assist with the evaluation of the impact and success of the Framework. This study found that while the Framework is being used effectively in some areas, it is not being used as much or in the ways that were anticipated. The findings will enable future work on revision, communication and dissemination, and will provide intelligence to those initiating education and training in the utilisation of the competences. PMID:28989348

The Isprs Benchmark on Indoor Modelling

NASA Astrophysics Data System (ADS)

Khoshelham, K.; Díaz Vilariño, L.; Peter, M.; Kang, Z.; Acharya, D.

2017-09-01

Automated generation of 3D indoor models from point cloud data has been a topic of intensive research in recent years. While results on various datasets have been reported in literature, a comparison of the performance of different methods has not been possible due to the lack of benchmark datasets and a common evaluation framework. The ISPRS benchmark on indoor modelling aims to address this issue by providing a public benchmark dataset and an evaluation framework for performance comparison of indoor modelling methods. In this paper, we present the benchmark dataset comprising several point clouds of indoor environments captured by different sensors. We also discuss the evaluation and comparison of indoor modelling methods based on manually created reference models and appropriate quality evaluation criteria. The benchmark dataset is available for download at: http://www2.isprs.org/commissions/comm4/wg5/benchmark-on-indoor-modelling.html.

A new framework for sustainable hydropower development project

NASA Astrophysics Data System (ADS)

Johan, Kartina; Turan, Faiz Mohd; Gani, Nur Syazwani Abdul

2018-03-01

This project studies on the establishment of a new framework for sustainable hydropower development. A hydropower development is listed as one of the prescribed activities under the Environmental Quality Order 1987. Thus, Environmental Impact Assessment (EIA) guidelines must be referred to comply with the Department of Environment (DoE) requirements. In order to execute EIA, an assessment tool that will be utilized in the final evaluation phase must be determined. The selected assessment tool that will be used is Systematic Sustainability Assessment(SSA) which is a new integrated tool to evaluate the sustainability performance. A pilot run is conducted in five different departments within the Energy Company to validate the efficiency of the SSA tool. The parameters to be evaluated are constructed aligned with the Sustainable Development Goals (SDG) to maintain the sustainability features. Consequently, the performance level of the sustainability with respect to People, Planet and Profit (3P’s) is able to be discovered during evaluation phase in the hydropower development for continuous improvement.

Team Performance Assessment and Measurement: Theory, Methods, and Applications. Series in Applied Psychology.

ERIC Educational Resources Information Center

Brannick, Michael T., Ed.; Salas, Eduardo, Ed.; Prince, Carolyn, Ed.

This volume presents thoughts on measuring team performance written by experts currently working with teams in fields such as training, evaluation, and process consultation. The chapters are: (1) "An Overview of Team Performance Measurement" (Michael T. Brannick and Carolyn Prince); (2) "A Conceptual Framework for Teamwork Measurement" (Terry L.…

Performance measurement evaluation framework and co-benefit / tradeoff analysis for connected and automated vehicles (CAV) applications : a survey : a research report from the National Center for Sustainable Transportation.

DOT National Transportation Integrated Search

2017-09-01

A number of Connected and/or Automated Vehicle (CAV) applications have recently been designed to improve the performance of our transportation system. Safety, mobility and environmental sustainability are three cornerstone performance metrics when ev...

Towards a Framework for Evaluating and Comparing Diagnosis Algorithms

NASA Technical Reports Server (NTRS)

Kurtoglu, Tolga; Narasimhan, Sriram; Poll, Scott; Garcia,David; Kuhn, Lukas; deKleer, Johan; vanGemund, Arjan; Feldman, Alexander

2009-01-01

Diagnostic inference involves the detection of anomalous system behavior and the identification of its cause, possibly down to a failed unit or to a parameter of a failed unit. Traditional approaches to solving this problem include expert/rule-based, model-based, and data-driven methods. Each approach (and various techniques within each approach) use different representations of the knowledge required to perform the diagnosis. The sensor data is expected to be combined with these internal representations to produce the diagnosis result. In spite of the availability of various diagnosis technologies, there have been only minimal efforts to develop a standardized software framework to run, evaluate, and compare different diagnosis technologies on the same system. This paper presents a framework that defines a standardized representation of the system knowledge, the sensor data, and the form of the diagnosis results and provides a run-time architecture that can execute diagnosis algorithms, send sensor data to the algorithms at appropriate time steps from a variety of sources (including the actual physical system), and collect resulting diagnoses. We also define a set of metrics that can be used to evaluate and compare the performance of the algorithms, and provide software to calculate the metrics.

Has computational creativity successfully made it "Beyond the Fence" in musical theatre?

NASA Astrophysics Data System (ADS)

Jordanous, Anna

2017-10-01

A significant test for software is to task it with replicating human performance, as done recently with creative software and the commercial project Beyond the Fence (undertaken for a television documentary Computer Says Show). The remit of this project was to use computer software as much as possible to produce "the world's first computer-generated musical". Several creative systems were used to generate this musical, which was performed in London's West End in 2016. This paper considers the challenge of evaluating this project. Current computational creativity evaluation methods are ill-suited to evaluating projects that involve creative input from multiple systems and people. Following recent inspiration within computational creativity research from interaction design, here the DECIDE evaluation framework is applied to evaluate the Beyond the Fence project. Evaluation finds that the project was reasonably successful at achieving the task of using computational generation to produce a credible musical. Lessons have been learned for future computational creativity projects though, particularly for affording creative software more agency and enabling software to interact with other creative partners. Upon reflection, the DECIDE framework emerges as a useful evaluation "checklist" (if not a tangible operational methodology) for evaluating multiple creative systems participating in a creative task.

Small drinking water systems under spatiotemporal water quality variability: a risk-based performance benchmarking framework.

PubMed

Bereskie, Ty; Haider, Husnain; Rodriguez, Manuel J; Sadiq, Rehan

2017-08-23

Traditional approaches for benchmarking drinking water systems are binary, based solely on the compliance and/or non-compliance of one or more water quality performance indicators against defined regulatory guidelines/standards. The consequence of water quality failure is dependent on location within a water supply system as well as time of the year (i.e., season) with varying levels of water consumption. Conventional approaches used for water quality comparison purposes fail to incorporate spatiotemporal variability and degrees of compliance and/or non-compliance. This can lead to misleading or inaccurate performance assessment data used in the performance benchmarking process. In this research, a hierarchical risk-based water quality performance benchmarking framework is proposed to evaluate small drinking water systems (SDWSs) through cross-comparison amongst similar systems. The proposed framework (R WQI framework) is designed to quantify consequence associated with seasonal and location-specific water quality issues in a given drinking water supply system to facilitate more efficient decision-making for SDWSs striving for continuous performance improvement. Fuzzy rule-based modelling is used to address imprecision associated with measuring performance based on singular water quality guidelines/standards and the uncertainties present in SDWS operations and monitoring. This proposed R WQI framework has been demonstrated using data collected from 16 SDWSs in Newfoundland and Labrador and Quebec, Canada, and compared to the Canadian Council of Ministers of the Environment WQI, a traditional, guidelines/standard-based approach. The study found that the R WQI framework provides an in-depth state of water quality and benchmarks SDWSs more rationally based on the frequency of occurrence and consequence of failure events.

Real-time long term measurement using integrated framework for ubiquitous smart monitoring

NASA Astrophysics Data System (ADS)

Heo, Gwanghee; Lee, Giu; Lee, Woosang; Jeon, Joonryong; Kim, Pil-Joong

2007-04-01

Ubiquitous monitoring combining internet technologies and wireless communication is one of the most promising technologies of infrastructure health monitoring against the natural of man-made hazards. In this paper, an integrated framework of the ubiquitous monitoring is developed for real-time long term measurement in internet environment. This framework develops a wireless sensor system based on Bluetooth technology and sends measured acceleration data to the host computer through TCP/IP protocol. And it is also designed to respond to the request of web user on real time basis. In order to verify this system, real time monitoring tests are carried out on a prototype self-anchored suspension bridge. Also, wireless measurement system is analyzed to estimate its sensing capacity and evaluate its performance for monitoring purpose. Based on the evaluation, this paper proposes the effective strategies for integrated framework in order to detect structural deficiencies and to design an early warning system.

Improving alignment in Tract-based spatial statistics: evaluation and optimization of image registration.

PubMed

de Groot, Marius; Vernooij, Meike W; Klein, Stefan; Ikram, M Arfan; Vos, Frans M; Smith, Stephen M; Niessen, Wiro J; Andersson, Jesper L R

2013-08-01

Anatomical alignment in neuroimaging studies is of such importance that considerable effort is put into improving the registration used to establish spatial correspondence. Tract-based spatial statistics (TBSS) is a popular method for comparing diffusion characteristics across subjects. TBSS establishes spatial correspondence using a combination of nonlinear registration and a "skeleton projection" that may break topological consistency of the transformed brain images. We therefore investigated feasibility of replacing the two-stage registration-projection procedure in TBSS with a single, regularized, high-dimensional registration. To optimize registration parameters and to evaluate registration performance in diffusion MRI, we designed an evaluation framework that uses native space probabilistic tractography for 23 white matter tracts, and quantifies tract similarity across subjects in standard space. We optimized parameters for two registration algorithms on two diffusion datasets of different quality. We investigated reproducibility of the evaluation framework, and of the optimized registration algorithms. Next, we compared registration performance of the regularized registration methods and TBSS. Finally, feasibility and effect of incorporating the improved registration in TBSS were evaluated in an example study. The evaluation framework was highly reproducible for both algorithms (R(2) 0.993; 0.931). The optimal registration parameters depended on the quality of the dataset in a graded and predictable manner. At optimal parameters, both algorithms outperformed the registration of TBSS, showing feasibility of adopting such approaches in TBSS. This was further confirmed in the example experiment. Copyright © 2013 Elsevier Inc. All rights reserved.

A collaborative computing framework of cloud network and WBSN applied to fall detection and 3-D motion reconstruction.

PubMed

Lai, Chin-Feng; Chen, Min; Pan, Jeng-Shyang; Youn, Chan-Hyun; Chao, Han-Chieh

2014-03-01

As cloud computing and wireless body sensor network technologies become gradually developed, ubiquitous healthcare services prevent accidents instantly and effectively, as well as provides relevant information to reduce related processing time and cost. This study proposes a co-processing intermediary framework integrated cloud and wireless body sensor networks, which is mainly applied to fall detection and 3-D motion reconstruction. In this study, the main focuses includes distributed computing and resource allocation of processing sensing data over the computing architecture, network conditions and performance evaluation. Through this framework, the transmissions and computing time of sensing data are reduced to enhance overall performance for the services of fall events detection and 3-D motion reconstruction.

Perceived Utility of the RE-AIM Framework for Health Promotion/Disease Prevention Initiatives for Older Adults: A Case Study from the U.S. Evidence-Based Disease Prevention Initiative

PubMed Central

Ory, Marcia G.; Altpeter, Mary; Belza, Basia; Helduser, Janet; Zhang, Chen; Smith, Matthew Lee

2015-01-01

Dissemination and implementation (D&I) frameworks are increasingly being promoted in public health research. However, less is known about their uptake in the field, especially for diverse sets of programs. Limited questionnaires exist to assess the ways that frameworks can be utilized in program planning and evaluation. We present a case study from the United States that describes the implementation of the RE-AIM framework by state aging services providers and public health partners and a questionnaire that can be used to assess the utility of such frameworks in practice. An online questionnaire was developed to capture community perspectives about the utility of the RE-AIM framework. Distributed to project leads in 27 funded states in an evidence-based disease prevention initiative for older adults, 40 key stakeholders responded representing a 100% state-participation rate among the 27 funded states. Findings suggest that there is perceived utility in using the RE-AIM framework when evaluating grand-scale initiatives for older adults. The RE-AIM framework was seen as useful for planning, implementation, and evaluation with relevance for evaluators, providers, community leaders, and policy makers. Yet, the uptake was not universal, and some respondents reported difficulties in use, especially adopting the framework as a whole. This questionnaire can serve as the basis to assess ways the RE-AIM framework can be utilized by practitioners in state-wide D&I efforts. Maximal benefit can be derived from examining the assessment of RE-AIM-related knowledge and confidence as part of a continual quality assurance process. We recommend such an assessment be performed before the implementation of new funding initiatives and throughout their course to assess RE-AIM uptake and to identify areas for technical assistance. PMID:25964897

Automated robot-assisted surgical skill evaluation: Predictive analytics approach.

PubMed

Fard, Mahtab J; Ameri, Sattar; Darin Ellis, R; Chinnam, Ratna B; Pandya, Abhilash K; Klein, Michael D

2018-02-01

Surgical skill assessment has predominantly been a subjective task. Recently, technological advances such as robot-assisted surgery have created great opportunities for objective surgical evaluation. In this paper, we introduce a predictive framework for objective skill assessment based on movement trajectory data. Our aim is to build a classification framework to automatically evaluate the performance of surgeons with different levels of expertise. Eight global movement features are extracted from movement trajectory data captured by a da Vinci robot for surgeons with two levels of expertise - novice and expert. Three classification methods - k-nearest neighbours, logistic regression and support vector machines - are applied. The result shows that the proposed framework can classify surgeons' expertise as novice or expert with an accuracy of 82.3% for knot tying and 89.9% for a suturing task. This study demonstrates and evaluates the ability of machine learning methods to automatically classify expert and novice surgeons using global movement features. Copyright © 2017 John Wiley & Sons, Ltd.

PCTFPeval: a web tool for benchmarking newly developed algorithms for predicting cooperative transcription factor pairs in yeast.

PubMed

Lai, Fu-Jou; Chang, Hong-Tsun; Wu, Wei-Sheng

2015-01-01

Computational identification of cooperative transcription factor (TF) pairs helps understand the combinatorial regulation of gene expression in eukaryotic cells. Many advanced algorithms have been proposed to predict cooperative TF pairs in yeast. However, it is still difficult to conduct a comprehensive and objective performance comparison of different algorithms because of lacking sufficient performance indices and adequate overall performance scores. To solve this problem, in our previous study (published in BMC Systems Biology 2014), we adopted/proposed eight performance indices and designed two overall performance scores to compare the performance of 14 existing algorithms for predicting cooperative TF pairs in yeast. Most importantly, our performance comparison framework can be applied to comprehensively and objectively evaluate the performance of a newly developed algorithm. However, to use our framework, researchers have to put a lot of effort to construct it first. To save researchers time and effort, here we develop a web tool to implement our performance comparison framework, featuring fast data processing, a comprehensive performance comparison and an easy-to-use web interface. The developed tool is called PCTFPeval (Predicted Cooperative TF Pair evaluator), written in PHP and Python programming languages. The friendly web interface allows users to input a list of predicted cooperative TF pairs from their algorithm and select (i) the compared algorithms among the 15 existing algorithms, (ii) the performance indices among the eight existing indices, and (iii) the overall performance scores from two possible choices. The comprehensive performance comparison results are then generated in tens of seconds and shown as both bar charts and tables. The original comparison results of each compared algorithm and each selected performance index can be downloaded as text files for further analyses. Allowing users to select eight existing performance indices and 15 existing algorithms for comparison, our web tool benefits researchers who are eager to comprehensively and objectively evaluate the performance of their newly developed algorithm. Thus, our tool greatly expedites the progress in the research of computational identification of cooperative TF pairs.

PCTFPeval: a web tool for benchmarking newly developed algorithms for predicting cooperative transcription factor pairs in yeast

PubMed Central

2015-01-01

Background Computational identification of cooperative transcription factor (TF) pairs helps understand the combinatorial regulation of gene expression in eukaryotic cells. Many advanced algorithms have been proposed to predict cooperative TF pairs in yeast. However, it is still difficult to conduct a comprehensive and objective performance comparison of different algorithms because of lacking sufficient performance indices and adequate overall performance scores. To solve this problem, in our previous study (published in BMC Systems Biology 2014), we adopted/proposed eight performance indices and designed two overall performance scores to compare the performance of 14 existing algorithms for predicting cooperative TF pairs in yeast. Most importantly, our performance comparison framework can be applied to comprehensively and objectively evaluate the performance of a newly developed algorithm. However, to use our framework, researchers have to put a lot of effort to construct it first. To save researchers time and effort, here we develop a web tool to implement our performance comparison framework, featuring fast data processing, a comprehensive performance comparison and an easy-to-use web interface. Results The developed tool is called PCTFPeval (Predicted Cooperative TF Pair evaluator), written in PHP and Python programming languages. The friendly web interface allows users to input a list of predicted cooperative TF pairs from their algorithm and select (i) the compared algorithms among the 15 existing algorithms, (ii) the performance indices among the eight existing indices, and (iii) the overall performance scores from two possible choices. The comprehensive performance comparison results are then generated in tens of seconds and shown as both bar charts and tables. The original comparison results of each compared algorithm and each selected performance index can be downloaded as text files for further analyses. Conclusions Allowing users to select eight existing performance indices and 15 existing algorithms for comparison, our web tool benefits researchers who are eager to comprehensively and objectively evaluate the performance of their newly developed algorithm. Thus, our tool greatly expedites the progress in the research of computational identification of cooperative TF pairs. PMID:26677932

Performance measurement: integrating quality management and activity-based cost management.

PubMed

McKeon, T

1996-04-01

The development of an activity-based management system provides a framework for developing performance measures integral to quality and cost management. Performance measures that cross operational boundaries and embrace core processes provide a mechanism to evaluate operational results related to strategic intention and internal and external customers. The author discusses this measurement process that allows managers to evaluate where they are and where they want to be, and to set a course of action that closes the gap between the two.

Feasibility and perceived benefits of a framework for physician-parent follow-up meetings after a child's death in the PICU.

PubMed

Meert, Kathleen L; Eggly, Susan; Berg, Robert A; Wessel, David L; Newth, Christopher J L; Shanley, Thomas P; Harrison, Rick; Dalton, Heidi; Clark, Amy E; Dean, J Michael; Doctor, Allan; Nicholson, Carol E

2014-01-01

To evaluate the feasibility and perceived benefits of conducting physician-parent follow-up meetings after a child's death in the PICU according to a framework developed by the Collaborative Pediatric Critical Care Research Network. Prospective observational study. Seven Collaborative Pediatric Critical Care Research Network-affiliated children's hospitals. Critical care attending physicians, bereaved parents, and meeting guests (i.e., parent support persons, other health professionals). Physician-parent follow-up meetings using the Collaborative Pediatric Critical Care Research Network framework. Forty-six critical care physicians were trained to conduct follow-up meetings using the framework. All meetings were video recorded. Videos were evaluated for the presence or absence of physician behaviors consistent with the framework. Present behaviors were evaluated for performance quality using a 5-point scale (1 = low, 5 = high). Participants completed meeting evaluation surveys. Parents of 194 deceased children were mailed an invitation to a follow-up meeting. Of these, one or both parents from 39 families (20%) agreed to participate, 80 (41%) refused, and 75 (39%) could not be contacted. Of 39 who initially agreed, three meetings were canceled due to conflicting schedules. Thirty-six meetings were conducted including 54 bereaved parents, 17 parent support persons, 23 critical care physicians, and 47 other health professionals. Physician adherence to the framework was high; 79% of behaviors consistent with the framework were rated as present with a quality score of 4.3 ± 0.2. Of 50 evaluation surveys completed by parents, 46 (92%) agreed or strongly agreed the meeting was helpful to them and 40 (89%) to others they brought with them. Of 36 evaluation surveys completed by critical care physicians (i.e., one per meeting), 33 (92%) agreed or strongly agreed the meeting was beneficial to parents and 31 (89%) to them. Follow-up meetings using the Collaborative Pediatric Critical Care Research Network framework are feasible and viewed as beneficial by meeting participants. Future research should evaluate the effects of follow-up meetings on bereaved parents' health outcomes.

From Determinism and Probability to Chaos: Chaotic Evolution towards Philosophy and Methodology of Chaotic Optimization

PubMed Central

2015-01-01

We present and discuss philosophy and methodology of chaotic evolution that is theoretically supported by chaos theory. We introduce four chaotic systems, that is, logistic map, tent map, Gaussian map, and Hénon map, in a well-designed chaotic evolution algorithm framework to implement several chaotic evolution (CE) algorithms. By comparing our previous proposed CE algorithm with logistic map and two canonical differential evolution (DE) algorithms, we analyse and discuss optimization performance of CE algorithm. An investigation on the relationship between optimization capability of CE algorithm and distribution characteristic of chaotic system is conducted and analysed. From evaluation result, we find that distribution of chaotic system is an essential factor to influence optimization performance of CE algorithm. We propose a new interactive EC (IEC) algorithm, interactive chaotic evolution (ICE) that replaces fitness function with a real human in CE algorithm framework. There is a paired comparison-based mechanism behind CE search scheme in nature. A simulation experimental evaluation is conducted with a pseudo-IEC user to evaluate our proposed ICE algorithm. The evaluation result indicates that ICE algorithm can obtain a significant better performance than or the same performance as interactive DE. Some open topics on CE, ICE, fusion of these optimization techniques, algorithmic notation, and others are presented and discussed. PMID:25879067

From determinism and probability to chaos: chaotic evolution towards philosophy and methodology of chaotic optimization.

PubMed

Pei, Yan

2015-01-01

We present and discuss philosophy and methodology of chaotic evolution that is theoretically supported by chaos theory. We introduce four chaotic systems, that is, logistic map, tent map, Gaussian map, and Hénon map, in a well-designed chaotic evolution algorithm framework to implement several chaotic evolution (CE) algorithms. By comparing our previous proposed CE algorithm with logistic map and two canonical differential evolution (DE) algorithms, we analyse and discuss optimization performance of CE algorithm. An investigation on the relationship between optimization capability of CE algorithm and distribution characteristic of chaotic system is conducted and analysed. From evaluation result, we find that distribution of chaotic system is an essential factor to influence optimization performance of CE algorithm. We propose a new interactive EC (IEC) algorithm, interactive chaotic evolution (ICE) that replaces fitness function with a real human in CE algorithm framework. There is a paired comparison-based mechanism behind CE search scheme in nature. A simulation experimental evaluation is conducted with a pseudo-IEC user to evaluate our proposed ICE algorithm. The evaluation result indicates that ICE algorithm can obtain a significant better performance than or the same performance as interactive DE. Some open topics on CE, ICE, fusion of these optimization techniques, algorithmic notation, and others are presented and discussed.

From expert-derived user needs to user-perceived ease of use and usefulness: a two-phase mixed-methods evaluation framework.

PubMed

Boland, Mary Regina; Rusanov, Alexander; So, Yat; Lopez-Jimenez, Carlos; Busacca, Linda; Steinman, Richard C; Bakken, Suzanne; Bigger, J Thomas; Weng, Chunhua

2014-12-01

Underspecified user needs and frequent lack of a gold standard reference are typical barriers to technology evaluation. To address this problem, this paper presents a two-phase evaluation framework involving usability experts (phase 1) and end-users (phase 2). In phase 1, a cross-system functionality alignment between expert-derived user needs and system functions was performed to inform the choice of "the best available" comparison system to enable a cognitive walkthrough in phase 1 and a comparative effectiveness evaluation in phase 2. During phase 2, five quantitative and qualitative evaluation methods are mixed to assess usability: time-motion analysis, software log, questionnaires - System Usability Scale and the Unified Theory of Acceptance of Use of Technology, think-aloud protocols, and unstructured interviews. Each method contributes data for a unique measure (e.g., time motion analysis contributes task-completion-time; software log contributes action transition frequency). The measures are triangulated to yield complementary insights regarding user-perceived ease-of-use, functionality integration, anxiety during use, and workflow impact. To illustrate its use, we applied this framework in a formative evaluation of a software called Integrated Model for Patient Care and Clinical Trials (IMPACT). We conclude that this mixed-methods evaluation framework enables an integrated assessment of user needs satisfaction and user-perceived usefulness and usability of a novel design. This evaluation framework effectively bridges the gap between co-evolving user needs and technology designs during iterative prototyping and is particularly useful when it is difficult for users to articulate their needs for technology support due to the lack of a baseline. Copyright © 2013 Elsevier Inc. All rights reserved.

Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation

PubMed Central

2013-01-01

Background Performance-based financing is increasingly being applied in a variety of contexts, with the expectation that it can improve the performance of health systems. However, while there is a growing literature on implementation issues and effects on outputs, there has been relatively little focus on interactions between PBF and health systems and how these should be studied. This paper aims to contribute to filling that gap by developing a framework for assessing the interactions between PBF and health systems, focusing on low and middle income countries. In doing so, it elaborates a general framework for monitoring and evaluating health system reforms in general. Methods This paper is based on an exploratory literature review and on the work of a group of academics and PBF practitioners. The group developed ideas for the monitoring and evaluation framework through exchange of emails and working documents. Ideas were further refined through discussion at the Health Systems Research symposium in Beijing in October 2012, through comments from members of the online PBF Community of Practice and Beijing participants, and through discussion with PBF experts in Bergen in June 2013. Results The paper starts with a discussion of definitions, to clarify the core concept of PBF and how the different terms are used. It then develops a framework for monitoring its interactions with the health system, structured around five domains of context, the development process, design, implementation and effects. Some of the key questions for monitoring and evaluation are highlighted, and a systematic approach to monitoring effects proposed, structured according to the health system pillars, but also according to inputs, processes and outputs. Conclusions The paper lays out a broad framework within which indicators can be prioritised for monitoring and evaluation of PBF or other health system reforms. It highlights the dynamic linkages between the domains and the different pillars. All of these are also framed within inter-sectoral and wider societal contexts. It highlights the importance of differentiating short term and long term effects, and also effects (intended and unintended) at different levels of the health system, and for different sectors and areas of the country. Outstanding work will include using and refining the framework and agreeing on the most important hypotheses to test using it, in relation to PBF but also other purchasing and provider payment reforms, as well as appropriate research methods to use for this task. PMID:24073625

Evaluating the evaluation of cancer driver genes

PubMed Central

Tokheim, Collin J.; Papadopoulos, Nickolas; Kinzler, Kenneth W.; Vogelstein, Bert; Karchin, Rachel

2016-01-01

Sequencing has identified millions of somatic mutations in human cancers, but distinguishing cancer driver genes remains a major challenge. Numerous methods have been developed to identify driver genes, but evaluation of the performance of these methods is hindered by the lack of a gold standard, that is, bona fide driver gene mutations. Here, we establish an evaluation framework that can be applied to driver gene prediction methods. We used this framework to compare the performance of eight such methods. One of these methods, described here, incorporated a machine-learning–based ratiometric approach. We show that the driver genes predicted by each of the eight methods vary widely. Moreover, the P values reported by several of the methods were inconsistent with the uniform values expected, thus calling into question the assumptions that were used to generate them. Finally, we evaluated the potential effects of unexplained variability in mutation rates on false-positive driver gene predictions. Our analysis points to the strengths and weaknesses of each of the currently available methods and offers guidance for improving them in the future. PMID:27911828

Macrostructure-dependent photocatalytic property of high-surface-area porous titania films

NASA Astrophysics Data System (ADS)

Kimura, T.

2014-11-01

Porous titania films with different macrostructures were prepared with precise control of condensation degree and density of the oxide frameworks in the presence of spherical aggregates of polystyrene-block-poly(oxyethylene) (PS-b-PEO) diblock copolymer. Following detailed explanation of the formation mechanisms of three (reticular, spherical, and large spherical) macrostructures by the colloidal PS-b-PEO templating, structural variation of the titania frameworks during calcination were investigated by X-ray diffraction and X-ray photoelectron spectroscopy. Then, photocatalytic performance of the macroporous titania films was evaluated through simple degradation experiments of methylene blue under an UV irradiation. Consequently, absolute surface area of the film and crystallinity of the titania frameworks were important for understanding the photocatalytic performance, but the catalytic performance can be improved further by the macrostructural design that controls diffusivity of the targeted molecules inside the film and their accessibility to active sites.

Evaluating Environmental Effects of Dredged Material Management Alternatives - A Technical Framework. Revision

DTIC Science & Technology

2004-05-01

Significant Impact HELPQ - Hydrologic Evaluation of Leachate Production and Quality HELP - Hydrologic Evaluation of Landfill Performance LDC...BIOASSAY SURFACE RUNOFF CONTROLS • PONDING • TREATMENT • OTHERS (5.3.6) AND EVALUATE AND /OR GROUNDWATER /OR LEACHATE • • • LEACHATE TESTING... LEACHATE CONTROLS COVERS LINERS TREATMENT PLANT BIOASSAY PLANT UPTAKE CONTROLS • COVERS • SELECTIVE VEGETATION (5.1) (5.2) (5.3

Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography.

PubMed

Kirişli, H A; Schaap, M; Metz, C T; Dharampal, A S; Meijboom, W B; Papadopoulou, S L; Dedic, A; Nieman, K; de Graaf, M A; Meijs, M F L; Cramer, M J; Broersen, A; Cetin, S; Eslami, A; Flórez-Valencia, L; Lor, K L; Matuszewski, B; Melki, I; Mohr, B; Oksüz, I; Shahzad, R; Wang, C; Kitslaar, P H; Unal, G; Katouzian, A; Örkisz, M; Chen, C M; Precioso, F; Najman, L; Masood, S; Ünay, D; van Vliet, L; Moreno, R; Goldenberg, R; Vuçini, E; Krestin, G P; Niessen, W J; van Walsum, T

2013-12-01

Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CTA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CTA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (1) (semi-)automatically detect and quantify stenosis on CTA, in comparison with quantitative coronary angiography (QCA) and CTA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CTA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CTA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at http://coronary.bigr.nl/stenoses/. Copyright © 2013 Elsevier B.V. All rights reserved.

The development of a competency framework for pharmacists providing cancer services.

PubMed

Carrington, Christine; Weir, Janet; Smith, Peter

2011-09-01

Health practitioners should possess relevant, up to date skills and be able to perform within their required scope of practice to ensure that they are competent. Maintaining the competency of health care professionals is a key principle of clinical governance and risk management. The aim of this project was to develop a competency framework for pharmacists providing pharmaceutical care to cancer patients. An initial draft framework was developed based on existing documentation and adapted to the needs of Queensland Health (QH) facilities. Pharmacists in QH and interstate were asked to review the framework for content and applicability. Cancer care pharmacists in QH were invited to evaluate and score the usefulness and relevance of the final framework. The framework consists of competency clusters, which describe core activities within three areas: patient care competencies, knowledge competencies, and advanced level competencies. The characteristics of the levels of practice at foundation, advanced, and consultant are defined. Twelve pharmacists evaluated the framework by self-assessing their own practice. Respondents reported that the framework was very to somewhat reflective of what they usually do and gave overall support for the content and applicability to practice. The framework has been developed using national and international documents and the input of experienced practitioners across Australia. It represents a set of key competencies for the pharmaceutical delivery of cancer care. The next essential step of the competency framework is to implement and integrate the framework into practice and to develop accompanying training tools.

Addressing social aspects associated with wastewater treatment facilities

DOE Office of Scientific and Technical Information (OSTI.GOV)

Padilla-Rivera, Alejandro; Morgan-Sagastume, Juan Manuel; Noyola, Adalberto

In wastewater treatment facilities (WWTF), technical and financial aspects have been considered a priority, while other issues, such as social aspects, have not been evaluated seriously and there is not an accepted methodology for assessing it. In this work, a methodology focused on social concerns related to WWTF is presented. The methodology proposes the use of 25 indicators as a framework for measuring social performance to evaluate the progress in moving towards sustainability. The methodology was applied to test its applicability and effectiveness in two WWTF in Mexico (urban and rural). This evaluation helped define the key elements, stakeholders andmore » barriers in the facilities. In this context, the urban facility showed a better overall performance, a result that may be explained mainly by the better socioeconomic context of the urban municipality. Finally, the evaluation of social aspects using the semi-qualitative approach proposed in this work allows for a comparison between different facilities and for the identification of strengths and weakness, and it provides an alternative tool for achieving and improving wastewater management. - Highlights: • The methodology proposes 25 indicators as a framework for measuring social performance in wastewater treatment facilities. • The evaluation helped to define the key elements, stakeholders and barriers in the wastewater treatment facilities. • The evaluation of social aspects allows the identification of strengths and weakness for improving wastewater management. • It provides a social profile of the facility that highlights the best and worst performances.« less

Effects of donor proliferation in development aid for health on health program performance: A conceptual framework.

PubMed

Pallas, Sarah Wood; Ruger, Jennifer Prah

2017-02-01

Development aid for health increased dramatically during the past two decades, raising concerns about inefficiency and lack of coherence among the growing number of global health donors. However, we lack a framework for how donor proliferation affects health program performance to inform theory-based evaluation of aid effectiveness policies. A review of academic and gray literature was conducted. Data were extracted from the literature sample on study design and evidence for hypothesized effects of donor proliferation on health program performance, which were iteratively grouped into categories and mapped into a new conceptual framework. In the framework, increases in the number of donors are hypothesized to increase inter-donor competition, transaction costs, donor poaching of recipient staff, recipient control over aid, and donor fragmentation, and to decrease donors' sense of accountability for overall development outcomes. There is mixed evidence on whether donor proliferation increases or decreases aid volume. These primary effects in turn affect donor innovation, information hoarding, and aid disbursement volatility, as well as recipient country health budget levels, human resource capacity, and corruption, and the determinants of health program performance. The net effect of donor proliferation on health will vary depending on the magnitude of the framework's competing effects in specific country settings. The conceptual framework provides a foundation for improving design of aid effectiveness practices to mitigate negative effects from donor proliferation while preserving its potential benefits. Copyright © 2017 Elsevier Ltd. All rights reserved.

Ensemble approach for differentiation of malignant melanoma

NASA Astrophysics Data System (ADS)

Rastgoo, Mojdeh; Morel, Olivier; Marzani, Franck; Garcia, Rafael

2015-04-01

Melanoma is the deadliest type of skin cancer, yet it is the most treatable kind depending on its early diagnosis. The early prognosis of melanoma is a challenging task for both clinicians and dermatologists. Due to the importance of early diagnosis and in order to assist the dermatologists, we propose an automated framework based on ensemble learning methods and dermoscopy images to differentiate melanoma from dysplastic and benign lesions. The evaluation of our framework on the recent and public dermoscopy benchmark (PH2 dataset) indicates the potential of proposed method. Our evaluation, using only global features, revealed that ensembles such as random forest perform better than single learner. Using random forest ensemble and combination of color and texture features, our framework achieved the highest sensitivity of 94% and specificity of 92%.

Faculty verbal evaluations reveal strategies used to promote medical student performance

PubMed Central

Hauer, Karen E.; Mazotti, Lindsay; O'Brien, Bridget; Hemmer, Paul A.; Tong, Lowell

2011-01-01

Background Preceptors rarely follow medical students' developing clinical performance over time and across disciplines. This study analyzes preceptors' descriptions of longitudinal integrated clerkship (LIC) students' clinical development and their identification of strategies to guide students' progress. Methods We used a common evaluation framework, reporter-interpreter-manager-educator, to guide multidisciplinary LIC preceptors' discussions of students' progress. We conducted thematic analysis of transcripts from preceptors' (seven longitudinal ambulatory preceptors per student) quarterly group discussions of 15 students' performance over one year. Results All students' clinical development progressed, although most experienced obstacles. Lack of structure in the history and physical exam commonly obstructed progression. Preceptors used templates for data gathering, and modeling or experiences in the inpatient setting to provide time and solidify structure. To advance students' knowledge acquisition, many preceptors identified focused learning topics with their students; to promote application of knowledge, preceptors used reasoning strategies to teach the steps involved in synthesizing clinical data. Preceptors shared accountability for helping students advance as the LIC allowed them to follow students' response to teaching strategies. Discussion These results depict preceptors' perceptions of LIC students' developmental continuum and illustrate how multidisciplinary preceptors can use a common evaluation framework to identify strategies to improve performance and follow students' performance longitudinally. PMID:21629669

Field-widened Michelson interferometer for spectral discrimination in high-spectral-resolution lidar: theoretical framework.

PubMed

Cheng, Zhongtao; Liu, Dong; Luo, Jing; Yang, Yongying; Zhou, Yudi; Zhang, Yupeng; Duan, Lulin; Su, Lin; Yang, Liming; Shen, Yibing; Wang, Kaiwei; Bai, Jian

2015-05-04

A field-widened Michelson interferometer (FWMI) is developed to act as the spectral discriminator in high-spectral-resolution lidar (HSRL). This realization is motivated by the wide-angle Michelson interferometer (WAMI) which has been used broadly in the atmospheric wind and temperature detection. This paper describes an independent theoretical framework about the application of the FWMI in HSRL for the first time. In the framework, the operation principles and application requirements of the FWMI are discussed in comparison with that of the WAMI. Theoretical foundations for designing this type of interferometer are introduced based on these comparisons. Moreover, a general performance estimation model for the FWMI is established, which can provide common guidelines for the performance budget and evaluation of the FWMI in the both design and operation stages. Examples incorporating many practical imperfections or conditions that may degrade the performance of the FWMI are given to illustrate the implementation of the modeling. This theoretical framework presents a complete and powerful tool for solving most of theoretical or engineering problems encountered in the FWMI application, including the designing, parameter calibration, prior performance budget, posterior performance estimation, and so on. It will be a valuable contribution to the lidar community to develop a new generation of HSRLs based on the FWMI spectroscopic filter.

Evaluation of Course-Specific Self-Efficacy Assessment Methods.

ERIC Educational Resources Information Center

Bong, Mimi

A study was conducted to compare three methods of assessing course-level self-efficacy beliefs within a multitrait multimethod (MTMM) framework. The methods involved: (1) successfully performing a number of domain-related tasks; (2) obtaining specific letter grades in the course; and (3) successfully performing generic academic tasks in the…

Impact of reconstruction strategies on system performance measures : maximizing safety and mobility while minimizing life-cycle costs : final report, December 8, 2008.

DOT National Transportation Integrated Search

2008-12-08

The objective of this research is to develop a general methodological framework for planning and : evaluating the effectiveness of highway reconstruction strategies on the systems performance : measures, in particular safety, mobility, and the tot...

Comparing the Performance of Indoor Localization Systems through the EvAAL Framework.

PubMed

Potortì, Francesco; Park, Sangjoon; Jiménez Ruiz, Antonio Ramón; Barsocchi, Paolo; Girolami, Michele; Crivello, Antonino; Lee, So Yeon; Lim, Jae Hyun; Torres-Sospedra, Joaquín; Seco, Fernando; Montoliu, Raul; Mendoza-Silva, Germán Martin; Pérez Rubio, Maria Del Carmen; Losada-Gutiérrez, Cristina; Espinosa, Felipe; Macias-Guarasa, Javier

2017-10-13

In recent years, indoor localization systems have been the object of significant research activity and of growing interest for their great expected social impact and their impressive business potential. Application areas include tracking and navigation, activity monitoring, personalized advertising, Active and Assisted Living (AAL), traceability, Internet of Things (IoT) networks, and Home-land Security. In spite of the numerous research advances and the great industrial interest, no canned solutions have yet been defined. The diversity and heterogeneity of applications, scenarios, sensor and user requirements, make it difficult to create uniform solutions. From that diverse reality, a main problem is derived that consists in the lack of a consensus both in terms of the metrics and the procedures used to measure the performance of the different indoor localization and navigation proposals. This paper introduces the general lines of the EvAAL benchmarking framework, which is aimed at a fair comparison of indoor positioning systems through a challenging competition under complex, realistic conditions. To evaluate the framework capabilities, we show how it was used in the 2016 Indoor Positioning and Indoor Navigation (IPIN) Competition. The 2016 IPIN competition considered three different scenario dimensions, with a variety of use cases: (1) pedestrian versus robotic navigation, (2) smartphones versus custom hardware usage and (3) real-time positioning versus off-line post-processing. A total of four competition tracks were evaluated under the same EvAAL benchmark framework in order to validate its potential to become a standard for evaluating indoor localization solutions. The experience gained during the competition and feedback from track organizers and competitors showed that the EvAAL framework is flexible enough to successfully fit the very different tracks and appears adequate to compare indoor positioning systems.

Comparing the Performance of Indoor Localization Systems through the EvAAL Framework

PubMed Central

2017-01-01

In recent years, indoor localization systems have been the object of significant research activity and of growing interest for their great expected social impact and their impressive business potential. Application areas include tracking and navigation, activity monitoring, personalized advertising, Active and Assisted Living (AAL), traceability, Internet of Things (IoT) networks, and Home-land Security. In spite of the numerous research advances and the great industrial interest, no canned solutions have yet been defined. The diversity and heterogeneity of applications, scenarios, sensor and user requirements, make it difficult to create uniform solutions. From that diverse reality, a main problem is derived that consists in the lack of a consensus both in terms of the metrics and the procedures used to measure the performance of the different indoor localization and navigation proposals. This paper introduces the general lines of the EvAAL benchmarking framework, which is aimed at a fair comparison of indoor positioning systems through a challenging competition under complex, realistic conditions. To evaluate the framework capabilities, we show how it was used in the 2016 Indoor Positioning and Indoor Navigation (IPIN) Competition. The 2016 IPIN competition considered three different scenario dimensions, with a variety of use cases: (1) pedestrian versus robotic navigation, (2) smartphones versus custom hardware usage and (3) real-time positioning versus off-line post-processing. A total of four competition tracks were evaluated under the same EvAAL benchmark framework in order to validate its potential to become a standard for evaluating indoor localization solutions. The experience gained during the competition and feedback from track organizers and competitors showed that the EvAAL framework is flexible enough to successfully fit the very different tracks and appears adequate to compare indoor positioning systems. PMID:29027948

Secure and Efficient Regression Analysis Using a Hybrid Cryptographic Framework: Development and Evaluation

PubMed Central

Jiang, Xiaoqian; Aziz, Md Momin Al; Wang, Shuang; Mohammed, Noman

2018-01-01

Background Machine learning is an effective data-driven tool that is being widely used to extract valuable patterns and insights from data. Specifically, predictive machine learning models are very important in health care for clinical data analysis. The machine learning algorithms that generate predictive models often require pooling data from different sources to discover statistical patterns or correlations among different attributes of the input data. The primary challenge is to fulfill one major objective: preserving the privacy of individuals while discovering knowledge from data. Objective Our objective was to develop a hybrid cryptographic framework for performing regression analysis over distributed data in a secure and efficient way. Methods Existing secure computation schemes are not suitable for processing the large-scale data that are used in cutting-edge machine learning applications. We designed, developed, and evaluated a hybrid cryptographic framework, which can securely perform regression analysis, a fundamental machine learning algorithm using somewhat homomorphic encryption and a newly introduced secure hardware component of Intel Software Guard Extensions (Intel SGX) to ensure both privacy and efficiency at the same time. Results Experimental results demonstrate that our proposed method provides a better trade-off in terms of security and efficiency than solely secure hardware-based methods. Besides, there is no approximation error. Computed model parameters are exactly similar to plaintext results. Conclusions To the best of our knowledge, this kind of secure computation model using a hybrid cryptographic framework, which leverages both somewhat homomorphic encryption and Intel SGX, is not proposed or evaluated to this date. Our proposed framework ensures data security and computational efficiency at the same time. PMID:29506966

Secure and Efficient Regression Analysis Using a Hybrid Cryptographic Framework: Development and Evaluation.

PubMed

Sadat, Md Nazmus; Jiang, Xiaoqian; Aziz, Md Momin Al; Wang, Shuang; Mohammed, Noman

2018-03-05

Machine learning is an effective data-driven tool that is being widely used to extract valuable patterns and insights from data. Specifically, predictive machine learning models are very important in health care for clinical data analysis. The machine learning algorithms that generate predictive models often require pooling data from different sources to discover statistical patterns or correlations among different attributes of the input data. The primary challenge is to fulfill one major objective: preserving the privacy of individuals while discovering knowledge from data. Our objective was to develop a hybrid cryptographic framework for performing regression analysis over distributed data in a secure and efficient way. Existing secure computation schemes are not suitable for processing the large-scale data that are used in cutting-edge machine learning applications. We designed, developed, and evaluated a hybrid cryptographic framework, which can securely perform regression analysis, a fundamental machine learning algorithm using somewhat homomorphic encryption and a newly introduced secure hardware component of Intel Software Guard Extensions (Intel SGX) to ensure both privacy and efficiency at the same time. Experimental results demonstrate that our proposed method provides a better trade-off in terms of security and efficiency than solely secure hardware-based methods. Besides, there is no approximation error. Computed model parameters are exactly similar to plaintext results. To the best of our knowledge, this kind of secure computation model using a hybrid cryptographic framework, which leverages both somewhat homomorphic encryption and Intel SGX, is not proposed or evaluated to this date. Our proposed framework ensures data security and computational efficiency at the same time. ©Md Nazmus Sadat, Xiaoqian Jiang, Md Momin Al Aziz, Shuang Wang, Noman Mohammed. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 05.03.2018.

Application of Adverse Outcome Pathways to U.S. EPA’s Endocrine Disruptor Screening Program

PubMed Central

Noyes, Pamela D.; Casey, Warren M.; Dix, David J.

2017-01-01

Background: The U.S. EPA’s Endocrine Disruptor Screening Program (EDSP) screens and tests environmental chemicals for potential effects in estrogen, androgen, and thyroid hormone pathways, and it is one of the only regulatory programs designed around chemical mode of action. Objectives: This review describes the EDSP’s use of adverse outcome pathway (AOP) and toxicity pathway frameworks to organize and integrate diverse biological data for evaluating the endocrine activity of chemicals. Using these frameworks helps to establish biologically plausible links between endocrine mechanisms and apical responses when those end points are not measured in the same assay. Results: Pathway frameworks can facilitate a weight of evidence determination of a chemical’s potential endocrine activity, identify data gaps, aid study design, direct assay development, and guide testing strategies. Pathway frameworks also can be used to evaluate the performance of computational approaches as alternatives for low-throughput and animal-based assays and predict downstream key events. In cases where computational methods can be validated based on performance, they may be considered as alternatives to specific assays or end points. Conclusions: A variety of biological systems affect apical end points used in regulatory risk assessments, and without mechanistic data, an endocrine mode of action cannot be determined. Because the EDSP was designed to consider mode of action, toxicity pathway and AOP concepts are a natural fit. Pathway frameworks have diverse applications to endocrine screening and testing. An estrogen pathway example is presented, and similar approaches are being used to evaluate alternative methods and develop predictive models for androgen and thyroid pathways. https://doi.org/10.1289/EHP1304 PMID:28934726

Breaking barriers: a competency-based framework for promoting the integration of the pediatrician's education.

PubMed

Naghettini, Alessandra V; Bollela, Valdes R; Costa, Nilce M S C; Salgado, Luciana M R

2011-01-01

To describe the process of integration and revision of a pediatric program curriculum which resulted in the creation of a competency-based framework recommended in the Brazilian National Curricular Guidelines. Quali-quantitative analysis of an intervention evaluating the students and professors' perception of the pediatric program curriculum (focus groups and semi-structured interviews). Results were discussed during teaching development workshops. A competency-based framework was suggested for the pediatric program from the 3rd to the 6th year. The new curriculum was approved, implemented, and reevaluated six months later. Twelve students (12%) from the 3rd to the 6th year participated in the focus groups, and 11 professors (78.5%) answered the questionnaire. Most participants reported lack of integration among the courses, lack of knowledge about the learning goals of the internships, few opportunities of practice, and predominance of theoretical evaluation. In the training workshops, a competency-based curriculum was created after pediatrics and collective health professors reached an agreement. The new curriculum was focused on general competency, learning goals, opportunities available to learn these goals, and evaluation system. After six months, 93% (104/112) of students and 79% (11/14) of professors reported greater integration of the program and highlighted the inclusion of the clinical performance evaluation. The collective creation of a competency-based curriculum promoted higher satisfaction of students and professors. After being implemented, the new curriculum was considered to integrate the teaching practices and contents, improving the quality of the clinical performance evaluation.

Medical-device risk management and public safety: using cost-benefit as a measurement of effectiveness

NASA Astrophysics Data System (ADS)

Hughes, Allen A.

1994-12-01

Public safety can be enhanced through the development of a comprehensive medical device risk management. This can be accomplished through case studies using a framework that incorporates cost-benefit analysis in the evaluation of risk management attributes. This paper presents a framework for evaluating the risk management system for regulatory Class III medical devices. The framework consists of the following sixteen attributes of a comprehensive medical device risk management system: fault/failure analysis, premarket testing/clinical trials, post-approval studies, manufacturer sponsored hospital studies, product labeling, establishment inspections, problem reporting program, mandatory hospital reporting, medical literature surveillance, device/patient registries, device performance monitoring, returned product analysis, autopsy program, emergency treatment funds/interim compensation, product liability, and alternative compensation mechanisms. Review of performance histories for several medical devices can reveal the value of information for many attributes, and also the inter-dependencies of the attributes in generating risk information flow. Such an information flow network is presented as a starting point for enhancing medical device risk management by focusing on attributes with high net benefit values and potential to spur information dissemination.

A framework for performing workplace hazard and risk analysis: a participative ergonomics approach.

PubMed

Morag, Ido; Luria, Gil

2013-01-01

Despite the unanimity among researchers about the centrality of workplace analysis based on participatory ergonomics (PE) as a basis for preventive interventions, there is still little agreement about the necessary of a theoretical framework for providing practical guidance. In an effort to develop a conceptual PE framework, the authors, focusing on 20 studies, found five primary dimensions for characterising an analytical structure: (1) extent of workforce involvement; (2) analysis duration; (3) diversity of reporter role types; (4) scope of analysis and (5) supportive information system for analysis management. An ergonomics analysis carried out in a chemical manufacturing plant serves as a case study for evaluating the proposed framework. The study simultaneously demonstrates the five dimensions and evaluates their feasibility. The study showed that managerial leadership was fundamental to the successful implementation of the analysis; that all job holders should participate in analysing their own workplace and simplified reporting methods contributed to a desirable outcome. This paper seeks to clarify the scope of workplace ergonomics analysis by offering a theoretical and structured framework for providing practical advice and guidance. Essential to successfully implementing the analytical framework are managerial involvement, participation of all job holders and simplified reporting methods.

Open framework for objective evaluation of crater detection algorithms with first test-field subsystem based on MOLA data

NASA Astrophysics Data System (ADS)

Salamunićcar, G.; Lončarić, S.

2008-07-01

Crater Detection Algorithms (CDAs) applications range from estimation of lunar/planetary surface age to autonomous landing on planets and asteroids and advanced statistical analyses. A large amount of work on CDAs has already been published. However, problems arise when evaluation results of some new CDA have to be compared with already published evaluation results. The problem is that different authors use different test-fields, different Ground-Truth (GT) catalogues, and even different methodologies for evaluation of their CDAs. Re-implementation of already published CDAs or its evaluation environment is a time-consuming and unpractical solution to this problem. In addition, implementation details are often insufficiently described in publications. As a result, there is a need in research community to develop a framework for objective evaluation of CDAs. A scientific question is how CDAs should be evaluated so that the results are easily and reliably comparable. In attempt to solve this issue we first analyzed previously published work on CDAs. In this paper, we propose a framework for solution of the problem of objective CDA evaluation. The framework includes: (1) a definition of the measure for differences between craters; (2) test-field topography based on the 1/64° MOLA data; (3) the GT catalogue wherein each of 17,582 craters is aligned with MOLA data and confirmed with catalogues by N.G. Barlow et al. and J.F. Rodionova et al.; (4) selection of methodology for training and testing; and (5) a Free-response Receiver Operating Characteristics (F-ROC) curves as a way to measure CDA performance. The handling of possible improvements of the framework in the future is additionally addressed as a part of discussion of results. Possible extensions with additional test-field subsystems based on visual images, data sets for other planets, evaluation methodologies for CDAs developed for different purposes than cataloguing of craters, are proposed as well. The goal of the proposed framework is to contribute to the research community by establishing guidelines for objective evaluation of CDAs.

An Integrative Framework of Stress, Attention, and Visuomotor Performance

PubMed Central

Vine, Samuel J.; Moore, Lee J.; Wilson, Mark R.

2016-01-01

The aim of this article is to present an integrative conceptual framework that depicts the effect of acute stress on the performance of visually guided motor skills. We draw upon seminal theories highlighting the importance of subjective interpretations of stress on subsequent performance and outline how models of disrupted attentional control might explain this effect through impairments in visuomotor control. We first synthesize and critically discuss empirical support for theories examining these relationships in isolation. We then outline our integrative framework that seeks to provide a more complete picture of the interacting influences of stress responses (challenge and threat) and attention in explaining how elevated stress may lead to different visuomotor performance outcomes. We propose a number of mechanisms that explain why evaluations of stress are related to attentional control, and highlight the emotion of anxiety as the most likely candidate to explain why negative reactions to stress lead to disrupted attention and poor visuomotor skill performance. Finally, we propose a number of feedback loops that explain why stress responses are often self-perpetuating, as well as a number of proposed interventions that are designed to help improve or maintain performance in real world performance environments (e.g., sport, surgery, military, and aviation). PMID:27847484

A comparison of laser-welded titanium and conventional cast frameworks supported by implants in the partially edentulous jaw: a 3-year prospective multicenter study.

PubMed

Jemt, T; Henry, P; Lindén, B; Naert, I; Weber, H; Bergström, C

2000-01-01

The purpose of this prospective multicenter study was to evaluate and compare the clinical performance of laser-welded titanium fixed partial implant-supported prostheses with conventional cast frameworks. Forty-two partially edentulous patients were provided with Brånemark system implants and arranged into 2 groups. Group A was provided with a conventional cast framework with porcelain veneers in one side of the jaw and a laser-welded titanium framework with low-fusing porcelain on the other side. The patients in group B had an old implant prosthesis replaced by a titanium framework prosthesis. The patients were followed for 3 years after prosthesis placement. Clinical and radiographic data were collected and analyzed. Only one implant was lost, and all prostheses were still in function after 3 years. The 2 framework designs showed similar clinical performance with few clinical complications. Only one abutment screw (1%) and 9 porcelain tooth units (5%) fractured. Four prostheses experienced loose gold screws (6%). In group A, marginal bone loss was similar for both designs of prostheses, with a mean of 1.0 mm and 0.3 mm in the maxilla and mandible, respectively. No bone loss was observed on average in group B. No significant relationship (P > 0.05) was observed between marginal bone loss and placement of prosthesis margin or prosthesis design. The use of laser-welded titanium frameworks seems to present similar clinical performance to conventional cast frameworks in partial implant situations after 3 years.

FAST: framework for heterogeneous medical image computing and visualization.

PubMed

Smistad, Erik; Bozorgi, Mohammadmehdi; Lindseth, Frank

2015-11-01

Computer systems are becoming increasingly heterogeneous in the sense that they consist of different processors, such as multi-core CPUs and graphic processing units. As the amount of medical image data increases, it is crucial to exploit the computational power of these processors. However, this is currently difficult due to several factors, such as driver errors, processor differences, and the need for low-level memory handling. This paper presents a novel FrAmework for heterogeneouS medical image compuTing and visualization (FAST). The framework aims to make it easier to simultaneously process and visualize medical images efficiently on heterogeneous systems. FAST uses common image processing programming paradigms and hides the details of memory handling from the user, while enabling the use of all processors and cores on a system. The framework is open-source, cross-platform and available online. Code examples and performance measurements are presented to show the simplicity and efficiency of FAST. The results are compared to the insight toolkit (ITK) and the visualization toolkit (VTK) and show that the presented framework is faster with up to 20 times speedup on several common medical imaging algorithms. FAST enables efficient medical image computing and visualization on heterogeneous systems. Code examples and performance evaluations have demonstrated that the toolkit is both easy to use and performs better than existing frameworks, such as ITK and VTK.

New pathways in the evaluation of programmes for men who perpetrate violence against their female partners.

PubMed

Wojnicka, Katarzyna; Scambor, Christian; Kraus, Heinrich

2016-08-01

Today, evaluation research in the field of intervention programmes for men who perpetrate violence against their female partners still makes a fragmentary impression. Across Europe various evaluation studies have been performed. However, the methodologies applied are too heterogeneous to allow the combination of the results in a meta-analytical way. In this paper we propose a future pathway for organising outcome evaluation studies of domestic violence perpetrator programmes in community settings, so that today's problems in this field can be overcome. In a pragmatic framework that acknowledges the limited pre-conditions for evaluation studies in the area of domestic violence perpetrator programmes as it is today, feasible approaches for outcome evaluation are outlined, with recent developments in the field taken as starting points. The framework for organising future evaluation studies of work with perpetrators of domestic violence is presented together with a strategy to promote this framework. International networks of practitioners and researchers play a central role in this strategy through upskilling the area of practical work, preparing the ground for evaluation research and improving cooperation between practitioners and researchers. This paper is based on the results of the European funded project IMPACT (under the Daphne-III-funding programme of the European Commission). Copyright © 2016 Elsevier Ltd. All rights reserved.

Remote sensing of multimodal transportation systems : research brief.

DOT National Transportation Integrated Search

2016-09-01

Remote Sensing of Multimodal Transportation Systems : Rapid condition monitoring and performance evaluations of the vast and vulnerable transportation infrastructure has been elusive. The framework and models developed in this research will enable th...

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ragan, Eric D.; Bowman, Doug A.; Kopper, Regis

Virtual reality training systems are commonly used in a variety of domains, and it is important to understand how the realism of a training simulation influences training effectiveness. The paper presents a framework for evaluating the effects of virtual reality fidelity based on an analysis of a simulation’s display, interaction, and scenario components. Following this framework, we conducted a controlled experiment to test the effects of fidelity on training effectiveness for a visual scanning task. The experiment varied the levels of field of view and visual realism during a training phase and then evaluated scanning performance with the simulator’s highestmore » level of fidelity. To assess scanning performance, we measured target detection and adherence to a prescribed strategy. The results show that both field of view and visual realism significantly affected target detection during training; higher field of view led to better performance and higher visual realism worsened performance. Additionally, the level of visual realism during training significantly affected learning of the prescribed visual scanning strategy, providing evidence that high visual realism was important for learning the technique. The results also demonstrate that task performance during training was not always a sufficient measure of mastery of an instructed technique. That is, if learning a prescribed strategy or skill is the goal of a training exercise, performance in a simulation may not be an appropriate indicator of effectiveness outside of training—evaluation in a more realistic setting may be necessary.« less

Integrating fisheries approaches and household utility models for improved resource management.

PubMed

Milner-Gulland, E J

2011-01-25

Natural resource management is littered with cases of overexploitation and ineffectual management, leading to loss of both biodiversity and human welfare. Disciplinary boundaries stifle the search for solutions to these issues. Here, I combine the approach of management strategy evaluation, widely applied in fisheries, with household utility models from the conservation and development literature, to produce an integrated framework for evaluating the effectiveness of competing management strategies for harvested resources against a range of performance metrics. I demonstrate the strengths of this approach with a simple model, and use it to examine the effect of manager ignorance of household decisions on resource management effectiveness, and an allocation tradeoff between monitoring resource stocks to reduce observation uncertainty and monitoring users to improve compliance. I show that this integrated framework enables management assessments to consider household utility as a direct metric for system performance, and that although utility and resource stock conservation metrics are well aligned, harvest yield is a poor proxy for both, because it is a product of household allocation decisions between alternate livelihood options, rather than an end in itself. This approach has potential far beyond single-species harvesting in situations where managers are in full control; I show that the integrated approach enables a range of management intervention options to be evaluated within the same framework.

Between-individual comparisons in performance evaluation: a perspective from prospect theory.

PubMed

Wong, Kin Fai Ellick; Kwong, Jessica Y Y

2005-03-01

This article examines how between-individual comparisons influence performance evaluations in rating tasks. The authors demonstrated a systematic change in the perceived difference across ratees as a result of changing the way performance information is expressed. Study 1 found that perceived performance difference between 2 individuals was greater when their objective performance levels were presented with small numbers (e.g., absence rates of 2% vs. 5%) than when they were presented with large numbers (e.g., attendance rates of 98% vs. 95%). Extending this finding to situations involving trade-offs between multiple performance attributes across ratees, Study 2 showed that the relative preference for 1 ratee over another actually reversed when the presentation format of the performance information changed. The authors draw upon prospect theory to offer a theoretical framework describing the between-individual comparison aspect of performance evaluation.

Microgravity isolation system design: A modern control analysis framework

NASA Technical Reports Server (NTRS)

Hampton, R. D.; Knospe, C. R.; Allaire, P. E.; Grodsinsky, C. M.

1994-01-01

Many acceleration-sensitive, microgravity science experiments will require active vibration isolation from the manned orbiters on which they will be mounted. The isolation problem, especially in the case of a tethered payload, is a complex three-dimensional one that is best suited to modern-control design methods. These methods, although more powerful than their classical counterparts, can nonetheless go only so far in meeting the design requirements for practical systems. Once a tentative controller design is available, it must still be evaluated to determine whether or not it is fully acceptable, and to compare it with other possible design candidates. Realistically, such evaluation will be an inherent part of a necessary iterative design process. In this paper, an approach is presented for applying complex mu-analysis methods to a closed-loop vibration isolation system (experiment plus controller). An analysis framework is presented for evaluating nominal stability, nominal performance, robust stability, and robust performance of active microgravity isolation systems, with emphasis on the effective use of mu-analysis methods.

A Reflective Learning Framework to Evaluate CME Effects on Practice Reflection

ERIC Educational Resources Information Center

Leung, Kit H.; Pluye, Pierre; Grad, Roland; Weston, Cynthia

2010-01-01

Introduction: The importance of reflective practice is recognized by the adoption of a reflective learning model in continuing medical education (CME), but little is known about how to evaluate reflective learning in CME. Reflective learning seldom is defined in terms of specific cognitive processes or observable performances. Competency-based…

Evaluation of iTunes University Courses through Instructional Design Strategies and m-Learning Framework

ERIC Educational Resources Information Center

Tseng, Hung Wei; Tang, Yingqi; Morris, Betty

2016-01-01

As mobile learning technology promotes learning accessibility and flexibility, students benefit from social interactivity and connective learning process which will also foster students' performance and satisfaction on learning content. The primary purpose of this research was to evaluate iTunes U courses based on instructional design strategies…

Advanced error diagnostics of the CMAQ and Chimere modelling systems within the AQMEII3 model evaluation framework

EPA Science Inventory

The work here complements the overview analysis of the modelling systems participating in the third phase of the Air Quality Model Evaluation International Initiative (AQMEII3) by focusing on the performance for hourly surface ozone by two modelling systems, Chimere for Europe an...

Development of indicators for measuring outcomes of water safety plans

PubMed Central

Lockhart, Gabriella; Oswald, William E.; Hubbard, Brian; Medlin, Elizabeth; Gelting, Richard J.

2015-01-01

Water safety plans (WSPs) are endorsed by the World Health Organization as the most effective method of protecting a water supply. With the increase in WSPs worldwide, several valuable resources have been developed to assist practitioners in the implementation of WSPs, yet there is still a need for a practical and standardized method of evaluating WSP effectiveness. In 2012, the Centers for Disease Control and Prevention (CDC) published a conceptual framework for the evaluation of WSPs, presenting four key outcomes of the WSP process: institutional, operational, financial and policy change. In this paper, we seek to operationalize this conceptual framework by providing a set of simple and practical indicators for assessing WSP outcomes. Using CDC’s WSP framework as a foundation and incorporating various existing performance monitoring indicators for water utilities, we developed a set of approximately 25 indicators of institutional, operational, financial and policy change within the WSP context. These outcome indicators hold great potential for the continued implementation and expansion of WSPs worldwide. Having a defined framework for evaluating a WSP’s effectiveness, along with a set of measurable indicators by which to carry out that evaluation, will help implementers assess key WSP outcomes internally, as well as benchmark their progress against other WSPs in their region and globally. PMID:26361540

Evaluation of Knowledge Development in a Healthcare Setting

NASA Astrophysics Data System (ADS)

Schaffer, Scott P.

Healthcare organizations worldwide have recently increased efforts to improve performance, quality, and knowledge transfer using information and communication technologies. Evaluation of the effectiveness and quality of such efforts is challenging. A macro and micro-level system evaluation conducted with a 14000 member US hospital administrative services organization examined the appropriateness of a blended face-to-face and technology-enabled performance improvement and knowledge development system. Furthermore, a successful team or microsystem in a high performing hospital was studied in-depth. Several types of data methods including interview, observation, and questionnaire were used to address evaluation questions within a knowledge development framework created for the study. Results of this preliminary study focus on how this organization attempted to organize clinical improvement efforts around quality and performance improvement processes supported by networked technologies.

Team table: a framework and tool for continuous factory planning

NASA Astrophysics Data System (ADS)

Sihn, Wilfried; Bischoff, Juergen; von Briel, Ralf; Josten, Marcus

2000-10-01

Growing market turbulences and shorter product life cycles require a continuous adaptation of factory structures resulting in a continuous factory planning process. Therefore a new framework is developed which focuses on configuration and data management process integration. This enable an online system performance evaluation based on continuous availability of current data. The use of this framework is especially helpful and will guarantee high cost and time savings, when used in the early stages of the planning, called the concept or rough planning phase. The new framework is supported by a planning round table as a tool for team-based configuration processes integrating the knowledge of all persons involved in planning processes. A case study conducted at a German company shows the advantages which can be achieved by implementing the new framework and methods.

The application of data mining and cloud computing techniques in data-driven models for structural health monitoring

NASA Astrophysics Data System (ADS)

Khazaeli, S.; Ravandi, A. G.; Banerji, S.; Bagchi, A.

2016-04-01

Recently, data-driven models for Structural Health Monitoring (SHM) have been of great interest among many researchers. In data-driven models, the sensed data are processed to determine the structural performance and evaluate the damages of an instrumented structure without necessitating the mathematical modeling of the structure. A framework of data-driven models for online assessment of the condition of a structure has been developed here. The developed framework is intended for automated evaluation of the monitoring data and structural performance by the Internet technology and resources. The main challenges in developing such framework include: (a) utilizing the sensor measurements to estimate and localize the induced damage in a structure by means of signal processing and data mining techniques, and (b) optimizing the computing and storage resources with the aid of cloud services. The main focus in this paper is to demonstrate the efficiency of the proposed framework for real-time damage detection of a multi-story shear-building structure in two damage scenarios (change in mass and stiffness) in various locations. Several features are extracted from the sensed data by signal processing techniques and statistical methods. Machine learning algorithms are deployed to select damage-sensitive features as well as classifying the data to trace the anomaly in the response of the structure. Here, the cloud computing resources from Amazon Web Services (AWS) have been used to implement the proposed framework.

Satisfactory surgical option for cartilage graft absorption in microtia reconstruction.

PubMed

Han, So-Eun; Oh, Kap Sung

2016-04-01

We routinely perform auricular elevation at least 6 months after implantation of framework in microtia reconstruction using costal cartilage. However, in a few cases, cartilage graft absorption has occurred, which has led to contour irregularity with unfavorable long-term results. In the present study, we recount the details of using additional rib cartilage augmentation to achieve an accentuated contour in cartilage graft absorption cases. The cartilage graft absorption was defined as contour irregularity or cartilage graft deformation as evaluated by the surgeon and patient. Depending on the extent of cartilage graft absorption, another rib cartilage framework was added to the previously implanted framework, targeting the absorption area. We used banked cartilage or harvested new cartilage based on three-dimensional rib computed tomography. Additional recontouring of framework was conducted in eight patients who were examined for cartilage graft absorption from 1.5 to 5 years after implantation of the framework. Four patients received additional rib cartilage augmentation and tissue expander insertion simultaneously prior to auricular elevation. Two patients underwent auricular elevation simultaneously. In another two patients, additional rib cartilage augmentation was performed before auricular elevation. The mean follow-up period was 18 months, and in all cases reconstructive results were acceptable. Although further follow-up evaluation is required, additional rib cartilage augmentation is an attractive surgical option for cartilage graft absorption cases. Copyright © 2016 European Association for Cranio-Maxillo-Facial Surgery. Published by Elsevier Ltd. All rights reserved.

Quantitative assessments of municipal waste management systems: using different indicators to compare and rank programs in New York State.

PubMed

Greene, Krista L; Tonjes, David J

2014-04-01

The primary objective of waste management technologies and policies in the United States is to reduce the harmful environmental impacts of waste, particularly those relating to energy consumption and climate change. Performance indicators are frequently used to evaluate the environmental quality of municipal waste systems, as well as to compare and rank programs relative to each other in terms of environmental performance. However, there currently is no consensus on the best indicator for performing these environmental evaluations. The purpose of this study is to examine the common performance indicators used to assess the environmental benefits of municipal waste systems to determine if there is agreement between them regarding which system performs best environmentally. Focus is placed on how indicator selection influences comparisons between municipal waste management programs and subsequent system rankings. The waste systems of ten municipalities in the state of New York, USA, were evaluated using each common performance indicator and Spearman correlations were calculated to see if there was a significant association between system rank orderings. Analyses showed that rank orders of waste systems differ substantially when different indicators are used. Therefore, comparative system assessments based on indicators should be considered carefully, especially those intended to gauge environmental quality. Insight was also gained into specific factors which may lead to one system achieving higher rankings than another. However, despite the insufficiencies of indicators for comparative quality assessments, they do provide important information for waste managers and they can assist in evaluating internal programmatic performance and progress. To enhance these types of assessments, a framework for scoring indicators based on criteria that evaluate their utility and value for system evaluations was developed. This framework was used to construct an improved model for waste system performance assessments. Copyright © 2014 Elsevier Ltd. All rights reserved.

Policy reconciliation for access control in dynamic cross-enterprise collaborations

NASA Astrophysics Data System (ADS)

Preuveneers, D.; Joosen, W.; Ilie-Zudor, E.

2018-03-01

In dynamic cross-enterprise collaborations, different enterprises form a - possibly temporary - business relationship. To integrate their business processes, enterprises may need to grant each other limited access to their information systems. Authentication and authorization are key to secure information handling. However, access control policies often rely on non-standardized attributes to describe the roles and permissions of their employees which convolutes cross-organizational authorization when business relationships evolve quickly. Our framework addresses the managerial overhead of continuous updates to access control policies for enterprise information systems to accommodate disparate attribute usage. By inferring attribute relationships, our framework facilitates attribute and policy reconciliation, and automatically aligns dynamic entitlements during the evaluation of authorization decisions. We validate our framework with a Industry 4.0 motivating scenario on networked production where such dynamic cross-enterprise collaborations are quintessential. The evaluation reveals the capabilities and performance of our framework, and illustrates the feasibility of liberating the security administrator from manually provisioning and aligning attributes, and verifying the consistency of access control policies for cross-enterprise collaborations.

Unstructured medical image query using big data - An epilepsy case study.

PubMed

Istephan, Sarmad; Siadat, Mohammad-Reza

2016-02-01

Big data technologies are critical to the medical field which requires new frameworks to leverage them. Such frameworks would benefit medical experts to test hypotheses by querying huge volumes of unstructured medical data to provide better patient care. The objective of this work is to implement and examine the feasibility of having such a framework to provide efficient querying of unstructured data in unlimited ways. The feasibility study was conducted specifically in the epilepsy field. The proposed framework evaluates a query in two phases. In phase 1, structured data is used to filter the clinical data warehouse. In phase 2, feature extraction modules are executed on the unstructured data in a distributed manner via Hadoop to complete the query. Three modules have been created, volume comparer, surface to volume conversion and average intensity. The framework allows for user-defined modules to be imported to provide unlimited ways to process the unstructured data hence potentially extending the application of this framework beyond epilepsy field. Two types of criteria were used to validate the feasibility of the proposed framework - the ability/accuracy of fulfilling an advanced medical query and the efficiency that Hadoop provides. For the first criterion, the framework executed an advanced medical query that spanned both structured and unstructured data with accurate results. For the second criterion, different architectures were explored to evaluate the performance of various Hadoop configurations and were compared to a traditional Single Server Architecture (SSA). The surface to volume conversion module performed up to 40 times faster than the SSA (using a 20 node Hadoop cluster) and the average intensity module performed up to 85 times faster than the SSA (using a 40 node Hadoop cluster). Furthermore, the 40 node Hadoop cluster executed the average intensity module on 10,000 models in 3h which was not even practical for the SSA. The current study is limited to epilepsy field and further research and more feature extraction modules are required to show its applicability in other medical domains. The proposed framework advances data-driven medicine by unleashing the content of unstructured medical data in an efficient and unlimited way to be harnessed by medical experts. Copyright © 2015 Elsevier Inc. All rights reserved.

Building a framework for ergonomic research on laparoscopic instrument handles.

PubMed

Li, Zheng; Wang, Guohui; Tan, Juan; Sun, Xulong; Lin, Hao; Zhu, Shaihong

2016-06-01

Laparoscopic surgery carries the advantage of minimal invasiveness, but ergonomic design of the instruments used has progressed slowly. Previous studies have demonstrated that the handle of laparoscopic instruments is vital for both surgical performance and surgeon's health. This review provides an overview of the sub-discipline of handle ergonomics, including an evaluation framework, objective and subjective assessment systems, data collection and statistical analyses. Furthermore, a framework for ergonomic research on laparoscopic instrument handles is proposed to standardize work on instrument design. Copyright © 2016 IJS Publishing Group Ltd. Published by Elsevier Ltd. All rights reserved.

[The GIPSY-RECPAM model: a versatile approach for integrated evaluation in cardiologic care].

PubMed

Carinci, F

2009-01-01

Tree-structured methodology applied for the GISSI-PSICOLOGIA project, although performed in the framework of earliest GISSI studies, represents a powerful tool to analyze different aspects of cardiologic care. The GISSI-PSICOLOGIA project has delivered a novel methodology based on the joint application of psychometric tools and sophisticated statistical techniques. Its prospective use could allow building effective epidemiological models relevant to the prognosis of the cardiologic patient. The various features of the RECPAM method allow a versatile use in the framework of modern e-health projects. The study used the Cognitive Behavioral Assessment H Form (CBA-H) psychometrics scales. The potential for its future application in the framework of Italian cardiology is relevant and particularly indicated to assist planning of systems for integrated care and routine evaluation of the cardiologic patient.

The Impact of the Government Performance and Results Act (GPRA) on Two State Cooperative Extension Systems

ERIC Educational Resources Information Center

Baughman, Sarah; Boyd, Heather H.; Kelsey, Kathleen D.

2012-01-01

The research reported here examined the impact of the Government Performance and Results Act on accountability and evaluation activities in two state Cooperative Extension Systems. Accountability was examined using five dimensions from Koppell's (2005) framework. Findings indicated both Extension systems transferred accountability activities to…

Monitoring the Performance of Educational Institutions: A Spur for the Implementation of Systemic Changes in Higher Education. Part Two

ERIC Educational Resources Information Center

Karelina, Irina Georgievna; Sobolev, Alexander Borisovich; Sorokin, Sviatoslav Olegovich

2016-01-01

The article discusses a comprehensive reporting and monitoring framework used to evaluate the performance of state and private higher education institutions. By analyzing diversified indicators including regulatory compliance, organizational and economic indicators, training and research, and other metrics, the authors spotlight key developments…

How discriminating are discriminative instruments?

PubMed

Hankins, Matthew

2008-05-27

The McMaster framework introduced by Kirshner & Guyatt is the dominant paradigm for the development of measures of health status and health-related quality of life (HRQL). The framework defines the functions of such instruments as evaluative, predictive or discriminative. Evaluative instruments are required to be sensitive to change (responsiveness), but there is no corresponding index of the degree to which discriminative instruments are sensitive to cross-sectional differences. This paper argues that indices of validity and reliability are not sufficient to demonstrate that a discriminative instrument performs its function of discriminating between individuals, and that the McMaster framework would be augmented by the addition of a separate index of discrimination. The coefficient proposed by Ferguson (Delta) is easily adapted to HRQL instruments and is a direct, non-parametric index of the degree to which an instrument distinguishes between individuals. While Delta should prove useful in the development and evaluation of discriminative instruments, further research is required to elucidate the relationship between the measurement properties of discrimination, reliability and responsiveness.

EMDS 3.0: A modeling framework for coping with complexity in environmental assessment and planning.

Treesearch

K.M. Reynolds

2006-01-01

EMDS 3.0 is implemented as an ArcMap® extension and integrates the logic engine of NetWeaver® to perform landscape evaluations, and the decision modeling engine of Criterium DecisionPlus® for evaluating management priorities. Key features of the system's evaluation component include abilities to (1) reason about large, abstract, multifaceted ecosystem management...

Teacher Voice: How Teachers Perceive Evaluations and How Leaders Can Use This Knowledge to Help Teachers Grow Professionally

ERIC Educational Resources Information Center

Hopkins, Paul

2016-01-01

The purpose of this study was to determine how K-12 public school teachers perceive the use of student performance data in teacher evaluations. The proprietary, utility, feasibility, and accuracy standards created by the Joint Committee on Standards for Education Evaluation served as a framework for the study. An online survey was deployed to a…

Evaluation of Historical and Projected Agricultural Climate Risk Over the Continental US

NASA Astrophysics Data System (ADS)

Zhu, X.; Troy, T. J.; Devineni, N.

2016-12-01

Food demands are rising due to an increasing population with changing food preferences, which places pressure on agricultural systems. In addition, in the past decade climate extremes have highlighted the vulnerability of our agricultural production to climate variability. Quantitative analyses in the climate-agriculture research field have been performed in many studies. However, climate risk still remains difficult to evaluate at large scales yet shows great potential of help us better understand historical climate change impacts and evaluate the future risk given climate projections. In this study, we developed a framework to evaluate climate risk quantitatively by applying statistical methods such as Bayesian regression, distribution fitting, and Monte Carlo simulation. We applied the framework over different climate regions in the continental US both historically and for modeled climate projections. The relative importance of any major growing season climate index, such as maximum dry period or heavy precipitation, was evaluated to determine what climate indices play a role in affecting crop yields. The statistical modeling framework was applied using county yields, with irrigated and rainfed yields separated to evaluate the different risk. This framework provides estimates of the climate risk facing agricultural production in the near-term that account for the full uncertainty of climate occurrences, range of crop response, and spatial correlation in climate. In particular, the method provides robust estimates of importance of irrigation in mitigating agricultural climate risk. The results of this study can contribute to decision making about crop choice and water use in an uncertain climate.

USDOT guidance summary for connected vehicle deployments : data sharing.

DOT National Transportation Integrated Search

2016-07-01

AbstractThe document provides guidance to Pilot Deployers in the timely and successful completion of Concept Development Phase deliverables, specifically in developing the Data Sharing Framework portion of the Performance Measurement and Evaluation S...

Do health economic evaluations using observational data provide reliable assessment of treatment effects?

PubMed Central

2013-01-01

Economic evaluation in modern health care systems is seen as a transparent scientific framework that can be used to advance progress towards improvements in population health at the best possible value. Despite the perceived superiority that trial-based studies have in terms of internal validity, economic evaluations often employ observational data. In this review, the interface between econometrics and economic evaluation is explored, with emphasis placed on highlighting methodological issues relating to the evaluation of cost-effectiveness within a bivariate framework. Studies that satisfied the eligibility criteria exemplified the use of matching, regression analysis, propensity scores, instrumental variables, as well as difference-in-differences approaches. All studies were reviewed and critically appraised using a structured template. The findings suggest that although state-of-the-art econometric methods have the potential to provide evidence on the causal effects of clinical and policy interventions, their application in economic evaluation is subject to a number of limitations. These range from no credible assessment of key assumptions and scarce evidence regarding the relative performance of different methods, to lack of reporting of important study elements, such as a summary outcome measure and its associated sampling uncertainty. Further research is required to better understand the ways in which observational data should be analysed in the context of the economic evaluation framework. PMID:24229445

Examining Primary Healthcare Performance through a Triple Aim Lens.

PubMed

Ryan, Bridget L; Brown, Judith Belle; Glazier, Richard H; Hutchison, Brian

2016-02-01

This study sought to apply a Triple Aim framework to the measurement and evaluation of primary healthcare (PHC) team performance. Triple Aim components were populated with 10 dimensions derived from survey and health administrative data for 17 Family Health Teams (FHTs) in Ontario, Canada. Bivariate analyses and rankings of sites examined the relationships among dimensions and among Triple Aim components. Readily available measures to fully populate the Triple Aim framework were lacking in FHTs. Within sites, there was little consistency in performance across the Triple Aim components (health, patient experience and cost). More and better measures are needed that can be readily used to examine the Triple Aim performance in PHC teams. FHTs, in this study, are partially achieving Triple Aim goals; however, there was a lack of consistency in performance. It is essential to collect appropriate measures and attend to performance across all components of the Triple Aim. Copyright © 2016 Longwoods Publishing.

Examining Primary Healthcare Performance through a Triple Aim Lens

PubMed Central

Ryan, Bridget L.; Brown, Judith Belle; Glazier, Richard H.; Hutchison, Brian

2016-01-01

Purpose: This study sought to apply a Triple Aim framework to the measurement and evaluation of primary healthcare (PHC) team performance. Methods: Triple Aim components were populated with 10 dimensions derived from survey and health administrative data for 17 Family Health Teams (FHTs) in Ontario, Canada. Bivariate analyses and rankings of sites examined the relationships among dimensions and among Triple Aim components. Results: Readily available measures to fully populate the Triple Aim framework were lacking in FHTs. Within sites, there was little consistency in performance across the Triple Aim components (health, patient experience and cost). Conclusions: More and better measures are needed that can be readily used to examine the Triple Aim performance in PHC teams. FHTs, in this study, are partially achieving Triple Aim goals; however, there was a lack of consistency in performance. It is essential to collect appropriate measures and attend to performance across all components of the Triple Aim. PMID:27027790

Nested Interrupt Analysis of Low Cost and High Performance Embedded Systems Using GSPN Framework

NASA Astrophysics Data System (ADS)

Lin, Cheng-Min

Interrupt service routines are a key technology for embedded systems. In this paper, we introduce the standard approach for using Generalized Stochastic Petri Nets (GSPNs) as a high-level model for generating CTMC Continuous-Time Markov Chains (CTMCs) and then use Markov Reward Models (MRMs) to compute the performance for embedded systems. This framework is employed to analyze two embedded controllers with low cost and high performance, ARM7 and Cortex-M3. Cortex-M3 is designed with a tail-chaining mechanism to improve the performance of ARM7 when a nested interrupt occurs on an embedded controller. The Platform Independent Petri net Editor 2 (PIPE2) tool is used to model and evaluate the controllers in terms of power consumption and interrupt overhead performance. Using numerical results, in spite of the power consumption or interrupt overhead, Cortex-M3 performs better than ARM7.

Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram

PubMed Central

2015-01-01

Objectives Technological advances using telemedicine and telehealth are growing in healthcare fields, but the evaluation framework for them is inconsistent and limited. This paper suggests a comprehensive evaluation framework for telemedicine system implementation and will support related stakeholders' decision-making by promoting general understanding, and resolving arguments and controversies. Methods This study focused on developing a comprehensive evaluation framework by summarizing themes across the range of evaluation techniques and organized foundational evaluation frameworks generally applicable through studies and cases of diverse telemedicine. Evaluation factors related to aspects of information technology; the evaluation of satisfaction of service providers and consumers, cost, quality, and information security are organized using the fishbone diagram. Results It was not easy to develop a monitoring and evaluation framework for telemedicine since evaluation frameworks for telemedicine are very complex with many potential inputs, activities, outputs, outcomes, and stakeholders. A conceptual framework was developed that incorporates the key dimensions that need to be considered in the evaluation of telehealth implementation for a formal structured approach to the evaluation of a service. The suggested framework consists of six major dimensions and the subsequent branches for each dimension. Conclusions To implement telemedicine and telehealth services, stakeholders should make decisions based on sufficient evidence in quality and safety measured by the comprehensive evaluation framework. Further work would be valuable in applying more comprehensive evaluations to verify and improve the comprehensive framework across a variety of contexts with more factors and participant group dimensions. PMID:26618028

Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram.

PubMed

Chang, Hyejung

2015-10-01

Technological advances using telemedicine and telehealth are growing in healthcare fields, but the evaluation framework for them is inconsistent and limited. This paper suggests a comprehensive evaluation framework for telemedicine system implementation and will support related stakeholders' decision-making by promoting general understanding, and resolving arguments and controversies. This study focused on developing a comprehensive evaluation framework by summarizing themes across the range of evaluation techniques and organized foundational evaluation frameworks generally applicable through studies and cases of diverse telemedicine. Evaluation factors related to aspects of information technology; the evaluation of satisfaction of service providers and consumers, cost, quality, and information security are organized using the fishbone diagram. It was not easy to develop a monitoring and evaluation framework for telemedicine since evaluation frameworks for telemedicine are very complex with many potential inputs, activities, outputs, outcomes, and stakeholders. A conceptual framework was developed that incorporates the key dimensions that need to be considered in the evaluation of telehealth implementation for a formal structured approach to the evaluation of a service. The suggested framework consists of six major dimensions and the subsequent branches for each dimension. To implement telemedicine and telehealth services, stakeholders should make decisions based on sufficient evidence in quality and safety measured by the comprehensive evaluation framework. Further work would be valuable in applying more comprehensive evaluations to verify and improve the comprehensive framework across a variety of contexts with more factors and participant group dimensions.

Framework for Deploying a Virtualized Computing Environment for Collaborative and Secure Data Analytics

PubMed Central

Meyer, Adrian; Green, Laura; Faulk, Ciearro; Galla, Stephen; Meyer, Anne-Marie

2016-01-01

Introduction: Large amounts of health data generated by a wide range of health care applications across a variety of systems have the potential to offer valuable insight into populations and health care systems, but robust and secure computing and analytic systems are required to leverage this information. Framework: We discuss our experiences deploying a Secure Data Analysis Platform (SeDAP), and provide a framework to plan, build and deploy a virtual desktop infrastructure (VDI) to enable innovation, collaboration and operate within academic funding structures. It outlines 6 core components: Security, Ease of Access, Performance, Cost, Tools, and Training. Conclusion: A platform like SeDAP is not simply successful through technical excellence and performance. It’s adoption is dependent on a collaborative environment where researchers and users plan and evaluate the requirements of all aspects. PMID:27683665

An Integrated Framework for Human-Robot Collaborative Manipulation.

PubMed

Sheng, Weihua; Thobbi, Anand; Gu, Ye

2015-10-01

This paper presents an integrated learning framework that enables humanoid robots to perform human-robot collaborative manipulation tasks. Specifically, a table-lifting task performed jointly by a human and a humanoid robot is chosen for validation purpose. The proposed framework is split into two phases: 1) phase I-learning to grasp the table and 2) phase II-learning to perform the manipulation task. An imitation learning approach is proposed for phase I. In phase II, the behavior of the robot is controlled by a combination of two types of controllers: 1) reactive and 2) proactive. The reactive controller lets the robot take a reactive control action to make the table horizontal. The proactive controller lets the robot take proactive actions based on human motion prediction. A measure of confidence of the prediction is also generated by the motion predictor. This confidence measure determines the leader/follower behavior of the robot. Hence, the robot can autonomously switch between the behaviors during the task. Finally, the performance of the human-robot team carrying out the collaborative manipulation task is experimentally evaluated on a platform consisting of a Nao humanoid robot and a Vicon motion capture system. Results show that the proposed framework can enable the robot to carry out the collaborative manipulation task successfully.

A new framework to enhance the interpretation of external validation studies of clinical prediction models.

PubMed

Debray, Thomas P A; Vergouwe, Yvonne; Koffijberg, Hendrik; Nieboer, Daan; Steyerberg, Ewout W; Moons, Karel G M

2015-03-01

It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models. We propose to quantify the degree of relatedness between development and validation samples on a scale ranging from reproducibility to transportability by evaluating their corresponding case-mix differences. We subsequently assess the models' performance in the validation sample and interpret the performance in view of the case-mix differences. Finally, we may adjust the model to the validation setting. We illustrate this three-step framework with a prediction model for diagnosing deep venous thrombosis using three validation samples with varying case mix. While one external validation sample merely assessed the model's reproducibility, two other samples rather assessed model transportability. The performance in all validation samples was adequate, and the model did not require extensive updating to correct for miscalibration or poor fit to the validation settings. The proposed framework enhances the interpretation of findings at external validation of prediction models. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.

Distributed Simulation as a modelling tool for the development of a simulation-based training programme for cardiovascular specialties.

PubMed

Kelay, Tanika; Chan, Kah Leong; Ako, Emmanuel; Yasin, Mohammad; Costopoulos, Charis; Gold, Matthew; Kneebone, Roger K; Malik, Iqbal S; Bello, Fernando

2017-01-01

Distributed Simulation is the concept of portable, high-fidelity immersive simulation. Here, it is used for the development of a simulation-based training programme for cardiovascular specialities. We present an evidence base for how accessible, portable and self-contained simulated environments can be effectively utilised for the modelling, development and testing of a complex training framework and assessment methodology. Iterative user feedback through mixed-methods evaluation techniques resulted in the implementation of the training programme. Four phases were involved in the development of our immersive simulation-based training programme: ( 1) initial conceptual stage for mapping structural criteria and parameters of the simulation training framework and scenario development ( n  = 16), (2) training facility design using Distributed Simulation , (3) test cases with clinicians ( n  = 8) and collaborative design, where evaluation and user feedback involved a mixed-methods approach featuring (a) quantitative surveys to evaluate the realism and perceived educational relevance of the simulation format and framework for training and (b) qualitative semi-structured interviews to capture detailed feedback including changes and scope for development. Refinements were made iteratively to the simulation framework based on user feedback, resulting in (4) transition towards implementation of the simulation training framework, involving consistent quantitative evaluation techniques for clinicians ( n  = 62). For comparative purposes, clinicians' initial quantitative mean evaluation scores for realism of the simulation training framework, realism of the training facility and relevance for training ( n  = 8) are presented longitudinally, alongside feedback throughout the development stages from concept to delivery, including the implementation stage ( n  = 62). Initially, mean evaluation scores fluctuated from low to average, rising incrementally. This corresponded with the qualitative component, which augmented the quantitative findings; trainees' user feedback was used to perform iterative refinements to the simulation design and components (collaborative design), resulting in higher mean evaluation scores leading up to the implementation phase. Through application of innovative Distributed Simulation techniques, collaborative design, and consistent evaluation techniques from conceptual, development, and implementation stages, fully immersive simulation techniques for cardiovascular specialities are achievable and have the potential to be implemented more broadly.

Comprehensive planning of data archive in Japanese planetary missions

NASA Astrophysics Data System (ADS)

Yamamoto, Yukio; Shinohara, Iku; Hoshino, Hirokazu; Tateno, Naoki; Hareyama, Makoto; Okada, Naoki; Ebisawa, Ken

Comprehensive planning of data archive in Japanese planetary missions Japan Aerospace Exploration Agency (JAXA) provides HAYABUSA and KAGUYA data as planetary data archives. These data archives, however, were prepared independently. Therefore the inconsistency of data format has occurred, and the knowledge of data archiving activity is not inherited. Recently, the discussion of comprehensive planning of data archive has started to prepare up-coming planetary missions, which indicates the comprehensive plan of data archive is required in several steps. The framework of the comprehensive plan is divided into four items: Preparation, Evaluation, Preservation, and Service. 1. PREPARATION FRAMEWORK Data is classified into several types: raw data, level-0, 1, 2 processing data, ancillary data, and etc. The task of mission data preparation is responsible for instrument teams, but preparations beside mission data and support of data management are essential to make unified conventions and formats over instruments in a mission, and over missions. 2. EVALUATION FRAMEWORK There are two meanings of evaluation: format and quality. The format evaluation is often discussed in the preparation framework. The data quality evaluation which is often called quality assurance (QA) or quality control (QC) must be performed by third party apart from preparation teams. An instrument team has the initiative for the preparation itself, and the third-party group is organized to evaluate the instrument team's activity. 3. PRESERVATION FRAMEWORK The main topic of this framework is document management, archiving structure, and simple access method. The mission produces many documents in the process of the development. Instrument de-velopment is no exception. During long-term development of a mission, many documents are obsoleted and updated repeatedly. A smart system will help instrument team to reduce some troubles of document management and archiving task. JAXA attempts to follow PDS manners to do this management since PDS has highly sophisticated archiving structure. In addition, the access method to archived data must be simple and standard well over a decade. 4. SERVICE FRAMEWORK The service framework including planetary data access protocol, PDAP, has been developed to share a stored data effectively. The sophisticated service framework will work not only for publication data, but also for low-level data. JAXA's data query services is under developed based on PDAP, which means that the low-level data can be published in the same manner as level 2 data. In this presentation, we report the detail structure of these four frameworks adopting upcoming Planet-C, Venus Climate Orbiter, mission.

Deterministic and fuzzy-based methods to evaluate community resilience

NASA Astrophysics Data System (ADS)

Kammouh, Omar; Noori, Ali Zamani; Taurino, Veronica; Mahin, Stephen A.; Cimellaro, Gian Paolo

2018-04-01

Community resilience is becoming a growing concern for authorities and decision makers. This paper introduces two indicator-based methods to evaluate the resilience of communities based on the PEOPLES framework. PEOPLES is a multi-layered framework that defines community resilience using seven dimensions. Each of the dimensions is described through a set of resilience indicators collected from literature and they are linked to a measure allowing the analytical computation of the indicator's performance. The first method proposed in this paper requires data on previous disasters as an input and returns as output a performance function for each indicator and a performance function for the whole community. The second method exploits a knowledge-based fuzzy modeling for its implementation. This method allows a quantitative evaluation of the PEOPLES indicators using descriptive knowledge rather than deterministic data including the uncertainty involved in the analysis. The output of the fuzzy-based method is a resilience index for each indicator as well as a resilience index for the community. The paper also introduces an open source online tool in which the first method is implemented. A case study illustrating the application of the first method and the usage of the tool is also provided in the paper.

The Getting To Outcomes Demonstration and Evaluation: An Illustration of the Prevention Support System

PubMed Central

Hunter, Sarah B.; Ebener, Patricia; Paddock, Susan M.; Stillman, Lindsey; Imm, Pamela; Wandersman, Abraham

2010-01-01

Communities are increasingly being required by state and federal funders to achieve outcomes and be accountable, yet are often not provided the guidance or the tools needed to successfully meet this challenge. To improve the likelihood of achieving positive outcomes, the Getting To Outcomes (GTO) intervention (manual, training, technical assistance) is designed to provide the necessary guidance and tools, tailored to community needs, in order to build individual capacity and program performance. GTO is an example of a Prevention Support System intervention, which as conceptualized by the Interactive Systems Framework, plays a key role in bridging the gap between prevention science (Prevention Synthesis and Translation System) and prevention practice (Prevention Delivery System). We evaluated the impact of GTO on individual capacity and program performance using survey- and interview-based methods. We tracked the implementation of GTO and gathered user feedback about its utility and acceptability. The evaluation of GTO suggests that it can build individual capacity and program performance and as such demonstrates that the Prevention Support System can successfully fulfill its intended role. Lessons learned from the implementation of GTO relevant to illuminating the framework are discussed. PMID:18278551

Medical Subject Headings (MeSH) for indexing and retrieving open-source healthcare data.

PubMed

Marc, David T; Khairat, Saif S

2014-01-01

The US federal government initiated the Open Government Directive where federal agencies are required to publish high value datasets so that they are available to the public. Data.gov and the community site Healthdata.gov were initiated to disperse such datasets. However, data searches and retrieval for these sites are keyword driven and severely limited in performance. The purpose of this paper is to address the issue of extracting relevant open-source data by proposing a method of adopting the MeSH framework for indexing and data retrieval. A pilot study was conducted to compare the performance of traditional keywords to MeSH terms for retrieving relevant open-source datasets related to "mortality". The MeSH framework resulted in greater sensitivity with comparable specificity to the keyword search. MeSH showed promise as a method for indexing and retrieving data, yet future research should conduct a larger scale evaluation of the performance of the MeSH framework for retrieving relevant open-source healthcare datasets.

Deductive Evaluation: Formal Code Analysis With Low User Burden

NASA Technical Reports Server (NTRS)

Di Vito, Ben. L

2016-01-01

We describe a framework for symbolically evaluating iterative C code using a deductive approach that automatically discovers and proves program properties. Although verification is not performed, the method can infer detailed program behavior. Software engineering work flows could be enhanced by this type of analysis. Floyd-Hoare verification principles are applied to synthesize loop invariants, using a library of iteration-specific deductive knowledge. When needed, theorem proving is interleaved with evaluation and performed on the fly. Evaluation results take the form of inferred expressions and type constraints for values of program variables. An implementation using PVS (Prototype Verification System) is presented along with results for sample C functions.

Developing a Three Processes Framework to Analyze Hydrologic Performance of Urban Stormwater Management in a Watershed Scale

NASA Astrophysics Data System (ADS)

Lyu, H.; Ni, G.; Sun, T.

2016-12-01

Urban stormwater management contributes to recover water cycle to a nearly natural situation. It is a challenge for analyzing the hydrologic performance in a watershed scale, since the measures are various of sorts and scales and work in different processes. A three processes framework is developed to simplify the urban hydrologic process on the surface and evaluate the urban stormwater management. The three processes include source utilization, transfer regulation and terminal detention, by which the stormwater is controlled in order or discharged. Methods for analyzing performance are based on the water controlled proportions by each process, which are calculated using USEPA Stormwater Management Model. A case study form Beijing is used to illustrate how the performance varies under a set of designed events of different return periods. This framework provides a method to assess urban stormwater management as a whole system considering the interaction between measures, and to examine if there is any weak process of an urban watershed to be improved. The results help to make better solutions of urban water crisis.

Assessment and self-assessment of the pharmacists' competencies using the global competency framework (GbCF) in Serbia.

PubMed

Stojkov, Svetlana; Tadić, Ivana; Crnjanski, Tatjana; Krajnović, Dušanka

2016-09-01

Pharmacists' competence represents a dynamic framework of knowledge, skills and abilities to carry out tasks, and it reflects on improving the quality of life and on patients’ health. One of the documents for the Evaluation and Competency Development of Pharmacists is the Global Competency Framework (GbCF). The aim of this study was to implement the GBCF document into Serbian pharmacies, to perform assessment and self assessment of the competencies. The assessment and self-assessment of pharmacists’ competencies were performed during the period 2012−13 year in eight community pharmacy chains, in seven cities in Serbia. For assessment and self-assessment of pharmacists competencies the GbCF model was applied, which was adjusted to pharmaceutical practice and legislation in Serbia. External assessment was conducted by teams of pharmacists using the structured observation of the work of pharmacists during regular working hours. Evaluated pharmacists filled out the questionnaire about demographic indicators about the pharmacist and the pharmacy where they work. A total of 123 pharmacists were evaluated. Pharmacists’ Professional Competency Cluster (KK1) had the lowest score (average value 2.98), while the cluster Management and Organizational Competency (KK2) had the highest score (average value 3.15). The competence Recognition of the Diagnosis and Patient Counseling (K8), which belonged to the cluster KK1, had the lowest score (average value for assessment and self-assessment were 2.09, and 2.34, respectively) among the all evaluated competencies. GbCF might be considered as an instrument for the competencies' evaluation/selfevaluation and their improvement, accordingly.

Characterizing the reliability of a bioMEMS-based cantilever sensor

NASA Astrophysics Data System (ADS)

Bhalerao, Kaustubh D.

2004-12-01

The cantilever-based BioMEMS sensor represents one instance from many competing ideas of biosensor technology based on Micro Electro Mechanical Systems. The advancement of BioMEMS from laboratory-scale experiments to applications in the field will require standardization of their components and manufacturing procedures as well as frameworks to evaluate their performance. Reliability, the likelihood with which a system performs its intended task, is a compact mathematical description of its performance. The mathematical and statistical foundation of systems-reliability has been applied to the cantilever-based BioMEMS sensor. The sensor is designed to detect one aspect of human ovarian cancer, namely the over-expression of the folate receptor surface protein (FR-alpha). Even as the application chosen is clinically motivated, the objective of this study was to demonstrate the underlying systems-based methodology used to design, develop and evaluate the sensor. The framework development can be readily extended to other BioMEMS-based devices for disease detection and will have an impact in the rapidly growing $30 bn industry. The Unified Modeling Language (UML) is a systems-based framework for design and development of object-oriented information systems which has potential application for use in systems designed to interact with biological environments. The UML has been used to abstract and describe the application of the biosensor, to identify key components of the biosensor, and the technology needed to link them together in a coherent manner. The use of the framework is also demonstrated in computation of system reliability from first principles as a function of the structure and materials of the biosensor. The outcomes of applying the systems-based framework to the study are the following: (1) Characterizing the cantilever-based MEMS device for disease (cell) detection. (2) Development of a novel chemical interface between the analyte and the sensor that provides a degree of selectivity towards the disease. (3) Demonstrating the performance and measuring the reliability of the biosensor prototype, and (4) Identification of opportunities in technological development in order to further refine the proposed biosensor. Application of the methodology to design develop and evaluate the reliability of BioMEMS devices will be beneficial in the streamlining the growth of the BioMEMS industry, while providing a decision-support tool in comparing and adopting suitable technologies from available competing options.

Societal and Economic Engagement of Universities in Finland: An Evaluation Model

ERIC Educational Resources Information Center

Ritsila, Jari; Nieminen, Mika; Sotarauta, Markku; Lahtonen, Jukka

2008-01-01

This paper is based on the work of an expert team invited by the Ministry of Education of Finland to develop criteria and an evaluation framework for societal and economic engagement for use in university performance management. The paper maps out possible indicators for the societal and economic engagement of universities in the light of national…

Generics Pricing: The Greek Paradox.

PubMed

Karafyllis, Ioannis; Variti, Lamprini

2017-01-01

This paper explains and develops a methodological framework to help evaluate the performance of generic pharmaceutical policies and the correct evaluation of generics sales. Until today erroneous recording of generics does not help proper pricing and their penetration in the Greek market. This classifies Greece on the outliners in every study or comparison that is referred on papers or studies.

The Development and Initial Evaluation of the Human Readiness Level Framework

DTIC Science & Technology

2010-06-01

View ICD Initial Capabilities Document ICW Interactive Course Ware ILE Interactive Learning Environment ILT Instructor Led Training IOC...Programmatic Environmental Safety and Health Evaluation PHA Preliminary Hazard Analysis PHL Preliminary Hazard List xiv PM Program Manager PQS...Occupational Health SOW Statement of Work SRD System Requirements Document SPS System Performance Specification SRR System Requirements Review SVR

Building Social Infrastructure through Public-Private Partnerships: The Case of Student Housing in Public Higher Education

ERIC Educational Resources Information Center

Cole, Bruce Kevin

2012-01-01

Evaluations of Public-Private Partnership arrangements as alternatives to traditional government procurement methods for the delivery of public infrastructure projects have been anecdotal at best. This paper proposes a framework to evaluate a public university's infrastructure asset management performance and a specific measure based on a new…

ORD Human Health Risk Assessment (HHRA) Research Program Mid-Cycle Progress Report and Review - 2010

EPA Pesticide Factsheets

The principal charge to the BOSC reviewers was to evaluate ORD’s HHRA Program from a program assessment framework relative to program relevance, structure, performance, quality, leadership, communication, and outcomes.

An Efficient Finite Element Framework to Assess Flexibility Performances of SMA Self-Expandable Carotid Artery Stents

PubMed Central

Ferraro, Mauro; Auricchio, Ferdinando; Boatti, Elisa; Scalet, Giulia; Conti, Michele; Morganti, Simone; Reali, Alessandro

2015-01-01

Computer-based simulations are nowadays widely exploited for the prediction of the mechanical behavior of different biomedical devices. In this aspect, structural finite element analyses (FEA) are currently the preferred computational tool to evaluate the stent response under bending. This work aims at developing a computational framework based on linear and higher order FEA to evaluate the flexibility of self-expandable carotid artery stents. In particular, numerical simulations involving large deformations and inelastic shape memory alloy constitutive modeling are performed, and the results suggest that the employment of higher order FEA allows accurately representing the computational domain and getting a better approximation of the solution with a widely-reduced number of degrees of freedom with respect to linear FEA. Moreover, when buckling phenomena occur, higher order FEA presents a superior capability of reproducing the nonlinear local effects related to buckling phenomena. PMID:26184329

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition.

PubMed

Ordóñez, Francisco Javier; Roggen, Daniel

2016-01-18

Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters' influence on performance to provide insights about their optimisation.

Trial Implementation of a Multihazard Risk Assessment Framework for High-Impact Low-Frequency Power Grid Events

DOE Office of Scientific and Technical Information (OSTI.GOV)

Veeramany, Arun; Coles, Garill A.; Unwin, Stephen D.

The Pacific Northwest National Laboratory developed a risk framework for modeling high-impact, low-frequency power grid events to support risk-informed decisions. In this paper, we briefly recap the framework and demonstrate its implementation for seismic and geomagnetic hazards using a benchmark reliability test system. We describe integration of a collection of models implemented to perform hazard analysis, fragility evaluation, consequence estimation, and postevent restoration. We demonstrate the value of the framework as a multihazard power grid risk assessment and management tool. As a result, the research will benefit transmission planners and emergency planners by improving their ability to maintain a resilientmore » grid infrastructure against impacts from major events.« less

Trial Implementation of a Multihazard Risk Assessment Framework for High-Impact Low-Frequency Power Grid Events

DOE PAGES

Veeramany, Arun; Coles, Garill A.; Unwin, Stephen D.; ...

2017-08-25

The Pacific Northwest National Laboratory developed a risk framework for modeling high-impact, low-frequency power grid events to support risk-informed decisions. In this paper, we briefly recap the framework and demonstrate its implementation for seismic and geomagnetic hazards using a benchmark reliability test system. We describe integration of a collection of models implemented to perform hazard analysis, fragility evaluation, consequence estimation, and postevent restoration. We demonstrate the value of the framework as a multihazard power grid risk assessment and management tool. As a result, the research will benefit transmission planners and emergency planners by improving their ability to maintain a resilientmore » grid infrastructure against impacts from major events.« less

A Framework For Analysis Of Coastal Infrastructure Vunerabilty To Global Sea Level Rise

NASA Astrophysics Data System (ADS)

Obrien, P. S.; White, K. D.; Veatch, W.; Marzion, R.; Moritz, H.; Moritz, H. R.

2017-12-01

Recorded impacts of global sea rise on coastal water levels have been documented over the past 100 to 150 years. In the recent 40 years the assumption of hydrologic stationarity has been recognized as invalid. New coastal infrastructure designs must recognize the paradigm shift from hydrologic stationarity to non-stationarity in coastal hydrology. A framework for the evaluation of existing coastal infrastructure is proposed to effectively assess design vulnerability. Two data sets developed from existing structures are chosen to test a proposed framework for vunerabilty to global sea level rise, with the proposed name Climate Preparedness and Resilience Register (CPRR). The CPRR framework consists of four major elements; Datum Adjustment, Coastal Water Levels, Scenario Projections and Performance Thresholds.

Integrated health monitoring and controls for rocket engines

NASA Technical Reports Server (NTRS)

Merrill, W. C.; Musgrave, J. L.; Guo, T. H.

1992-01-01

Current research in intelligent control systems at the Lewis Research Center is described in the context of a functional framework. The framework is applicable to a variety of reusable space propulsion systems for existing and future launch vehicles. It provides a 'road map' technology development to enable enhanced engine performance with increased reliability, durability, and maintainability. The framework hierarchy consists of a mission coordination level, a propulsion system coordination level, and an engine control level. Each level is described in the context of the Space Shuttle Main Engine. The concept of integrating diagnostics with control is discussed within the context of the functional framework. A distributed real time simulation testbed is used to realize and evaluate the functionalities in closed loop.

Advances in Landslide Hazard Forecasting: Evaluation of Global and Regional Modeling Approach

NASA Technical Reports Server (NTRS)

Kirschbaum, Dalia B.; Adler, Robert; Hone, Yang; Kumar, Sujay; Peters-Lidard, Christa; Lerner-Lam, Arthur

2010-01-01

A prototype global satellite-based landslide hazard algorithm has been developed to identify areas that exhibit a high potential for landslide activity by combining a calculation of landslide susceptibility with satellite-derived rainfall estimates. A recent evaluation of this algorithm framework found that while this tool represents an important first step in larger-scale landslide forecasting efforts, it requires several modifications before it can be fully realized as an operational tool. The evaluation finds that the landslide forecasting may be more feasible at a regional scale. This study draws upon a prior work's recommendations to develop a new approach for considering landslide susceptibility and forecasting at the regional scale. This case study uses a database of landslides triggered by Hurricane Mitch in 1998 over four countries in Central America: Guatemala, Honduras, EI Salvador and Nicaragua. A regional susceptibility map is calculated from satellite and surface datasets using a statistical methodology. The susceptibility map is tested with a regional rainfall intensity-duration triggering relationship and results are compared to global algorithm framework for the Hurricane Mitch event. The statistical results suggest that this regional investigation provides one plausible way to approach some of the data and resolution issues identified in the global assessment, providing more realistic landslide forecasts for this case study. Evaluation of landslide hazards for this extreme event helps to identify several potential improvements of the algorithm framework, but also highlights several remaining challenges for the algorithm assessment, transferability and performance accuracy. Evaluation challenges include representation errors from comparing susceptibility maps of different spatial resolutions, biases in event-based landslide inventory data, and limited nonlandslide event data for more comprehensive evaluation. Additional factors that may improve algorithm performance accuracy include incorporating additional triggering factors such as tectonic activity, anthropogenic impacts and soil moisture into the algorithm calculation. Despite these limitations, the methodology presented in this regional evaluation is both straightforward to calculate and easy to interpret, making results transferable between regions and allowing findings to be placed within an inter-comparison framework. The regional algorithm scenario represents an important step in advancing regional and global-scale landslide hazard assessment and forecasting.

OPPE, FPPE, QPS, and why the alphabet soup of physician assessment is essential for safer patient care.

PubMed

Loftus, Michael L

Creating a successful quality and patient safety program requires a multifaceted approach that systematically reviews overall systems and processes, but also creates a standardized framework for evaluating individual practitioner performance on a routine basis. There are two required elements of competency assessment that are typically tied to the hospital credentialing process: ongoing professional practice evaluation (OPPE) and focused professional practice evaluation (FPPE). Each of these processes are mandated by the Joint Commission, and form an important cornerstone for ensuring adequate physician performance and knowledge base. Copyright © 2017. Published by Elsevier Inc.

Overcoming complexities: Damage detection using dictionary learning framework

NASA Astrophysics Data System (ADS)

Alguri, K. Supreet; Melville, Joseph; Deemer, Chris; Harley, Joel B.

2018-04-01

For in situ damage detection, guided wave structural health monitoring systems have been widely researched due to their ability to evaluate large areas and their ability detect many types of damage. These systems often evaluate structural health by recording initial baseline measurements from a pristine (i.e., undamaged) test structure and then comparing later measurements with that baseline. Yet, it is not always feasible to have a pristine baseline. As an alternative, substituting the baseline with data from a surrogate (nearly identical and pristine) structure is a logical option. While effective in some circumstance, surrogate data is often still a poor substitute for pristine baseline measurements due to minor differences between the structures. To overcome this challenge, we present a dictionary learning framework to adapt surrogate baseline data to better represent an undamaged test structure. We compare the performance of our framework with two other surrogate-based damage detection strategies: (1) using raw surrogate data for comparison and (2) using sparse wavenumber analysis, a precursor to our framework for improving the surrogate data. We apply our framework to guided wave data from two 108 mm by 108 mm aluminum plates. With 20 measurements, we show that our dictionary learning framework achieves a 98% accuracy, raw surrogate data achieves a 92% accuracy, and sparse wavenumber analysis achieves a 57% accuracy.

A generic framework to simulate realistic lung, liver and renal pathologies in CT imaging

NASA Astrophysics Data System (ADS)

Solomon, Justin; Samei, Ehsan

2014-11-01

Realistic three-dimensional (3D) mathematical models of subtle lesions are essential for many computed tomography (CT) studies focused on performance evaluation and optimization. In this paper, we develop a generic mathematical framework that describes the 3D size, shape, contrast, and contrast-profile characteristics of a lesion, as well as a method to create lesion models based on CT data of real lesions. Further, we implemented a technique to insert the lesion models into CT images in order to create hybrid CT datasets. This framework was used to create a library of realistic lesion models and corresponding hybrid CT images. The goodness of fit of the models was assessed using the coefficient of determination (R2) and the visual appearance of the hybrid images was assessed with an observer study using images of both real and simulated lesions and receiver operator characteristic (ROC) analysis. The average R2 of the lesion models was 0.80, implying that the models provide a good fit to real lesion data. The area under the ROC curve was 0.55, implying that the observers could not readily distinguish between real and simulated lesions. Therefore, we conclude that the lesion-modeling framework presented in this paper can be used to create realistic lesion models and hybrid CT images. These models could be instrumental in performance evaluation and optimization of novel CT systems.

Determining the causal relationships among balanced scorecard perspectives on school safety performance: case of Saudi Arabia.

PubMed

Alolah, Turki; Stewart, Rodney A; Panuwatwanich, Kriengsak; Mohamed, Sherif

2014-07-01

In the public schools of many developing countries, numerous accidents and incidents occur because of poor safety regulations and management systems. To improve the educational environment in Saudi Arabia, the Ministry of Education seeks novel approaches to measure school safety performance in order to decrease incidents and accidents. The main objective of this research was to develop a systematic approach for measuring Saudi school safety performance using the balanced scorecard framework philosophy. The evolved third generation balanced scorecard framework is considered to be a suitable and robust framework that captures the system-wide leading and lagging indicators of business performance. The balanced scorecard architecture is ideal for adaptation to complex areas such as safety management where a holistic system evaluation is more effective than traditional compartmentalised approaches. In developing the safety performance balanced scorecard for Saudi schools, the conceptual framework was first developed and peer-reviewed by eighteen Saudi education experts. Next, 200 participants, including teachers, school executives, and Ministry of Education officers, were recruited to rate both the importance and the performance of 79 measurement items used in the framework. Exploratory factor analysis, followed by the confirmatory partial least squares method, was then conducted in order to operationalise the safety performance balanced scorecard, which encapsulates the following five salient perspectives: safety management and leadership; safety learning and training; safety policy, procedures and processes; workforce safety culture; and safety performance. Partial least squares based structural equation modelling was then conducted to reveal five significant relationships between perspectives, namely, safety management and leadership had a significant effect on safety learning and training and safety policy, procedures and processes, both safety learning and training and safety policy, procedures and processes had significant effects on workforce safety culture, and workforce safety culture had a significant effect on safety performance. Copyright © 2014 Elsevier Ltd. All rights reserved.

Monitoring the Performance of Educational Institutions: A Spur for the Implementation of Systemic Changes in Higher Education. Part One

ERIC Educational Resources Information Center

Karelina, Irina Georgievna; Sobolev, Alexander Borisovich; Sorokin, Sviatoslav Olegovich

2016-01-01

The article discusses the deployment of a comprehensive reporting and monitoring framework used to evaluate the performance of state and private higher education institutions in Russia. By referring to diversified indicators including organizational, financial and economic, training, research, graduate employment, and other metrics, the authors…

Can a Self-Efficacy-Based Intervention Decrease Burnout, Increase Engagement, and Enhance Performance? A Quasi-Experimental Study

ERIC Educational Resources Information Center

Breso, Edgar; Schaufeli, Wilmar; Salanova, Marisa

2011-01-01

Using the Social Cognitive Theory as a theoretical framework, this study evaluated a 4-month, individual cognitive-behavioral intervention program to decrease burnout and increase self-efficacy, engagement, and performance among university students. The main objective of the intervention was to decrease the anxiety the students coped with before…

AB1417 Performance Framework for the California Community Colleges: A Report to the Legislature, Pursuant to AB1417

ERIC Educational Resources Information Center

Research and Planning Group for California Community Colleges (RP Group), 2005

2005-01-01

This report to the Legislature and Governor provides recommendations on the design of a workable structure for the annual evaluation of district-level performance for California Community College districts. Under contract with the California Community Colleges Chancellor's Office, a six-member panel of the Research and Planning Group for…

Applying the Balanced Scorecard approach in teaching hospitals: a literature review and conceptual framework.

PubMed

Trotta, Annarita; Cardamone, Emma; Cavallaro, Giusy; Mauro, Marianna

2013-01-01

Teaching hospitals (THs) simultaneously serve three different roles: offering medical treatment, teaching future doctors and promoting research. The international literature recognises such organisations as 'peaks of excellence' and highlights their economic function in the health system. In addition, the literature describes the urgent need to manage the complex dynamics and inefficiency issues that threaten the survival of teaching hospitals worldwide. In this context, traditional performance measurement systems that focus only on accounting and financial measures appear to be inadequate. Given that THs are highly specific and complex, a multidimensional system of performance measurement, such as the Balanced Scorecard (BSC), may be more appropriate because of the multitude of stakeholders, each of whom seek a specific type of accountability. The aim of the paper was twofold: (i) to review the literature on the BSC and its applications in teaching hospitals and (ii) to propose a scorecard framework that is suitable for assessing the performance of THs and serving as a guide for scholars and practitioners. In addition, this research will contribute to the ongoing debate on performance evaluation systems by suggesting a revised BSC framework and proposing specific performance indicators for THs. Copyright © 2012 John Wiley & Sons, Ltd.

Effects of Field of View and Visual Complexity on Virtual Reality Training Effectiveness for a Visual Scanning Task

DOE PAGES

Ragan, Eric D.; Bowman, Doug A.; Kopper, Regis; ...

2015-02-13

Virtual reality training systems are commonly used in a variety of domains, and it is important to understand how the realism of a training simulation influences training effectiveness. The paper presents a framework for evaluating the effects of virtual reality fidelity based on an analysis of a simulation’s display, interaction, and scenario components. Following this framework, we conducted a controlled experiment to test the effects of fidelity on training effectiveness for a visual scanning task. The experiment varied the levels of field of view and visual realism during a training phase and then evaluated scanning performance with the simulator’s highestmore » level of fidelity. To assess scanning performance, we measured target detection and adherence to a prescribed strategy. The results show that both field of view and visual realism significantly affected target detection during training; higher field of view led to better performance and higher visual realism worsened performance. Additionally, the level of visual realism during training significantly affected learning of the prescribed visual scanning strategy, providing evidence that high visual realism was important for learning the technique. The results also demonstrate that task performance during training was not always a sufficient measure of mastery of an instructed technique. That is, if learning a prescribed strategy or skill is the goal of a training exercise, performance in a simulation may not be an appropriate indicator of effectiveness outside of training—evaluation in a more realistic setting may be necessary.« less

A UML-based meta-framework for system design in public health informatics.

PubMed

Orlova, Anna O; Lehmann, Harold

2002-01-01

The National Agenda for Public Health Informatics calls for standards in data and knowledge representation within public health, which requires a multi-level framework that links all aspects of public health. The literature of public health informatics and public health informatics application were reviewed. A UML-based systems analysis was performed. Face validity of results was evaluated in analyzing the public health domain of lead poisoning. The core class of the UML-based system of public health is the Public Health Domain, which is associated with multiple Problems, for which Actors provide Perspectives. Actors take Actions that define, generate, utilize and/or evaluate Data Sources. The life cycle of the domain is a sequence of activities attributed to its problems that spirals through multiple iterations and realizations within a domain. The proposed Public Health Informatics Meta-Framework broadens efforts in applying informatics principles to the field of public health

An ovine in vivo framework for tracheobronchial stent analysis.

PubMed

McGrath, Donnacha J; Thiebes, Anja Lena; Cornelissen, Christian G; O'Shea, Mary B; O'Brien, Barry; Jockenhoevel, Stefan; Bruzzi, Mark; McHugh, Peter E

2017-10-01

Tracheobronchial stents are most commonly used to restore patency to airways stenosed by tumour growth. Currently all tracheobronchial stents are associated with complications such as stent migration, granulation tissue formation, mucous plugging and stent strut fracture. The present work develops a computational framework to evaluate tracheobronchial stent designs in vivo. Pressurised computed tomography is used to create a biomechanical lung model which takes into account the in vivo stress state, global lung deformation and local loading from pressure variation. Stent interaction with the airway is then evaluated for a number of loading conditions including normal breathing, coughing and ventilation. Results of the analysis indicate that three of the major complications associated with tracheobronchial stents can potentially be analysed with this framework, which can be readily applied to the human case. Airway deformation caused by lung motion is shown to have a significant effect on stent mechanical performance, including implications for stent migration, granulation formation and stent fracture.

A portfolio evaluation framework for air transportation improvement projects

NASA Astrophysics Data System (ADS)

Baik, Hyeoncheol

This thesis explores the application of portfolio theory to the Air Transportation System (ATS) improvement. The ATS relies on complexly related resources and different stakeholder groups. Moreover, demand for air travel is significantly increasing relative to capacity of air transportation. In this environment, improving the ATS is challenging. Many projects, which are defined as technologies or initiatives, for improvement have been proposed and some have been demonstrated in practice. However, there is no clear understanding of how well these projects work in different conditions nor of how they interact with each other or with existing systems. These limitations make it difficult to develop good project combinations, or portfolios that maximize improvement. To help address this gap, a framework for identifying good portfolios is proposed. The framework can be applied to individual projects or portfolios of projects. Projects or portfolios are evaluated using four different groups of factors (effectiveness, time-to-implement, scope of applicability, and stakeholder impacts). Portfolios are also evaluated in terms of interaction-determining factors (prerequisites, co-requisites, limiting factors, and amplifying factors) because, while a given project might work well in isolation, interdependencies between projects or with existing systems could result in lower overall performance in combination. Ways to communicate a portfolio to decision makers are also introduced. The framework is unique because (1) it allows using a variety of available data, and (2) it covers diverse benefit metrics. For demonstrating the framework, an application to ground delay management projects serves as a case study. The portfolio evaluation approach introduced in this thesis can aid decision makers and researchers at universities and aviation agencies such as Federal Aviation Administration (FAA), National Aeronautics and Space Administration (NASA), and Department of Defense (DoD), in more fully evaluating the air transportation improvement projects.

Evaluation of Web Accessibility of Consumer Health Information Websites

PubMed Central

Zeng, Xiaoming; Parmanto, Bambang

2003-01-01

The objectives of the study are to construct a comprehensive framework for web accessibility evaluation, to evaluate the current status of web accessibility of consumer health information websites and to investigate the relationship between web accessibility and property of the websites. We selected 108 consumer health information websites from the directory service of a Web search engine. We used Web accessibility specifications to construct a framework for the measurement of Web Accessibility Barriers (WAB) of website. We found that none of the websites is completely accessible to people with disabilities, but governmental and educational health information websites exhibit better performance on web accessibility than other categories of websites. We also found that the correlation between the WAB score and the popularity of a website is statistically significant. PMID:14728272

Evaluation of web accessibility of consumer health information websites.

PubMed

Zeng, Xiaoming; Parmanto, Bambang

2003-01-01

The objectives of the study are to construct a comprehensive framework for web accessibility evaluation, to evaluate the current status of web accessibility of consumer health information websites and to investigate the relationship between web accessibility and property of the websites. We selected 108 consumer health information websites from the directory service of a Web search engine. We used Web accessibility specifications to construct a framework for the measurement of Web Accessibility Barriers (WAB) of website. We found that none of the websites is completely accessible to people with disabilities, but governmental and educational health information websites exhibit better performance on web accessibility than other categories of websites. We also found that the correlation between the WAB score and the popularity of a website is statistically significant.

Dynamic decision-making for reliability and maintenance analysis of manufacturing systems based on failure effects

NASA Astrophysics Data System (ADS)

Zhang, Ding; Zhang, Yingjie

2017-09-01

A framework for reliability and maintenance analysis of job shop manufacturing systems is proposed in this paper. An efficient preventive maintenance (PM) policy in terms of failure effects analysis (FEA) is proposed. Subsequently, reliability evaluation and component importance measure based on FEA are performed under the PM policy. A job shop manufacturing system is applied to validate the reliability evaluation and dynamic maintenance policy. Obtained results are compared with existed methods and the effectiveness is validated. Some vague understandings for issues such as network modelling, vulnerabilities identification, the evaluation criteria of repairable systems, as well as PM policy during manufacturing system reliability analysis are elaborated. This framework can help for reliability optimisation and rational maintenance resources allocation of job shop manufacturing systems.

Framework for quantitative evaluation of 3D vessel segmentation approaches using vascular phantoms in conjunction with 3D landmark localization and registration

NASA Astrophysics Data System (ADS)

Wörz, Stefan; Hoegen, Philipp; Liao, Wei; Müller-Eschner, Matthias; Kauczor, Hans-Ulrich; von Tengg-Kobligk, Hendrik; Rohr, Karl

2016-03-01

We introduce a framework for quantitative evaluation of 3D vessel segmentation approaches using vascular phantoms. Phantoms are designed using a CAD system and created with a 3D printer, and comprise realistic shapes including branches and pathologies such as abdominal aortic aneurysms (AAA). To transfer ground truth information to the 3D image coordinate system, we use a landmark-based registration scheme utilizing fiducial markers integrated in the phantom design. For accurate 3D localization of the markers we developed a novel 3D parametric intensity model that is directly fitted to the markers in the images. We also performed a quantitative evaluation of different vessel segmentation approaches for a phantom of an AAA.

Evaluating metal-organic frameworks for post-combustion carbon dioxide capture via temperature swing adsorption

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mason, JA; Sumida, K; Herm, ZR

Two representative metal-organic frameworks, Zn4O(BTB)(2)(BTB3- = 1,3,5-benzenetribenzoate; MOF-177) and Mg-2(dobdc) (dobdc(4-) = 1,4-dioxido-2,5-benzenedicarboxylate; Mg-MOF-74, CPO-27-Mg), are evaluated in detail for their potential use in post-combustion CO2 capture via temperature swing adsorption (TSA). Low-pressure single-component CO2 and N-2 adsorption isotherms were measured every 10 degrees C from 20 to 200 degrees C, allowing the performance of each material to be analyzed precisely. In order to gain a more complete understanding of the separation phenomena and the thermodynamics of CO2 adsorption, the isotherms were analyzed using a variety of methods. With regard to the isosteric heat of CO2 adsorption, Mg-2(dobdc) exhibits anmore » abrupt drop at loadings approaching the saturation of the Mg2+ sites, which has significant implications for regeneration in different industrial applications. The CO2/N-2 selectivities were calculated using ideal adsorbed solution theory (IAST) for MOF-177, Mg-2(dobdc), and zeolite NaX, and working capacities were estimated using a simplified TSA model. Significantly, MOF-177 fails to exhibit a positive working capacity even at regeneration temperatures as high as 200 degrees C, while Mg-2(dobdc) reaches a working capacity of 17.6 wt% at this temperature. Breakthrough simulations were also performed for the three materials, demonstrating the superior performance of Mg-2(dobdc) over MOF-177 and zeolite NaX. These results show that the presence of strong CO2 adsorption sites is essential for a metal-organic framework to be of utility in post-combustion CO2 capture via a TSA process, and present a methodology for the evaluation of new metal-organic frameworks via analysis of single-component gas adsorption isotherms.« less

The Role of Interpersonal Relations in Healthcare Team Communication and Patient Safety: A Proposed Model of Interpersonal Process in Teamwork.

PubMed

Lee, Charlotte Tsz-Sum; Doran, Diane Marie

2017-06-01

Patient safety is compromised by medical errors and adverse events related to miscommunications among healthcare providers. Communication among healthcare providers is affected by human factors, such as interpersonal relations. Yet, discussions of interpersonal relations and communication are lacking in healthcare team literature. This paper proposes a theoretical framework that explains how interpersonal relations among healthcare team members affect communication and team performance, such as patient safety. We synthesized studies from health and social science disciplines to construct a theoretical framework that explicates the links among these constructs. From our synthesis, we identified two relevant theories: framework on interpersonal processes based on social relation model and the theory of relational coordination. The former involves three steps: perception, evaluation, and feedback; and the latter captures relational communicative behavior. We propose that manifestations of provider relations are embedded in the third step of the framework on interpersonal processes: feedback. Thus, varying team-member relationships lead to varying collaborative behavior, which affects patient-safety outcomes via a change in team communication. The proposed framework offers new perspectives for understanding how workplace relations affect healthcare team performance. The framework can be used by nurses, administrators, and educators to improve patient safety, team communication, or to resolve conflicts.

Combining Cryptography with EEG Biometrics

PubMed Central

Kazanavičius, Egidijus; Woźniak, Marcin

2018-01-01

Cryptographic frameworks depend on key sharing for ensuring security of data. While the keys in cryptographic frameworks must be correctly reproducible and not unequivocally connected to the identity of a user, in biometric frameworks this is different. Joining cryptography techniques with biometrics can solve these issues. We present a biometric authentication method based on the discrete logarithm problem and Bose-Chaudhuri-Hocquenghem (BCH) codes, perform its security analysis, and demonstrate its security characteristics. We evaluate a biometric cryptosystem using our own dataset of electroencephalography (EEG) data collected from 42 subjects. The experimental results show that the described biometric user authentication system is effective, achieving an Equal Error Rate (ERR) of 0.024.

Combining Cryptography with EEG Biometrics.

PubMed

Damaševičius, Robertas; Maskeliūnas, Rytis; Kazanavičius, Egidijus; Woźniak, Marcin

2018-01-01

Cryptographic frameworks depend on key sharing for ensuring security of data. While the keys in cryptographic frameworks must be correctly reproducible and not unequivocally connected to the identity of a user, in biometric frameworks this is different. Joining cryptography techniques with biometrics can solve these issues. We present a biometric authentication method based on the discrete logarithm problem and Bose-Chaudhuri-Hocquenghem (BCH) codes, perform its security analysis, and demonstrate its security characteristics. We evaluate a biometric cryptosystem using our own dataset of electroencephalography (EEG) data collected from 42 subjects. The experimental results show that the described biometric user authentication system is effective, achieving an Equal Error Rate (ERR) of 0.024.

Model-based transportation performance : a comparative framework and literature synthesis.

DOT National Transportation Integrated Search

2012-03-01

In an era of limited resources and a proliferation of data, there is increasing pressure to conduct careful evaluations of the economic, environmental, and equity effects of investments and policies that influence transportation and land-use systems....

Evaluation of warm mix asphalt technology in flexible pavements.

DOT National Transportation Integrated Search

2009-09-01

The primary goal of this research project is to quantify the performance of field produced and placed mixtures that utilize WMA technology and develop a framework for design, construction, and implementation of this technology in Louisiana. This rese...

Evaluating a Web-Based Educational Module on Oral Cancer Examination Based on a Behavioral Framework.

PubMed

Wee, Alvin G; Zimmerman, Lani M; Pullen, Carol H; Allen, Carl M; Lambert, Paul M; Paskett, Electra D

2016-03-01

Patients at risk of developing oral and/or oropharyngeal cancer (OPC) are more likely to see primary care providers (PCPs) than a dentist. Many PCPs do not regularly perform oral cancer examination (OCE). The purpose of this study was to design a web-based educational program based on a behavioral framework to encourage PCPs to conduct OCE. PCPs were solicited to provide feedback on the program and to evaluate their short-term knowledge. The integrated behavioral model was used to design the program. Fifteen PCPs (five in each group: physicians, physician assistants, and nurse practitioners) reviewed the program and took a posttest: (1) index of knowledge of risk factors for oral cancer (RiskOC) and (2) index of knowledge of diagnostic procedures for oral cancer (DiagOC). Findings from the process evaluation were mainly positive, with comments on the length of the program comprising the ten negative comments. No significant difference among groups of PCPs (physicians, physician assistants, and nurse practitioners) was detected for DiagOC (p = 0.43) or RiskOC (p = 0.201). A program on OPC for PCPs should be less than 40 min. Postviewing knowledge outcomes were similar for all PCPs. The web-based program on OPC based on a behavioral framework could have similar short-term knowledge outcomes for all PCPs and may increase the number of PCPs performing OCEs.

A framework for characterizing eHealth literacy demands and barriers.

PubMed

Chan, Connie V; Kaufman, David R

2011-11-17

Consumer eHealth interventions are of a growing importance in the individual management of health and health behaviors. However, a range of access, resources, and skills barriers prevent health care consumers from fully engaging in and benefiting from the spectrum of eHealth interventions. Consumers may engage in a range of eHealth tasks, such as participating in health discussion forums and entering information into a personal health record. eHealth literacy names a set of skills and knowledge that are essential for productive interactions with technology-based health tools, such as proficiency in information retrieval strategies, and communicating health concepts effectively. We propose a theoretical and methodological framework for characterizing complexity of eHealth tasks, which can be used to diagnose and describe literacy barriers and inform the development of solution strategies. We adapted and integrated two existing theoretical models relevant to the analysis of eHealth literacy into a single framework to systematically categorize and describe task demands and user performance on tasks needed by health care consumers in the information age. The method derived from the framework is applied to (1) code task demands using a cognitive task analysis, and (2) code user performance on tasks. The framework and method are applied to the analysis of a Web-based consumer eHealth task with information-seeking and decision-making demands. We present the results from the in-depth analysis of the task performance of a single user as well as of 20 users on the same task to illustrate both the detailed analysis and the aggregate measures obtained and potential analyses that can be performed using this method. The analysis shows that the framework can be used to classify task demands as well as the barriers encountered in user performance of the tasks. Our approach can be used to (1) characterize the challenges confronted by participants in performing the tasks, (2) determine the extent to which application of the framework to the cognitive task analysis can predict and explain the problems encountered by participants, and (3) inform revisions to the framework to increase accuracy of predictions. The results of this illustrative application suggest that the framework is useful for characterizing task complexity and for diagnosing and explaining barriers encountered in task completion. The framework and analytic approach can be a potentially powerful generative research platform to inform development of rigorous eHealth examination and design instruments, such as to assess eHealth competence, to design and evaluate consumer eHealth tools, and to develop an eHealth curriculum.

A Framework for Characterizing eHealth Literacy Demands and Barriers

PubMed Central

Chan, Connie V

2011-01-01

Background Consumer eHealth interventions are of a growing importance in the individual management of health and health behaviors. However, a range of access, resources, and skills barriers prevent health care consumers from fully engaging in and benefiting from the spectrum of eHealth interventions. Consumers may engage in a range of eHealth tasks, such as participating in health discussion forums and entering information into a personal health record. eHealth literacy names a set of skills and knowledge that are essential for productive interactions with technology-based health tools, such as proficiency in information retrieval strategies, and communicating health concepts effectively. Objective We propose a theoretical and methodological framework for characterizing complexity of eHealth tasks, which can be used to diagnose and describe literacy barriers and inform the development of solution strategies. Methods We adapted and integrated two existing theoretical models relevant to the analysis of eHealth literacy into a single framework to systematically categorize and describe task demands and user performance on tasks needed by health care consumers in the information age. The method derived from the framework is applied to (1) code task demands using a cognitive task analysis, and (2) code user performance on tasks. The framework and method are applied to the analysis of a Web-based consumer eHealth task with information-seeking and decision-making demands. We present the results from the in-depth analysis of the task performance of a single user as well as of 20 users on the same task to illustrate both the detailed analysis and the aggregate measures obtained and potential analyses that can be performed using this method. Results The analysis shows that the framework can be used to classify task demands as well as the barriers encountered in user performance of the tasks. Our approach can be used to (1) characterize the challenges confronted by participants in performing the tasks, (2) determine the extent to which application of the framework to the cognitive task analysis can predict and explain the problems encountered by participants, and (3) inform revisions to the framework to increase accuracy of predictions. Conclusions The results of this illustrative application suggest that the framework is useful for characterizing task complexity and for diagnosing and explaining barriers encountered in task completion. The framework and analytic approach can be a potentially powerful generative research platform to inform development of rigorous eHealth examination and design instruments, such as to assess eHealth competence, to design and evaluate consumer eHealth tools, and to develop an eHealth curriculum. PMID:22094891

Management of children exposed to Mycobacterium tuberculosis: a public health evaluation in West Java, Indonesia.

PubMed

Rutherford, Merrin E; Ruslami, Rovina; Anselmo, Melissa; Alisjahbana, Bachti; Yulianti, Neti; Sampurno, Hedy; van Crevel, Reinout; Hill, Philip C

2013-12-01

To investigate qualitatively and quantitatively the performance of a programme for managing the child contacts of adult tuberculosis patients in Indonesia. A public health evaluation framework was used to assess gaps in a child contact management programme at a lung clinic. Targets for programme performance indicators were derived from established programme indicator targets, the scientific literature and expert opinion. Compliance with tuberculosis screening, the initiation of isoniazid preventive therapy in children younger than 5 years, the accuracy of tuberculosis diagnosis and adherence to preventive therapy were assessed in 755 child contacts in two cohorts. In addition, 22 primary caregivers and 34 clinic staff were interviewed to evaluate knowledge and acceptance of child contact management. The cost to caregivers was recorded. Gaps between observed and target indicator values were quantified. THE GAPS BETWEEN OBSERVED AND TARGET PERFORMANCE INDICATORS WERE: 82% for screening compliance; 64 to 100% for diagnostic accuracy, 50% for the initiation of preventive therapy, 54% for adherence to therapy and 50% for costs. Many staff did not have adequate knowledge of, or an appropriate attitude towards, child contact management, especially regarding isoniazid preventive therapy. Caregivers had good knowledge of screening but not of preventive therapy and had difficulty travelling to the clinic and paying costs. The study identified widespread gaps in the performance of a child contact management system in Indonesia, all of which appear amenable to intervention. The public health evaluation framework used could be applied in other settings where child contact management is failing.

msCompare: A Framework for Quantitative Analysis of Label-free LC-MS Data for Comparative Candidate Biomarker Studies*

PubMed Central

Hoekman, Berend; Breitling, Rainer; Suits, Frank; Bischoff, Rainer; Horvatovich, Peter

2012-01-01

Data processing forms an integral part of biomarker discovery and contributes significantly to the ultimate result. To compare and evaluate various publicly available open source label-free data processing workflows, we developed msCompare, a modular framework that allows the arbitrary combination of different feature detection/quantification and alignment/matching algorithms in conjunction with a novel scoring method to evaluate their overall performance. We used msCompare to assess the performance of workflows built from modules of publicly available data processing packages such as SuperHirn, OpenMS, and MZmine and our in-house developed modules on peptide-spiked urine and trypsin-digested cerebrospinal fluid (CSF) samples. We found that the quality of results varied greatly among workflows, and interestingly, heterogeneous combinations of algorithms often performed better than the homogenous workflows. Our scoring method showed that the union of feature matrices of different workflows outperformed the original homogenous workflows in some cases. msCompare is open source software (https://trac.nbic.nl/mscompare), and we provide a web-based data processing service for our framework by integration into the Galaxy server of the Netherlands Bioinformatics Center (http://galaxy.nbic.nl/galaxy) to allow scientists to determine which combination of modules provides the most accurate processing for their particular LC-MS data sets. PMID:22318370

Measuring performance in off-patent drug markets: a methodological framework and empirical evidence from twelve EU Member States.

PubMed

Kanavos, Panos

2014-11-01

This paper develops a methodological framework to help evaluate the performance of generic pharmaceutical policies post-patent expiry or after loss of exclusivity in non-tendering settings, comprising five indicators (generic availability, time delay to and speed of generic entry, number of generic competitors, price developments, and generic volume share evolution) and proposes a series of metrics to evaluate performance. The paper subsequently tests this framework across twelve EU Member States (MS) by using IMS data on 101 patent expired molecules over the 1998-2010 period. Results indicate that significant variation exists in generic market entry, price competition and generic penetration across the study countries. Size of a geographical market is not a predictor of generic market entry intensity or price decline. Regardless of geographic or product market size, many off patent molecules lack generic competitors two years after loss of exclusivity. The ranges in each of the five proposed indicators suggest, first, that there are numerous factors--including institutional ones--contributing to the success of generic entry, price decline and market penetration and, second, MS should seek a combination of supply and demand-side policies in order to maximise cost-savings from generics. Overall, there seems to be considerable potential for faster generic entry, uptake and greater generic competition, particularly for molecules at the lower end of the market. Copyright © 2014. Published by Elsevier Ireland Ltd.

A Web-Based Common Framework to Support the Test and Evaluation Process Any Time, Anywhere, and Anyhow

DOE Office of Scientific and Technical Information (OSTI.GOV)

Schur, Anne; Brown, James C.; Eaton, Sharon L.

Test and evaluation (T and E) is an enterprise. For any product, large or small, performance data is desired on many aspects to evaluate the product?s effectiveness for the intended users. Representing the many T and E facets without bewildering the user is challenging when there is a range of people, from the system developers to the manager of the organization, that want specific feedback. A web-based One-Stop Evaluation Center was created to meet these needs for a particular project. The evaluation center is usable at any time in the systems development lifecycle and streamlines the T and E enterprise.more » This paper discusses a common framework that unifies the T and E process with many stakeholders involved and is flexible to accommodate each stakeholders?specific evaluative processes and content. Our success has translated to many cost savings by enabling quick responses to change and a better line of communication between the users, developers, and managers.« less

Using a fuzzy comprehensive evaluation method to determine product usability: A test case

PubMed Central

Zhou, Ronggang; Chan, Alan H. S.

2016-01-01

BACKGROUND: In order to take into account the inherent uncertainties during product usability evaluation, Zhou and Chan [1] proposed a comprehensive method of usability evaluation for products by combining the analytic hierarchy process (AHP) and fuzzy evaluation methods for synthesizing performance data and subjective response data. This method was designed to provide an integrated framework combining the inevitable vague judgments from the multiple stages of the product evaluation process. OBJECTIVE AND METHODS: In order to illustrate the effectiveness of the model, this study used a summative usability test case to assess the application and strength of the general fuzzy usability framework. To test the proposed fuzzy usability evaluation framework [1], a standard summative usability test was conducted to benchmark the overall usability of a specific network management software. Based on the test data, the fuzzy method was applied to incorporate both the usability scores and uncertainties involved in the multiple components of the evaluation. Then, with Monte Carlo simulation procedures, confidence intervals were used to compare the reliabilities among the fuzzy approach and two typical conventional methods combining metrics based on percentages. RESULTS AND CONCLUSIONS: This case study showed that the fuzzy evaluation technique can be applied successfully for combining summative usability testing data to achieve an overall usability quality for the network software evaluated. Greater differences of confidence interval widths between the method of averaging equally percentage and weighted evaluation method, including the method of weighted percentage averages, verified the strength of the fuzzy method. PMID:28035942

Using a fuzzy comprehensive evaluation method to determine product usability: A test case.

PubMed

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to take into account the inherent uncertainties during product usability evaluation, Zhou and Chan [1] proposed a comprehensive method of usability evaluation for products by combining the analytic hierarchy process (AHP) and fuzzy evaluation methods for synthesizing performance data and subjective response data. This method was designed to provide an integrated framework combining the inevitable vague judgments from the multiple stages of the product evaluation process. In order to illustrate the effectiveness of the model, this study used a summative usability test case to assess the application and strength of the general fuzzy usability framework. To test the proposed fuzzy usability evaluation framework [1], a standard summative usability test was conducted to benchmark the overall usability of a specific network management software. Based on the test data, the fuzzy method was applied to incorporate both the usability scores and uncertainties involved in the multiple components of the evaluation. Then, with Monte Carlo simulation procedures, confidence intervals were used to compare the reliabilities among the fuzzy approach and two typical conventional methods combining metrics based on percentages. This case study showed that the fuzzy evaluation technique can be applied successfully for combining summative usability testing data to achieve an overall usability quality for the network software evaluated. Greater differences of confidence interval widths between the method of averaging equally percentage and weighted evaluation method, including the method of weighted percentage averages, verified the strength of the fuzzy method.

Evaluating time to cancer recurrence as a surrogate marker for survival from an information theory perspective.

PubMed

Alonso, Ariel; Molenberghs, Geert

2008-10-01

The last two decades have seen a lot of development in the area of surrogate marker validation. One of these approaches places the evaluation in a meta-analytic framework, leading to definitions in terms of trial- and individual-level association. A drawback of this methodology is that different settings have led to different measures at the individual level. Using information theory, Alonso et al. proposed a unified framework, leading to a new definition of surrogacy, which offers interpretational advantages and is applicable in a wide range of situations. In this work, we illustrate how this information-theoretic approach can be used to evaluate surrogacy when both endpoints are of a time-to-event type. Two meta-analyses, in early and advanced colon cancer, respectively, are then used to evaluate the performance of time to cancer recurrence as a surrogate for overall survival.

The Partners in Prevention Program: The Evaluation and Evolution of the Task-Centered Case Management Model

ERIC Educational Resources Information Center

Colvin, Julanne; Lee, Mingun; Magnano, Julienne; Smith, Valerie

2008-01-01

This article reports on the further development of the task-centered model for difficulties in school performance. We used Bailey-Dempsey and Reid's (1996) application of Rothman and Thomas's (1994) design and development framework and annual evaluations of the Partners in Prevention (PIP) Program to refine the task-centered case management model.…

Civil Affairs Framework For Executing Governance

DTIC Science & Technology

2017-12-01

evaluation (p. 3). An example of performance management is the evaluation of the above school construction project beginning with receiving notification...Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management ...and Budget, Paperwork Reduction Project (0704-0188) Washington, DC 20503. 1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE December 2017 3

Bibliometrics as a Performance Measurement Tool for Research Evaluation: The Case of Research Funded by the National Cancer Institute of Canada

ERIC Educational Resources Information Center

Campbell, David; Picard-Aitken, Michelle; Cote, Gregoire; Caruso, Julie; Valentim, Rodolfo; Edmonds, Stuart; Williams, Gregory Thomas; Macaluso, Benoit; Robitaille, Jean-Pierre; Bastien, Nicolas; Laframboise, Marie-Claude; Lebeau, Louis-Michel; Mirabel, Philippe; Lariviere, Vincent; Archambault, Eric

2010-01-01

As bibliometric indicators are objective, reliable, and cost-effective measures of peer-reviewed research outputs, they are expected to play an increasingly important role in research assessment/management. Recently, a bibliometric approach was developed and integrated within the evaluation framework of research funded by the National Cancer…

Comparison of Expert-Based and Empirical Evaluation Methodologies in the Case of a CBL Environment: The ''Orestis'' Experience

ERIC Educational Resources Information Center

Karoulis, Athanasis; Demetriadis, Stavros; Pombortsis, Andreas

2006-01-01

This paper compares several interface evaluation methods applied in the case of a computer based learning (CBL) environment, during a longitudinal study performed in three European countries, Greece, Germany, and Holland, and within the framework of an EC funded Leonardo da Vinci program. The paper firstly considers the particularities of the CBL…

A lack of standardization: the basis for the ethical issues surrounding quality and performance reports.

PubMed

Suchy, Kirsten

2010-01-01

Consumers in the United States are taking advantage of the proliferation of publicly available, internet-based performance reports and quality appraisals of health plans, healthcare organizations, hospitals, and physicians to aid in their healthcare decision making. However, these appraisal practices have given rise to controversy and debate over certain distinctive ethical issues. This article advocates a standardized ethical framework to guide current and future development and implementation of performance reports. This framework, which would resolve a number of the major issues, includes the following ethical principles to guide the practice of public reporting on the Internet and facilitate enhanced quality improvement in the healthcare industry: legitimacy, data integrity and quality, transparency, informed understanding, equity, privacy and confidentiality, collaboration, accountability, and evaluation and continuous improvement.

Gathering Validity Evidence for Surgical Simulation: A Systematic Review.

PubMed

Borgersen, Nanna Jo; Naur, Therese M H; Sørensen, Stine M D; Bjerrum, Flemming; Konge, Lars; Subhi, Yousif; Thomsen, Ann Sofia S

2018-06-01

To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.

Design and Evaluation of a Scalable and Reconfigurable Multi-Platform System for Acoustic Imaging

PubMed Central

Izquierdo, Alberto; Villacorta, Juan José; del Val Puente, Lara; Suárez, Luis

2016-01-01

This paper proposes a scalable and multi-platform framework for signal acquisition and processing, which allows for the generation of acoustic images using planar arrays of MEMS (Micro-Electro-Mechanical Systems) microphones with low development and deployment costs. Acoustic characterization of MEMS sensors was performed, and the beam pattern of a module, based on an 8 × 8 planar array and of several clusters of modules, was obtained. A flexible framework, formed by an FPGA, an embedded processor, a computer desktop, and a graphic processing unit, was defined. The processing times of the algorithms used to obtain the acoustic images, including signal processing and wideband beamforming via FFT, were evaluated in each subsystem of the framework. Based on this analysis, three frameworks are proposed, defined by the specific subsystems used and the algorithms shared. Finally, a set of acoustic images obtained from sound reflected from a person are presented as a case study in the field of biometric identification. These results reveal the feasibility of the proposed system. PMID:27727174

Developing weighted criteria to evaluate lean reverse logistics through analytical network process

NASA Astrophysics Data System (ADS)

Zagloel, Teuku Yuri M.; Hakim, Inaki Maulida; Krisnawardhani, Rike Adyartie

2017-11-01

Reverse logistics is a part of supply chain that bring materials from consumers back to manufacturer in order to gain added value or do a proper disposal. Nowadays, most companies are still facing several problems on reverse logistics implementation which leads to high waste along reverse logistics processes. In order to overcome this problem, Madsen [Framework for Reverse Lean Logistics to Enable Green Manufacturing, Eco Design 2009: 6th International Symposium on Environmentally Conscious Design and Inverse Manufacturing, Sapporo, 2009] has developed a lean reverse logistics framework as a step to eliminate waste by implementing lean on reverse logistics. However, the resulted framework sets aside criteria used to evaluate its performance. This research aims to determine weighted criteria that can be used as a base on reverse logistics evaluation by considering lean principles. The resulted criteria will ensure reverse logistics are kept off from waste, thus implemented efficiently. Analytical Network Process (ANP) is used in this research to determine the weighted criteria. The result shows that criteria used for evaluation lean reverse logistics are Innovation and Learning (35%), Economic (30%), Process Flow Management (14%), Customer Relationship Management (13%), Environment (6%), and Social (2%).

Evaluating auditory perception and communication demands required to carry out work tasks and complimentary hearing resources and skills for older workers with hearing loss.

PubMed

Jennings, M B; Shaw, L; Hodgins, H; Kuchar, D A; Bataghva, L Poost-Foroosh

2010-01-01

For older workers with acquired hearing loss, this loss as well as the changing nature of work and the workforce, may lead to difficulties and disadvantages in obtaining and maintaining employment. Currently there are very few instruments that can assist workplaces, employers and workers to prepare for older workers with hearing loss or with the evaluation of auditory perception demands of work, especially those relevant to communication, and safety sensitive workplaces that require high levels of communication. This paper introduces key theoretical considerations that informed the development of a new framework, The Audiologic Ergonomic (AE) Framework to guide audiologists, work rehabilitation professionals and workers in developing tools to support the identification and evaluation of auditory perception demands in the workplace, the challenges to communication and the subsequent productivity and safety in the performance of work duties by older workers with hearing loss. The theoretical concepts underpinning this framework are discussed along with next steps in developing tools such as the Canadian Hearing Demands Tool (C-HearD Tool) in advancing approaches to evaluate auditory perception and communication demands in the workplace.

Evaluating the performance of infectious disease forecasts: A comparison of climate-driven and seasonal dengue forecasts for Mexico.

PubMed

Johansson, Michael A; Reich, Nicholas G; Hota, Aditi; Brownstein, John S; Santillana, Mauricio

2016-09-26

Dengue viruses, which infect millions of people per year worldwide, cause large epidemics that strain healthcare systems. Despite diverse efforts to develop forecasting tools including autoregressive time series, climate-driven statistical, and mechanistic biological models, little work has been done to understand the contribution of different components to improved prediction. We developed a framework to assess and compare dengue forecasts produced from different types of models and evaluated the performance of seasonal autoregressive models with and without climate variables for forecasting dengue incidence in Mexico. Climate data did not significantly improve the predictive power of seasonal autoregressive models. Short-term and seasonal autocorrelation were key to improving short-term and long-term forecasts, respectively. Seasonal autoregressive models captured a substantial amount of dengue variability, but better models are needed to improve dengue forecasting. This framework contributes to the sparse literature of infectious disease prediction model evaluation, using state-of-the-art validation techniques such as out-of-sample testing and comparison to an appropriate reference model.

Evaluating the performance of infectious disease forecasts: A comparison of climate-driven and seasonal dengue forecasts for Mexico

PubMed Central

Johansson, Michael A.; Reich, Nicholas G.; Hota, Aditi; Brownstein, John S.; Santillana, Mauricio

2016-01-01

Dengue viruses, which infect millions of people per year worldwide, cause large epidemics that strain healthcare systems. Despite diverse efforts to develop forecasting tools including autoregressive time series, climate-driven statistical, and mechanistic biological models, little work has been done to understand the contribution of different components to improved prediction. We developed a framework to assess and compare dengue forecasts produced from different types of models and evaluated the performance of seasonal autoregressive models with and without climate variables for forecasting dengue incidence in Mexico. Climate data did not significantly improve the predictive power of seasonal autoregressive models. Short-term and seasonal autocorrelation were key to improving short-term and long-term forecasts, respectively. Seasonal autoregressive models captured a substantial amount of dengue variability, but better models are needed to improve dengue forecasting. This framework contributes to the sparse literature of infectious disease prediction model evaluation, using state-of-the-art validation techniques such as out-of-sample testing and comparison to an appropriate reference model. PMID:27665707

Strengthening and sustainability of national immunization technical advisory groups (NITAGs) globally: Lessons and recommendations from the founding meeting of the global NITAG network.

PubMed

Adjagba, Alex; MacDonald, Noni E; Ortega-Pérez, Inmaculada; Duclos, Philippe

2017-05-25

National Immunization Technical Advisory Groups (NITAGs) provide independent, evidence-informed advice to assist their governments in immunization policy formation. However, many NITAGs face challenges in fulfilling their roles. Hence the many requests for formation of a network linking NITAGs together so they can learn from each other. To address this request, the Health Policy and Institutional Development (HPID) Center (a WHO Collaborating Center at the Agence de Médecine Préventive - AMP), in collaboration with WHO, organized a meeting in Veyrier-du-Lac, France, on 11 and 12 May 2016, to establish a Global NITAG Network (GNN). The meeting focused on two areas: the requirements for (a) the establishment of a global NITAG collaborative network; and (b) the global assessment/evaluation of the performance of NITAGs. 35 participants from 26 countries reviewed the proposed GNN framework documents and NITAG performance evaluation. Participants recommended that a GNN should be established, agreed on its governance, function, scope and a proposed work plan as well as setting a framework for NITAG evaluation. Copyright © 2017.

Evaluating a multispecies adaptive management framework: Must uncertainty impede effective decision-making?

USGS Publications Warehouse

Smith, David R.; McGowan, Conor P.; Daily, Jonathan P.; Nichols, James D.; Sweka, John A.; Lyons, James E.

2013-01-01

Application of adaptive management to complex natural resource systems requires careful evaluation to ensure that the process leads to improved decision-making. As part of that evaluation, adaptive policies can be compared with alternative nonadaptive management scenarios. Also, the value of reducing structural (ecological) uncertainty to achieving management objectives can be quantified.A multispecies adaptive management framework was recently adopted by the Atlantic States Marine Fisheries Commission for sustainable harvest of Delaware Bay horseshoe crabs Limulus polyphemus, while maintaining adequate stopover habitat for migrating red knots Calidris canutus rufa, the focal shorebird species. The predictive model set encompassed the structural uncertainty in the relationships between horseshoe crab spawning, red knot weight gain and red knot vital rates. Stochastic dynamic programming was used to generate a state-dependent strategy for harvest decisions given that uncertainty. In this paper, we employed a management strategy evaluation approach to evaluate the performance of this adaptive management framework. Active adaptive management was used by including model weights as state variables in the optimization and reducing structural uncertainty by model weight updating.We found that the value of information for reducing structural uncertainty is expected to be low, because the uncertainty does not appear to impede effective management. Harvest policy responded to abundance levels of both species regardless of uncertainty in the specific relationship that generated those abundances. Thus, the expected horseshoe crab harvest and red knot abundance were similar when the population generating model was uncertain or known, and harvest policy was robust to structural uncertainty as specified.Synthesis and applications. The combination of management strategy evaluation with state-dependent strategies from stochastic dynamic programming was an informative approach to evaluate adaptive management performance and value of learning. Although natural resource decisions are characterized by uncertainty, not all uncertainty will cause decisions to be altered substantially, as we found in this case. It is important to incorporate uncertainty into the decision framing and evaluate the effect of reducing that uncertainty on achieving the desired outcomes

Comparison of hand-craft feature based SVM and CNN based deep learning framework for automatic polyp classification.

PubMed

Younghak Shin; Balasingham, Ilangko

2017-07-01

Colonoscopy is a standard method for screening polyps by highly trained physicians. Miss-detected polyps in colonoscopy are potential risk factor for colorectal cancer. In this study, we investigate an automatic polyp classification framework. We aim to compare two different approaches named hand-craft feature method and convolutional neural network (CNN) based deep learning method. Combined shape and color features are used for hand craft feature extraction and support vector machine (SVM) method is adopted for classification. For CNN approach, three convolution and pooling based deep learning framework is used for classification purpose. The proposed framework is evaluated using three public polyp databases. From the experimental results, we have shown that the CNN based deep learning framework shows better classification performance than the hand-craft feature based methods. It achieves over 90% of classification accuracy, sensitivity, specificity and precision.

Design of the HELICS High-Performance Transmission-Distribution-Communication-Market Co-Simulation Framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Palmintier, Bryan S; Krishnamurthy, Dheepak; Top, Philip

This paper describes the design rationale for a new cyber-physical-energy co-simulation framework for electric power systems. This new framework will support very large-scale (100,000+ federates) co-simulations with off-the-shelf power-systems, communication, and end-use models. Other key features include cross-platform operating system support, integration of both event-driven (e.g. packetized communication) and time-series (e.g. power flow) simulation, and the ability to co-iterate among federates to ensure model convergence at each time step. After describing requirements, we begin by evaluating existing co-simulation frameworks, including HLA and FMI, and conclude that none provide the required features. Then we describe the design for the new layeredmore » co-simulation architecture.« less

Design of the HELICS High-Performance Transmission-Distribution-Communication-Market Co-Simulation Framework: Preprint

DOE Office of Scientific and Technical Information (OSTI.GOV)

Palmintier, Bryan S; Krishnamurthy, Dheepak; Top, Philip

This paper describes the design rationale for a new cyber-physical-energy co-simulation framework for electric power systems. This new framework will support very large-scale (100,000+ federates) co-simulations with off-the-shelf power-systems, communication, and end-use models. Other key features include cross-platform operating system support, integration of both event-driven (e.g. packetized communication) and time-series (e.g. power flow) simulation, and the ability to co-iterate among federates to ensure model convergence at each time step. After describing requirements, we begin by evaluating existing co-simulation frameworks, including HLA and FMI, and conclude that none provide the required features. Then we describe the design for the new layeredmore » co-simulation architecture.« less

Framework for Architecture Trade Study Using MBSE and Performance Simulation

NASA Technical Reports Server (NTRS)

Ryan, Jessica; Sarkani, Shahram; Mazzuchim, Thomas

2012-01-01

Increasing complexity in modern systems as well as cost and schedule constraints require a new paradigm of system engineering to fulfill stakeholder needs. Challenges facing efficient trade studies include poor tool interoperability, lack of simulation coordination (design parameters) and requirements flowdown. A recent trend toward Model Based System Engineering (MBSE) includes flexible architecture definition, program documentation, requirements traceability and system engineering reuse. As a new domain MBSE still lacks governing standards and commonly accepted frameworks. This paper proposes a framework for efficient architecture definition using MBSE in conjunction with Domain Specific simulation to evaluate trade studies. A general framework is provided followed with a specific example including a method for designing a trade study, defining candidate architectures, planning simulations to fulfill requirements and finally a weighted decision analysis to optimize system objectives.

Extending the Distributed Lag Model framework to handle chemical mixtures.

PubMed

Bello, Ghalib A; Arora, Manish; Austin, Christine; Horton, Megan K; Wright, Robert O; Gennings, Chris

2017-07-01

Distributed Lag Models (DLMs) are used in environmental health studies to analyze the time-delayed effect of an exposure on an outcome of interest. Given the increasing need for analytical tools for evaluation of the effects of exposure to multi-pollutant mixtures, this study attempts to extend the classical DLM framework to accommodate and evaluate multiple longitudinally observed exposures. We introduce 2 techniques for quantifying the time-varying mixture effect of multiple exposures on an outcome of interest. Lagged WQS, the first technique, is based on Weighted Quantile Sum (WQS) regression, a penalized regression method that estimates mixture effects using a weighted index. We also introduce Tree-based DLMs, a nonparametric alternative for assessment of lagged mixture effects. This technique is based on the Random Forest (RF) algorithm, a nonparametric, tree-based estimation technique that has shown excellent performance in a wide variety of domains. In a simulation study, we tested the feasibility of these techniques and evaluated their performance in comparison to standard methodology. Both methods exhibited relatively robust performance, accurately capturing pre-defined non-linear functional relationships in different simulation settings. Further, we applied these techniques to data on perinatal exposure to environmental metal toxicants, with the goal of evaluating the effects of exposure on neurodevelopment. Our methods identified critical neurodevelopmental windows showing significant sensitivity to metal mixtures. Copyright © 2017 Elsevier Inc. All rights reserved.

A framework for evaluating the formation, implementation, and performance of accountable care organizations.

PubMed

Fisher, Elliott S; Shortell, Stephen M; Kreindler, Sara A; Van Citters, Aricca D; Larson, Bridget K

2012-11-01

The implementation of accountable care organizations (ACOs), a new health care payment and delivery model designed to improve care and lower costs, is proceeding rapidly. We build on our experience tracking early ACOs to identify the major factors-such as contract characteristics; structure, capabilities, and activities; and local context-that would be likely to influence ACO formation, implementation, and performance. We then propose how an ACO evaluation program could be structured to guide policy makers and payers in improving the design of ACO contracts, while providing insights for providers on approaches to care transformation that are most likely to be successful in different contexts. We also propose key activities to support evaluation of ACOs in the near term, including tracking their formation, developing a set of performance measures across all ACOs and payers, aggregating those performance data, conducting qualitative and quantitative research, and coordinating different evaluation activities.

Evaluation of the Performance Driven Budget Initiative of the New York City Board of Education. First Annual Report.

ERIC Educational Resources Information Center

Siegel, Dorothy; Naphtali, Zvia Segal; Fruchter, Norm; Berne, Robert

In 1996 the Chancellor introduced Performance Driven Budgeting (PDB) to the New York City schools. PDB is a form of decentralized budgetary decision making intended to provide local educators with increased control and flexibility over the use of resources. The plan established a framework of goals and principles, outlined a phased-in…

A framework for performance and data quality assessment of Radio Frequency IDentification (RFID) systems in health care settings.

PubMed

van der Togt, Remko; Bakker, Piet J M; Jaspers, Monique W M

2011-04-01

RFID offers great opportunities to health care. Nevertheless, prior experiences also show that RFID systems have not been designed and tested in response to the particular needs of health care settings and might introduce new risks. The aim of this study is to present a framework that can be used to assess the performance of RFID systems particularly in health care settings. We developed a framework describing a systematic approach that can be used for assessing the feasibility of using an RFID technology in a particular healthcare setting; more specific for testing the impact of environmental factors on the quality of RFID generated data and vice versa. This framework is based on our own experiences with an RFID pilot implementation in an academic hospital in The Netherlands and a literature review concerning RFID test methods and current insights of RFID implementations in healthcare. The implementation of an RFID system within the blood transfusion chain inside a hospital setting was used as a show case to explain the different phases of the framework. The framework consists of nine phases, including an implementation development plan, RFID and medical equipment interference tests, data accuracy- and data completeness tests to be run in laboratory, simulated field and real field settings. The potential risks that RFID technologies may bring to the healthcare setting should be thoroughly evaluated before they are introduced into a vital environment. The RFID performance assessment framework that we present can act as a reference model to start an RFID development, engineering, implementation and testing plan and more specific, to assess the potential risks of interference and to test the quality of the RFID generated data potentially influenced by physical objects in specific health care environments. Copyright © 2010 Elsevier Inc. All rights reserved.

Quality Evaluation of Zirconium Dioxide Frameworks Produced in Five Dental Laboratories from Different Countries.

PubMed

Schneebeli, Esther; Brägger, Urs; Scherrer, Susanne S; Keller, Andrea; Wittneben, Julia G; Hicklin, Stefan P

2017-07-01

The aim of this study was to assess and compare quality as well as economic aspects of CAD/CAM high strength ceramic three-unit FDP frameworks ordered from dental laboratories located in emerging countries and Switzerland. The master casts of six cases were sent to five dental laboratories located in Thailand (Bangkok), China (Peking and Shenzhen), Turkey (Izmir), and Switzerland (Bern). Each laboratory was using a different CAD/CAM system. The clinical fit of the frameworks was qualitatively assessed, and the thickness of the framework material, the connector height, the width, and the diameter were evaluated using a measuring sensor. The analysis of the internal fit of the frameworks was performed by means of a replica technique, whereas the inner and outer surfaces of the frameworks were evaluated for traces of postprocessing and damage to the intaglio surface with light and electronic microscopes. Groups (dental laboratories and cases) were compared for statistically significant differences using Mann-Whitney U-tests after Bonferroni correction. An acceptable clinical fit was found at 97.9% of the margins produced in laboratory E, 87.5% in B, 93.7% in C, 79.2% in A, and 62.5% in D. The mean framework thicknesses were not statistically significantly different for the premolar regions; however, for the molar area 4/8 of the evaluated sites were statistically significantly different. Circumference, surface, and width of the connectors produced in the different laboratories were statistically significantly different but not the height. There were great differences in the designs for the pontic and connector regions, and some of the frameworks would not be recommended for clinical use. Traces of heavy postprocessing were found in frameworks from some of the laboratories. The prices per framework ranged from US$177 to US$896. By ordering laboratory work in developing countries, a considerable price reduction was obtained compared to the price level in Switzerland. Despite the use of the standardized CAD/CAM chains of production in all laboratories, a large variability in the quality aspects, such as clinical marginal fit, connector and pontic design, as well as postprocessing traces was noted. Recommended sound handling of postprocessing was not applied in all laboratories. Dentists should be aware of the true and factitious advantages of CAD/CAM production chains and not lose control over the process. © 2015 by the American College of Prosthodontists.

A Sustainable Evaluation Framework and Its Application

ERIC Educational Resources Information Center

Powell, Robert B.; Stern, Marc J.; Ardoin, Nicole

2006-01-01

This article presents a framework for developing internally sustainable evaluation systems for environmental education organizations, although the framework can be applied to other types of organizations. The authors developed a sustainable evaluation framework (SEF) with the intent of creating an evaluation system that could be self-administered…

An urban energy performance evaluation system and its computer implementation.

PubMed

Wang, Lei; Yuan, Guan; Long, Ruyin; Chen, Hong

2017-12-15

To improve the urban environment and effectively reflect and promote urban energy performance, an urban energy performance evaluation system was constructed, thereby strengthening urban environmental management capabilities. From the perspectives of internalization and externalization, a framework of evaluation indicators and key factors that determine urban energy performance and explore the reasons for differences in performance was proposed according to established theory and previous studies. Using the improved stochastic frontier analysis method, an urban energy performance evaluation and factor analysis model was built that brings performance evaluation and factor analysis into the same stage for study. According to data obtained for the Chinese provincial capitals from 2004 to 2013, the coefficients of the evaluation indicators and key factors were calculated by the urban energy performance evaluation and factor analysis model. These coefficients were then used to compile the program file. The urban energy performance evaluation system developed in this study was designed in three parts: a database, a distributed component server, and a human-machine interface. Its functions were designed as login, addition, edit, input, calculation, analysis, comparison, inquiry, and export. On the basis of these contents, an urban energy performance evaluation system was developed using Microsoft Visual Studio .NET 2015. The system can effectively reflect the status of and any changes in urban energy performance. Beijing was considered as an example to conduct an empirical study, which further verified the applicability and convenience of this evaluation system. Copyright © 2017 Elsevier Ltd. All rights reserved.

A Quantitative Human Spacecraft Design Evaluation Model for Assessing Crew Accommodation and Utilization

NASA Astrophysics Data System (ADS)

Fanchiang, Christine

Crew performance, including both accommodation and utilization factors, is an integral part of every human spaceflight mission from commercial space tourism, to the demanding journey to Mars and beyond. Spacecraft were historically built by engineers and technologists trying to adapt the vehicle into cutting edge rocketry with the assumption that the astronauts could be trained and will adapt to the design. By and large, that is still the current state of the art. It is recognized, however, that poor human-machine design integration can lead to catastrophic and deadly mishaps. The premise of this work relies on the idea that if an accurate predictive model exists to forecast crew performance issues as a result of spacecraft design and operations, it can help designers and managers make better decisions throughout the design process, and ensure that the crewmembers are well-integrated with the system from the very start. The result should be a high-quality, user-friendly spacecraft that optimizes the utilization of the crew while keeping them alive, healthy, and happy during the course of the mission. Therefore, the goal of this work was to develop an integrative framework to quantitatively evaluate a spacecraft design from the crew performance perspective. The approach presented here is done at a very fundamental level starting with identifying and defining basic terminology, and then builds up important axioms of human spaceflight that lay the foundation for how such a framework can be developed. With the framework established, a methodology for characterizing the outcome using a mathematical model was developed by pulling from existing metrics and data collected on human performance in space. Representative test scenarios were run to show what information could be garnered and how it could be applied as a useful, understandable metric for future spacecraft design. While the model is the primary tangible product from this research, the more interesting outcome of this work is the structure of the framework and what it tells future researchers in terms of where the gaps and limitations exist for developing a better framework. It also identifies metrics that can now be collected as part of future validation efforts for the model.

Rebooting Kirkpatrick: Integrating Information System Theory Into the Evaluation of Web-based Continuing Professional Development Interventions for Interprofessional Education.

PubMed

Shen, Nelson; Yufe, Shira; Saadatfard, Omid; Sockalingam, Sanjeev; Wiljer, David

2017-01-01

Information system research has stressed the importance of theory in understanding how user perceptions can motivate the use and adoption of technology such as web-based continuing professional development programs for interprofessional education (WCPD-IPE). A systematic review was conducted to provide an information system perspective on the current state of WCPD-IPE program evaluation and how current evaluations capture essential theoretical constructs in promoting technology adoption. Six databases were searched to identify studies evaluating WCPD-IPE. Three investigators determined eligibility of the articles. Evaluation items extracted from the studies were assessed using the Kirkpatrick-Barr framework and mapped to the Benefits Evaluation Framework. Thirty-seven eligible studies yielded 362 evaluation items for analysis. Most items (n = 252) were assessed as Kirkpatrick-Barr level 1 (reaction) and were mainly focused on the quality (information, service, and quality) and satisfaction dimensions of the Benefits Evaluation. System quality was the least evaluated quality dimension, accounting for 26 items across 13 studies. WCPD-IPE use was reported in 17 studies and its antecedent factors were evaluated in varying degrees of comprehensiveness. Although user reactions were commonly evaluated, greater focus on user perceptions of system quality (ie, functionality and performance), usefulness, and usability of the web-based platform is required. Surprisingly, WCPD-IPE use was reported in less than half of the studies. This is problematic as use is a prerequisite to realizing any individual, organizational, or societal benefit of WCPD-IPE. This review proposes an integrated framework which accounts for these factors and provides a theoretically grounded guide for future evaluations.

Conceptual frameworks of individual work performance: a systematic review.

PubMed

Koopmans, Linda; Bernaards, Claire M; Hildebrandt, Vincent H; Schaufeli, Wilmar B; de Vet Henrica, C W; van der Beek, Allard J

2011-08-01

Individual work performance is differently conceptualized and operationalized in different disciplines. The aim of the current review was twofold: (1) identifying conceptual frameworks of individual work performance and (2) integrating these to reach a heuristic conceptual framework. A systematic review was conducted in medical, psychological, and management databases. Studies were selected independently by two researchers and included when they presented a conceptual framework of individual work performance. A total of 17 generic frameworks (applying across occupations) and 18 job-specific frameworks (applying to specific occupations) were identified. Dimensions frequently used to describe individual work performance were task performance, contextual performance, counterproductive work behavior, and adaptive performance. On the basis of the literature, a heuristic conceptual framework of individual work performance was proposed. This framework can serve as a theoretical basis for future research and practice.

Automatic Structural Parcellation of Mouse Brain MRI Using Multi-Atlas Label Fusion

PubMed Central

Ma, Da; Cardoso, Manuel J.; Modat, Marc; Powell, Nick; Wells, Jack; Holmes, Holly; Wiseman, Frances; Tybulewicz, Victor; Fisher, Elizabeth; Lythgoe, Mark F.; Ourselin, Sébastien

2014-01-01

Multi-atlas segmentation propagation has evolved quickly in recent years, becoming a state-of-the-art methodology for automatic parcellation of structural images. However, few studies have applied these methods to preclinical research. In this study, we present a fully automatic framework for mouse brain MRI structural parcellation using multi-atlas segmentation propagation. The framework adopts the similarity and truth estimation for propagated segmentations (STEPS) algorithm, which utilises a locally normalised cross correlation similarity metric for atlas selection and an extended simultaneous truth and performance level estimation (STAPLE) framework for multi-label fusion. The segmentation accuracy of the multi-atlas framework was evaluated using publicly available mouse brain atlas databases with pre-segmented manually labelled anatomical structures as the gold standard, and optimised parameters were obtained for the STEPS algorithm in the label fusion to achieve the best segmentation accuracy. We showed that our multi-atlas framework resulted in significantly higher segmentation accuracy compared to single-atlas based segmentation, as well as to the original STAPLE framework. PMID:24475148

Conceptual framework for development of comprehensive e-health evaluation tool.

PubMed

Khoja, Shariq; Durrani, Hammad; Scott, Richard E; Sajwani, Afroz; Piryani, Usha

2013-01-01

The main objective of this study was to develop an e-health evaluation tool based on a conceptual framework including relevant theories for evaluating use of technology in health programs. This article presents the development of an evaluation framework for e-health programs. The study was divided into three stages: Stage 1 involved a detailed literature search of different theories and concepts on evaluation of e-health, Stage 2 plotted e-health theories to identify relevant themes, and Stage 3 developed a matrix of evaluation themes and stages of e-health programs. The framework identifies and defines different stages of e-health programs and then applies evaluation theories to each of these stages for development of the evaluation tool. This framework builds on existing theories of health and technology evaluation and presents a conceptual framework for developing an e-health evaluation tool to examine and measure different factors that play a definite role in the success of e-health programs. The framework on the horizontal axis divides e-health into different stages of program implementation, while the vertical axis identifies different themes and areas of consideration for e-health evaluation. The framework helps understand various aspects of e-health programs and their impact that require evaluation at different stages of the life cycle. The study led to the development of a new and comprehensive e-health evaluation tool, named the Khoja-Durrani-Scott Framework for e-Health Evaluation.

A sensor simulation framework for the testing and evaluation of external hazard monitors and integrated alerting and notification functions

NASA Astrophysics Data System (ADS)

Uijt de Haag, Maarten; Venable, Kyle; Bezawada, Rajesh; Adami, Tony; Vadlamani, Ananth K.

2009-05-01

This paper discusses a sensor simulator/synthesizer framework that can be used to test and evaluate various sensor integration strategies for the implementation of an External Hazard Monitor (EHM) and Integrated Alerting and Notification (IAN) function as part of NASA's Integrated Intelligent Flight Deck (IIFD) project. The IIFD project under the NASA's Aviation Safety program "pursues technologies related to the flight deck that ensure crew workload and situational awareness are both safely optimized and adapted to the future operational environment as envisioned by NextGen." Within the simulation framework, various inputs to the IIFD and its subsystems, the EHM and IAN, are simulated, synthesized from actual collected data, or played back from actual flight test sensor data. Sensors and avionics included in this framework are TCAS, ADS-B, Forward-Looking Infrared, Vision cameras, GPS, Inertial navigators, EGPWS, Laser Detection and Ranging sensors, altimeters, communication links with ATC, and weather radar. The framework is implemented in Simulink, a modeling language developed by The Mathworks. This modeling language allows for test and evaluation of various sensor and communication link configurations as well as the inclusion of feedback from the pilot on the performance of the aircraft. Specifically, this paper addresses the architecture of the simulator, the sensor model interfaces, the timing and database (environment) aspects of the sensor models, the user interface of the modeling environment, and the various avionics implementations.

The STEP database through the end-users eyes--USABILITY STUDY.

PubMed

Salunke, Smita; Tuleu, Catherine

2015-08-15

The user-designed database of Safety and Toxicity of Excipients for Paediatrics ("STEP") is created to address the shared need of drug development community to access the relevant information of excipients effortlessly. Usability testing was performed to validate if the database satisfies the need of the end-users. Evaluation framework was developed to assess the usability. The participants performed scenario based tasks and provided feedback and post-session usability ratings. Failure Mode Effect Analysis (FMEA) was performed to prioritize the problems and improvements to the STEP database design and functionalities. The study revealed several design vulnerabilities. Tasks such as limiting the results, running complex queries, location of data and registering to access the database were challenging. The three critical attributes identified to have impact on the usability of the STEP database included (1) content and presentation (2) the navigation and search features (3) potential end-users. Evaluation framework proved to be an effective method for evaluating database effectiveness and user satisfaction. This study provides strong initial support for the usability of the STEP database. Recommendations would be incorporated into the refinement of the database to improve its usability and increase user participation towards the advancement of the database. Copyright © 2015 Elsevier B.V. All rights reserved.

[Development of a Conceptual Framework for the Assessment of Chronic Care in the Spanish National Health System].

PubMed

Espallargues, Mireia; Serra-Sutton, Vicky; Solans-Domènech, Maite; Torrente, Elena; Moharra, Montse; Benítez, Dolors; Robles, Noemí; Domíngo, Laia; Escarrabill Sanglas, Joan

2016-07-07

The aim was to develop a conceptual framework for the assessment of new healthcare initiatives on chronic diseases within the Spanish National Health System. A comprehensive literature review between 2002 and 2013, including systematic reviews, meta-analysis, and reports with evaluation frameworks and/or assessment of initiatives was carried out; integrated care initiatives established in Catalonia were studied and described; and semistructured interviews with key stakeholders were performed. The scope and conceptual framework were defined by using the brainstorming approach.Of 910 abstracts identified, a total of 116 studies were included. They referred to several conceptual frameworks and/or assessment indicators at a national and international level. An overall of 24 established chronic care initiatives were identified (9 integrated care initiatives); 10 in-depth interviews were carried out. The proposed conceptual framework envisages: 1)the target population according to complexity levels; 2)an evaluation approach of the structure, processes, and outcomes considering the health status achieved, the recovery process and the maintenance of health; and 3)the dimensions or attributes to be assessed. The proposed conceptual framework will be helpful has been useful to develop indicators and implement them with a community-based and result-oriented approach and a territorial or population-based perspective within the Spanish Health System. This will be essential to know which are the most effective strategies, what are the key elements that determine greater success and what are the groups of patients who can most benefit.

Develop nondestructive rapid pavement quality assurance/quality control evaluation test methods and supporting technology : project summary.

DOT National Transportation Integrated Search

2017-01-01

The findings from the proof of concept with mechanics-based models for flexible base suggest additional validation work should be performed, draft construction specification frameworks should be developed, and work extending the technology to stabili...

Develop nondestructive rapid pavement quality Assurance/quality control evaluation test methods and supporting technology : project summary.

DOT National Transportation Integrated Search

2017-01-01

The findings from the proof of concept with mechanics-based models for flexible base suggest additional validation work should be performed, draft construction specification frameworks should be developed, and work extending the technology to stabili...

A review and preliminary evaluation of methodological factors in performance assessments of time-varying aircraft noise effects

NASA Technical Reports Server (NTRS)

Coates, G. D.; Alluisi, E. A.

1975-01-01

The effects of aircraft noise on human performance is considered. Progress is reported in the following areas: (1) review of the literature to identify the methodological and stimulus parameters involved in the study of noise effects on human performance; (2) development of a theoretical framework to provide working hypotheses as to the effects of noise on complex human performance; and (3) data collection on the first of several experimental investigations designed to provide tests of the hypotheses.

Establishing a Numerical Modeling Framework for Hydrologic Engineering Analyses of Extreme Storm Events

DOE Office of Scientific and Technical Information (OSTI.GOV)

Chen, Xiaodong; Hossain, Faisal; Leung, L. Ruby

In this study a numerical modeling framework for simulating extreme storm events was established using the Weather Research and Forecasting (WRF) model. Such a framework is necessary for the derivation of engineering parameters such as probable maximum precipitation that are the cornerstone of large water management infrastructure design. Here this framework was built based on a heavy storm that occurred in Nashville (USA) in 2010, and verified using two other extreme storms. To achieve the optimal setup, several combinations of model resolutions, initial/boundary conditions (IC/BC), cloud microphysics and cumulus parameterization schemes were evaluated using multiple metrics of precipitation characteristics. Themore » evaluation suggests that WRF is most sensitive to IC/BC option. Simulation generally benefits from finer resolutions up to 5 km. At the 15km level, NCEP2 IC/BC produces better results, while NAM IC/BC performs best at the 5km level. Recommended model configuration from this study is: NAM or NCEP2 IC/BC (depending on data availability), 15km or 15km-5km nested grids, Morrison microphysics and Kain-Fritsch cumulus schemes. Validation of the optimal framework suggests that these options are good starting choices for modeling extreme events similar to the test cases. This optimal framework is proposed in response to emerging engineering demands of extreme storm events forecasting and analyses for design, operations and risk assessment of large water infrastructures.« less

Agent-based Large-Scale Emergency Evacuation Using Real-Time Open Government Data

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lu, Wei; Liu, Cheng; Bhaduri, Budhendra L

The open government initiatives have provided tremendous data resources for the transportation system and emergency services in urban areas. This paper proposes a traffic simulation framework using high temporal resolution demographic data and real time open government data for evacuation planning and operation. A comparison study using real-world data in Seattle, Washington is conducted to evaluate the framework accuracy and evacuation efficiency. The successful simulations of selected area prove the concept to take advantage open government data, open source data, and high resolution demographic data in emergency management domain. There are two aspects of parameters considered in this study: usermore » equilibrium (UE) conditions of traffic assignment model (simple Non-UE vs. iterative UE) and data temporal resolution (Daytime vs. Nighttime). Evacuation arrival rate, average travel time, and computation time are adopted as Measure of Effectiveness (MOE) for evacuation performance analysis. The temporal resolution of demographic data has significant impacts on urban transportation dynamics during evacuation scenarios. Better evacuation performance estimation can be approached by integrating both Non-UE and UE scenarios. The new framework shows flexibility in implementing different evacuation strategies and accuracy in evacuation performance. The use of this framework can be explored to day-to-day traffic assignment to support daily traffic operations.« less

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

PubMed Central

Ordóñez, Francisco Javier; Roggen, Daniel

2016-01-01

Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation. PMID:26797612

Guidelines for evaluating the educational performance of medical school faculty: priming a national conversation.

PubMed

Baldwin, Constance; Chandran, Latha; Gusic, Maryellen

2011-01-01

The academic community needs a sound framework for the promotion and advancement of educators. The Group on Educational Affairs of the Association of American Medical Colleges organized a consensus conference that affirmed the use of five domains for documenting the quantity and quality of scholarly engagement in educational activities: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment. In this article, we offer detailed guidelines to evaluate these five domains of educator performance and the essential elements of scholarly activity. The guidelines are adapted from our developmental educator portfolio template and educator portfolio analysis tool, previously published in MedEdPORTAL. A short tool for educator performance evaluation that summarizes items in the guidelines is proposed for discussion. Our goal in this article is to itemize criteria for systematic faculty evaluation that can be applied in any institutional setting to assist promotion decision makers in their task of evaluating medical school faculty.

Evaluation Criteria of Noninvasive Telemonitoring for Patients With Heart Failure: Systematic Review.

PubMed

Farnia, Troskah; Jaulent, Marie-Christine; Steichen, Olivier

2018-01-16

Telemonitoring can improve heart failure (HF) management, but there is no standardized evaluation framework to comprehensively evaluate its impact. Our objectives were to list the criteria used in published evaluations of noninvasive HF telemonitoring projects, describe how they are used in the evaluation studies, and organize them into a consistent scheme. Articles published from January 1990 to August 2015 were obtained through MEDLINE, Web of Science, and EMBASE. Articles were eligible if they were original reports of a noninvasive HF telemonitoring evaluation study in the English language. Studies of implantable telemonitoring devices were excluded. Each selected article was screened to extract the description of the telemonitoring project and the evaluation process and criteria. A qualitative synthesis was performed. We identified and reviewed 128 articles leading to 52 evaluation criteria classified into 6 dimensions: clinical, economic, user perspective, educational, organizational, and technical. The clinical and economic impacts were evaluated in more than 70% of studies, whereas the educational, organizational, and technical impacts were studied in fewer than 15%. User perspective was the most frequently covered dimension in the development phase of telemonitoring projects, whereas clinical and economic impacts were the focus of later phases. Telemonitoring evaluation frameworks should cover all 6 dimensions appropriately distributed along the telemonitoring project lifecycle. Our next goal is to build such a comprehensive evaluation framework for telemonitoring and test it on an ongoing noninvasive HF telemonitoring project. ©Troskah Farnia, Marie-Christine Jaulent, Olivier Steichen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.01.2018.

A framework for correcting brain retraction based on an eXtended Finite Element Method using a laser range scanner.

PubMed

Li, Ping; Wang, Weiwei; Song, Zhijian; An, Yong; Zhang, Chenxi

2014-07-01

Brain retraction causes great distortion that limits the accuracy of an image-guided neurosurgery system that uses preoperative images. Therefore, brain retraction correction is an important intraoperative clinical application. We used a linear elastic biomechanical model, which deforms based on the eXtended Finite Element Method (XFEM) within a framework for brain retraction correction. In particular, a laser range scanner was introduced to obtain a surface point cloud of the exposed surgical field including retractors inserted into the brain. A brain retraction surface tracking algorithm converted these point clouds into boundary conditions applied to XFEM modeling that drive brain deformation. To test the framework, we performed a brain phantom experiment involving the retraction of tissue. Pairs of the modified Hausdorff distance between Canny edges extracted from model-updated images, pre-retraction, and post-retraction CT images were compared to evaluate the morphological alignment of our framework. Furthermore, the measured displacements of beads embedded in the brain phantom and the predicted ones were compared to evaluate numerical performance. The modified Hausdorff distance of 19 pairs of images decreased from 1.10 to 0.76 mm. The forecast error of 23 stainless steel beads in the phantom was between 0 and 1.73 mm (mean 1.19 mm). The correction accuracy varied between 52.8 and 100 % (mean 81.4 %). The results demonstrate that the brain retraction compensation can be incorporated intraoperatively into the model-updating process in image-guided neurosurgery systems.

Using a Malcolm Baldrige framework to understand high-performing clinical microsystems.

PubMed

Foster, Tina C; Johnson, Julie K; Nelson, Eugene C; Batalden, Paul B

2007-10-01

BACKGROUND, OBJECTIVES AND METHOD: The Malcolm Baldrige National Quality Award (MBNQA) provides a set of criteria for organisational quality assessment and improvement that has been used by thousands of business, healthcare and educational organisations for more than a decade. The criteria can be used as a tool for self-evaluation, and are widely recognised as a robust framework for design and evaluation of healthcare systems. The clinical microsystem, as an organisational construct, is a systems approach for providing clinical care based on theories from organisational development, leadership and improvement. This study compared the MBNQA criteria for healthcare and the success factors of high-performing clinical microsystems to (1) determine whether microsystem success characteristics cover the same range of issues addressed by the Baldrige criteria and (2) examine whether this comparison might better inform our understanding of either framework. Both Baldrige criteria and microsystem success characteristics cover a wide range of areas crucial to high performance. Those particularly called out by this analysis are organisational leadership, work systems and service processes from a Baldrige standpoint, and leadership, performance results, process improvement, and information and information technology from the microsystem success characteristics view. Although in many cases the relationship between Baldrige criteria and microsystem success characteristics are obvious, in others the analysis points to ways in which the Baldrige criteria might be better understood and worked with by a microsystem through the design of work systems and a deep understanding of processes. Several tools are available for those who wish to engage in self-assessment based on MBNQA criteria and microsystem characteristics.

Using a Malcolm Baldrige framework to understand high‐performing clinical microsystems

PubMed Central

Foster, Tina C; Johnson, Julie K; Nelson, Eugene C; Batalden, Paul B

2007-01-01

Background, objectives and method The Malcolm Baldrige National Quality Award (MBNQA) provides a set of criteria for organisational quality assessment and improvement that has been used by thousands of business, healthcare and educational organisations for more than a decade. The criteria can be used as a tool for self‐evaluation, and are widely recognised as a robust framework for design and evaluation of healthcare systems. The clinical microsystem, as an organisational construct, is a systems approach for providing clinical care based on theories from organisational development, leadership and improvement. This study compared the MBNQA criteria for healthcare and the success factors of high‐performing clinical microsystems to (1) determine whether microsystem success characteristics cover the same range of issues addressed by the Baldrige criteria and (2) examine whether this comparison might better inform our understanding of either framework. Results and conclusions Both Baldrige criteria and microsystem success characteristics cover a wide range of areas crucial to high performance. Those particularly called out by this analysis are organisational leadership, work systems and service processes from a Baldrige standpoint, and leadership, performance results, process improvement, and information and information technology from the microsystem success characteristics view. Although in many cases the relationship between Baldrige criteria and microsystem success characteristics are obvious, in others the analysis points to ways in which the Baldrige criteria might be better understood and worked with by a microsystem through the design of work systems and a deep understanding of processes. Several tools are available for those who wish to engage in self‐assessment based on MBNQA criteria and microsystem characteristics. PMID:17913773

An evaluation framework for effective public participation in EIA in Pakistan

DOE Office of Scientific and Technical Information (OSTI.GOV)

Nadeem, Obaidullah, E-mail: obaidnadeem@yahoo.co; Fischer, Thomas B., E-mail: fischer@liverpool.ac.u

2011-01-15

Evaluating the effectiveness of public participation in EIA related decisions is of crucial importance for developing a better understanding of overall EIA effectiveness. This paper aims to contribute to the professional debate by establishing a country specific evaluation framework for Pakistan, which, it is suggested, could also potentially be used in other developing countries. The framework is used to evaluate performance of public participation in EIA in terms of 40 attributes for four selected projects from the province of Punjab. The evaluation is based on interviews with stakeholders, review of EIA reports as well as public hearing proceedings and environmentalmore » approval conditions. The evaluation of the selected projects revealed an overall weak influence of public participation on substantive quality of EIA and on the final decision. Overall, EIA public participation has succeeded in providing a more egalitarian environment. Furthermore, it appears fair to say that sufficient time for submitting written comments on EIA reports as well as for raising concerns during public hearings had been given. Also, public consultation was significantly contributing to educating participants. Despite some impediments, it is argued that public participation in EIA is gradually gaining ground in Pakistan. Recommendations to enhance EIA public participation effectiveness in Pakistan include applying a more proactive approach which should take place before EIA is conducted and before site selection for development projects is happening.« less

Achieving Fairness in Data Fusion Performance Evaluation Development

DTIC Science & Technology

2006-04-30

display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM-YYYY) 2.REPORT TYPE 13. DATES...for and evaluation (T&E) organization is how to affordably achieve fairness in the application of its PE systems. Our PE framework provides a ...as a fusion system design balances probability of mission success with complexity/cost). The key issue for and evaluation (T&E) organization is how to

Evaluating a Modular Decision Support Application for Colorectal Cancer Screening

PubMed Central

Diiulio, Julie B.; Borders, Morgan R.; Sushereba, Christen E.; Saleem, Jason J.; Haverkamp, Donald; Imperiale, Thomas F.

2017-01-01

Summary Background There is a need for health information technology evaluation that goes beyond randomized controlled trials to include consideration of usability, cognition, feedback from representative users, and impact on efficiency, data quality, and clinical workflow. This article presents an evaluation illustrating one approach to this need using the Decision-Centered Design framework. Objective To evaluate, through a Decision-Centered Design framework, the ability of the Screening and Surveillance App to support primary care clinicians in tracking and managing colorectal cancer testing. Methods We leveraged two evaluation formats, online and in-person, to obtain feedback from a range primary care clinicians and obtain comparative data. Both the online and in-person evaluations used mock patient data to simulate challenging patient scenarios. Primary care clinicians responded to a series of colorectal cancer-related questions about each patient and made recommendations for screening. We collected data on performance, perceived workload, and usability. Key elements of Decision-Centered Design include evaluation in the context of realistic, challenging scenarios and measures designed to explore impact on cognitive performance. Results Comparison of means revealed increases in accuracy, efficiency, and usability and decreases in perceived mental effort and workload when using the Screening and Surveillance App. Conclusion The results speak to the benefits of using the Decision-Centered Design approach in the analysis, design, and evaluation of Health Information Technology. Furthermore, the Screening and Surveillance App shows promise for filling decision support gaps in current electronic health records. PMID:28197619

Dynamic and quantitative evaluation of degenerative mitral valve disease: a dedicated framework based on cardiac magnetic resonance imaging.

PubMed

Sturla, Francesco; Onorati, Francesco; Puppini, Giovanni; Pappalardo, Omar A; Selmi, Matteo; Votta, Emiliano; Faggian, Giuseppe; Redaelli, Alberto

2017-04-01

Accurate quantification of mitral valve (MV) morphology and dynamic behavior over the cardiac cycle is crucial to understand the mechanisms of degenerative MV dysfunction and to guide the surgical intervention. Cardiac magnetic resonance (CMR) imaging has progressively been adopted to evaluate MV pathophysiology, although a dedicated framework is required to perform a quantitative assessment of the functional MV anatomy. We investigated MV dynamic behavior in subjects with normal MV anatomy (n=10) and patients referred to surgery due to degenerative MV prolapse, classified as fibro-elastic deficiency (FED, n=9) and Barlow's disease (BD, n=10). A CMR-dedicated framework was adopted to evaluate prolapse height and volume and quantitatively assess valvular morphology and papillary muscles (PAPs) function over the cardiac cycle. Multiple comparison was used to investigate the hallmarks associated to MV degenerative prolapse and evaluate the feasibility of anatomical and functional distinction between FED and BD phenotypes. On average, annular dimensions were significantly (P<0.05) larger in BD than in FED and normal subjects while no significant differences were noticed between FED and normal. MV eccentricity progressively decreased passing from normal to FED and BD, with the latter exhibiting a rounder annulus shape. Over the cardiac cycle, we noticed significant differences for BD during systole with an abnormal annular enlargement between mid and late systole (LS) (P<0.001 vs. normal); the PAPs dynamics remained comparable in the three groups. Prolapse height and volume highlighted significant differences among normal, FED and BD valves. Our CMR-dedicated framework allows for the quantitative and dynamic evaluation of MV apparatus, with quantifiable annular alterations representing the primary hallmark of severe MV degeneration. This may aid surgeons in the evaluation of the severity of MV dysfunction and the selection of the appropriate MV treatment.

Facility Energy Performance Benchmarking in a Data-Scarce Environment

DTIC Science & Technology

2017-08-01

environment, and analyze occupant-, system-, and component-level faults contributing to energy in- efficiency. A methodology for developing DoD-specific...Research, Development, Test, and Evaluation (RDTE) Program to develop an intelligent framework, encompassing methodology and model- ing, that...energy performers by installation, climate zone, and other criteria. A methodology for creating the DoD-specific EUIs would be an important part of a

Healthcare4VideoStorm: Making Smart Decisions Based on Storm Metrics.

PubMed

Zhang, Weishan; Duan, Pengcheng; Chen, Xiufeng; Lu, Qinghua

2016-04-23

Storm-based stream processing is widely used for real-time large-scale distributed processing. Knowing the run-time status and ensuring performance is critical to providing expected dependability for some applications, e.g., continuous video processing for security surveillance. The existing scheduling strategies' granularity is too coarse to have good performance, and mainly considers network resources without computing resources while scheduling. In this paper, we propose Healthcare4Storm, a framework that finds Storm insights based on Storm metrics to gain knowledge from the health status of an application, finally ending up with smart scheduling decisions. It takes into account both network and computing resources and conducts scheduling at a fine-grained level using tuples instead of topologies. The comprehensive evaluation shows that the proposed framework has good performance and can improve the dependability of the Storm-based applications.

Application-level regression testing framework using Jenkins

DOE PAGES

Budiardja, Reuben; Bouvet, Timothy; Arnold, Galen

2017-09-26

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integratemore » with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.« less

Application-level regression testing framework using Jenkins

DOE Office of Scientific and Technical Information (OSTI.GOV)

Budiardja, Reuben; Bouvet, Timothy; Arnold, Galen

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integratemore » with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.« less

Supportive supervision and constructive relationships with healthcare workers support CHW performance: Use of a qualitative framework to evaluate CHW programming in Uganda.

PubMed

Ludwick, Teralynn; Turyakira, Eleanor; Kyomuhangi, Teddy; Manalili, Kimberly; Robinson, Sheila; Brenner, Jennifer L

2018-02-13

While evidence supports community health worker (CHW) capacity to improve maternal and newborn health in less-resourced countries, key implementation gaps remain. Tools for assessing CHW performance and evidence on what programmatic components affect performance are lacking. This study developed and tested a qualitative evaluative framework and tool to assess CHW team performance in a district program in rural Uganda. A new assessment framework was developed to collect and analyze qualitative evidence based on CHW perspectives on seven program components associated with effectiveness (selection; training; community embeddedness; peer support; supportive supervision; relationship with other healthcare workers; retention and incentive structures). Focus groups were conducted with four high/medium-performing CHW teams and four low-performing CHW teams selected through random, stratified sampling. Content analysis involved organizing focus group transcripts according to the seven program effectiveness components, and assigning scores to each component per focus group. Four components, 'supportive supervision', 'good relationships with other healthcare workers', 'peer support', and 'retention and incentive structures' received the lowest overall scores. Variances in scores between 'high'/'medium'- and 'low'-performing CHW teams were largest for 'supportive supervision' and 'good relationships with other healthcare workers.' Our analysis suggests that in the Bushenyi intervention context, CHW team performance is highly correlated with the quality of supervision and relationships with other healthcare workers. CHWs identified key performance-related issues of absentee supervisors, referral system challenges, and lack of engagement/respect by health workers. Other less-correlated program components warrant further study and may have been impacted by relatively consistent program implementation within our limited study area. Applying process-oriented measurement tools are needed to better understand CHW performance-related factors and build a supportive environment for CHW program effectiveness and sustainability. Findings from a qualitative, multi-component tool developed and applied in this study suggest that factors related to (1) supportive supervision and (2) relationships with other healthcare workers may be strongly associated with variances in performance outcomes within a program. Careful consideration of supervisory structure and health worker orientation during program implementation are among strategies proposed to increase CHW performance.

Management of children exposed to Mycobacterium tuberculosis: a public health evaluation in West Java, Indonesia

PubMed Central

Ruslami, Rovina; Anselmo, Melissa; Alisjahbana, Bachti; Yulianti, Neti; Sampurno, Hedy; van Crevel, Reinout; Hill, Philip C

2013-01-01

Abstract Objective To investigate qualitatively and quantitatively the performance of a programme for managing the child contacts of adult tuberculosis patients in Indonesia. Methods A public health evaluation framework was used to assess gaps in a child contact management programme at a lung clinic. Targets for programme performance indicators were derived from established programme indicator targets, the scientific literature and expert opinion. Compliance with tuberculosis screening, the initiation of isoniazid preventive therapy in children younger than 5 years, the accuracy of tuberculosis diagnosis and adherence to preventive therapy were assessed in 755 child contacts in two cohorts. In addition, 22 primary caregivers and 34 clinic staff were interviewed to evaluate knowledge and acceptance of child contact management. The cost to caregivers was recorded. Gaps between observed and target indicator values were quantified. Findings The gaps between observed and target performance indicators were: 82% for screening compliance; 64 to 100% for diagnostic accuracy, 50% for the initiation of preventive therapy, 54% for adherence to therapy and 50% for costs. Many staff did not have adequate knowledge of, or an appropriate attitude towards, child contact management, especially regarding isoniazid preventive therapy. Caregivers had good knowledge of screening but not of preventive therapy and had difficulty travelling to the clinic and paying costs. Conclusion The study identified widespread gaps in the performance of a child contact management system in Indonesia, all of which appear amenable to intervention. The public health evaluation framework used could be applied in other settings where child contact management is failing. PMID:24347732

Auditing Staff-Management Communication in Schools: A Framework for Evaluating Performance.

ERIC Educational Resources Information Center

Tourish, Dennis; Hargie, Owen

1998-01-01

Outlines the principles and procedures of an organizational communication audit as a measure of the effectiveness of personnel management in elementary and secondary schools, examining the principal tools used and the benefits that this approach offers to school management teams. (Author/MSE)

LIFE CYCLE DESIGN OF AMORPHOUS SILICON PHOTOVOLTAIC MODULES

EPA Science Inventory

The life cycle design framework was applied to photovoltaic module design. The primary objective of this project was to develop and evaluate design metrics for assessing and guiding the Improvement of PV product systems. Two metrics were used to assess life cycle energy perform...

EVALUATIONS OF DNAPL REMEDIAL PERFORMANCE BASED ON FIELD MEASUREMENTS OF CONTAMINANT FLUX

EPA Science Inventory

Under a concentration-based regulatory framework, the benefits of conducting dense nonaqueous phase liquid (DNAPL) source-zone remediation are questionable because of the impracticality of complete DNAPL elimination at most sites. Removing a sufficient mass of DNAPL to achieve c...

Back-support large laser mirror unit: mounting modeling and analysis

NASA Astrophysics Data System (ADS)

Wang, Hui; Zhang, Zheng; Long, Kai; Liu, Tianye; Li, Jun; Liu, Changchun; Xiong, Zhao; Yuan, Xiaodong

2018-01-01

In high-power laser system, the surface wavefront of large optics has a close link with its structure design and mounting method. The back-support transport mirror design is presently being investigated as a means in China's high-power laser system to hold the optical component firmly while minimizing the distortion of its reflecting surface. We have proposed a comprehensive analytical framework integrated numerical modeling and precise metrology for the mirror's mounting performance evaluation while treating the surface distortion as a key decision variable. The combination of numerical simulation and field tests demonstrates that the comprehensive analytical framework provides a detailed and accurate approach to evaluate the performance of the transport mirror. It is also verified that the back-support transport mirror is effectively compatible with state-of-the-art optical quality specifications. This study will pave the way for future research to solidify the design of back-support large laser optics in China's next generation inertial confinement fusion facility.

The development of an evaluation framework for injury surveillance systems

PubMed Central

Mitchell, Rebecca J; Williamson, Ann M; O'Connor, Rod

2009-01-01

Background Access to good quality information from injury surveillance is essential to develop and monitor injury prevention activities. To determine if information obtained from surveillance is of high quality, the limitations and strengths of a surveillance system are often examined. Guidelines have been developed to assist in evaluating certain types of surveillance systems. However, to date, no standard guidelines have been developed to specifically evaluate an injury surveillance system. The aim of this research is to develop a framework to guide the evaluation of injury surveillance systems. Methods The development of an Evaluation Framework for Injury Surveillance Systems (EFISS) involved a four stage process. First, a literature review was conducted to identify an initial set of characteristics that were recognised as important and/or had been recommended to be assessed in an evaluation of a surveillance system. Second, this set of characteristics was assessed using SMART criteria. Third, those surviving were presented to an expert panel using a two round modified-Delphi study to gain an alternative perspective on characteristic definitions, practicality of assessment, and characteristic importance. Finally, a rating system was created for the EFISS characteristics. Results The resulting EFISS consisted of 18 characteristics that assess three areas of an injury surveillance system – five characteristics assess data quality, nine characteristics assess the system's operation, and four characteristics assess the practical capability of an injury surveillance system. A rating system assesses the performance of each characteristic. Conclusion The development of the EFISS builds upon existing evaluation guidelines for surveillance systems and provides a framework tailored to evaluate an injury surveillance system. Ultimately, information obtained through an evaluation of an injury data collection using the EFISS would be useful for agencies to recommend how a collection could be improved to increase its usefulness for injury surveillance and in the long-term injury prevention. PMID:19627617

Towards computerizing intensive care sedation guidelines: design of a rule-based architecture for automated execution of clinical guidelines

PubMed Central

2010-01-01

Background Computerized ICUs rely on software services to convey the medical condition of their patients as well as assisting the staff in taking treatment decisions. Such services are useful for following clinical guidelines quickly and accurately. However, the development of services is often time-consuming and error-prone. Consequently, many care-related activities are still conducted based on manually constructed guidelines. These are often ambiguous, which leads to unnecessary variations in treatments and costs. The goal of this paper is to present a semi-automatic verification and translation framework capable of turning manually constructed diagrams into ready-to-use programs. This framework combines the strengths of the manual and service-oriented approaches while decreasing their disadvantages. The aim is to close the gap in communication between the IT and the medical domain. This leads to a less time-consuming and error-prone development phase and a shorter clinical evaluation phase. Methods A framework is proposed that semi-automatically translates a clinical guideline, expressed as an XML-based flow chart, into a Drools Rule Flow by employing semantic technologies such as ontologies and SWRL. An overview of the architecture is given and all the technology choices are thoroughly motivated. Finally, it is shown how this framework can be integrated into a service-oriented architecture (SOA). Results The applicability of the Drools Rule language to express clinical guidelines is evaluated by translating an example guideline, namely the sedation protocol used for the anaesthetization of patients, to a Drools Rule Flow and executing and deploying this Rule-based application as a part of a SOA. The results show that the performance of Drools is comparable to other technologies such as Web Services and increases with the number of decision nodes present in the Rule Flow. Most delays are introduced by loading the Rule Flows. Conclusions The framework is an effective solution for computerizing clinical guidelines as it allows for quick development, evaluation and human-readable visualization of the Rules and has a good performance. By monitoring the parameters of the patient to automatically detect exceptional situations and problems and by notifying the medical staff of tasks that need to be performed, the computerized sedation guideline improves the execution of the guideline. PMID:20082700

Towards computerizing intensive care sedation guidelines: design of a rule-based architecture for automated execution of clinical guidelines.

PubMed

Ongenae, Femke; De Backere, Femke; Steurbaut, Kristof; Colpaert, Kirsten; Kerckhove, Wannes; Decruyenaere, Johan; De Turck, Filip

2010-01-18

Computerized ICUs rely on software services to convey the medical condition of their patients as well as assisting the staff in taking treatment decisions. Such services are useful for following clinical guidelines quickly and accurately. However, the development of services is often time-consuming and error-prone. Consequently, many care-related activities are still conducted based on manually constructed guidelines. These are often ambiguous, which leads to unnecessary variations in treatments and costs.The goal of this paper is to present a semi-automatic verification and translation framework capable of turning manually constructed diagrams into ready-to-use programs. This framework combines the strengths of the manual and service-oriented approaches while decreasing their disadvantages. The aim is to close the gap in communication between the IT and the medical domain. This leads to a less time-consuming and error-prone development phase and a shorter clinical evaluation phase. A framework is proposed that semi-automatically translates a clinical guideline, expressed as an XML-based flow chart, into a Drools Rule Flow by employing semantic technologies such as ontologies and SWRL. An overview of the architecture is given and all the technology choices are thoroughly motivated. Finally, it is shown how this framework can be integrated into a service-oriented architecture (SOA). The applicability of the Drools Rule language to express clinical guidelines is evaluated by translating an example guideline, namely the sedation protocol used for the anaesthetization of patients, to a Drools Rule Flow and executing and deploying this Rule-based application as a part of a SOA. The results show that the performance of Drools is comparable to other technologies such as Web Services and increases with the number of decision nodes present in the Rule Flow. Most delays are introduced by loading the Rule Flows. The framework is an effective solution for computerizing clinical guidelines as it allows for quick development, evaluation and human-readable visualization of the Rules and has a good performance. By monitoring the parameters of the patient to automatically detect exceptional situations and problems and by notifying the medical staff of tasks that need to be performed, the computerized sedation guideline improves the execution of the guideline.

Diversification strategy and performance: implications for health services research.

PubMed

Rivers, P A; Glover, S H; Munchus, G

1999-01-01

Health care represents a promising area of research due to its uniqueness. In recent years, considerable progress has been made in diversification strategy and performance research but not the study of health services strategy research. This article reviews diversification strategy and performance in health services domains. Adopting Datta, Rajagopalan, and Rasheed's (1991) framework, the authors evaluate the theoretical and empirical contributions of this research. The limitations and theoretical implications of these efforts are also explored.

Parallelizing Compiler Framework and API for Power Reduction and Software Productivity of Real-Time Heterogeneous Multicores

NASA Astrophysics Data System (ADS)

Hayashi, Akihiro; Wada, Yasutaka; Watanabe, Takeshi; Sekiguchi, Takeshi; Mase, Masayoshi; Shirako, Jun; Kimura, Keiji; Kasahara, Hironori

Heterogeneous multicores have been attracting much attention to attain high performance keeping power consumption low in wide spread of areas. However, heterogeneous multicores force programmers very difficult programming. The long application program development period lowers product competitiveness. In order to overcome such a situation, this paper proposes a compilation framework which bridges a gap between programmers and heterogeneous multicores. In particular, this paper describes the compilation framework based on OSCAR compiler. It realizes coarse grain task parallel processing, data transfer using a DMA controller, power reduction control from user programs with DVFS and clock gating on various heterogeneous multicores from different vendors. This paper also evaluates processing performance and the power reduction by the proposed framework on a newly developed 15 core heterogeneous multicore chip named RP-X integrating 8 general purpose processor cores and 3 types of accelerator cores which was developed by Renesas Electronics, Hitachi, Tokyo Institute of Technology and Waseda University. The framework attains speedups up to 32x for an optical flow program with eight general purpose processor cores and four DRP(Dynamically Reconfigurable Processor) accelerator cores against sequential execution by a single processor core and 80% of power reduction for the real-time AAC encoding.

A framework using cluster-based hybrid network architecture for collaborative virtual surgery.

PubMed

Qin, Jing; Choi, Kup-Sze; Poon, Wai-Sang; Heng, Pheng-Ann

2009-12-01

Research on collaborative virtual environments (CVEs) opens the opportunity for simulating the cooperative work in surgical operations. It is however a challenging task to implement a high performance collaborative surgical simulation system because of the difficulty in maintaining state consistency with minimum network latencies, especially when sophisticated deformable models and haptics are involved. In this paper, an integrated framework using cluster-based hybrid network architecture is proposed to support collaborative virtual surgery. Multicast transmission is employed to transmit updated information among participants in order to reduce network latencies, while system consistency is maintained by an administrative server. Reliable multicast is implemented using distributed message acknowledgment based on cluster cooperation and sliding window technique. The robustness of the framework is guaranteed by the failure detection chain which enables smooth transition when participants join and leave the collaboration, including normal and involuntary leaving. Communication overhead is further reduced by implementing a number of management approaches such as computational policies and collaborative mechanisms. The feasibility of the proposed framework is demonstrated by successfully extending an existing standalone orthopedic surgery trainer into a collaborative simulation system. A series of experiments have been conducted to evaluate the system performance. The results demonstrate that the proposed framework is capable of supporting collaborative surgical simulation.

A FRAMEWORK TO DESIGN AND OPTIMIZE CHEMICAL FLOODING PROCESSES

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mojdeh Delshad; Gary A. Pope; Kamy Sepehrnoori

2005-07-01

The goal of this proposed research is to provide an efficient and user friendly simulation framework for screening and optimizing chemical/microbial enhanced oil recovery processes. The framework will include (1) a user friendly interface to identify the variables that have the most impact on oil recovery using the concept of experimental design and response surface maps, (2) UTCHEM reservoir simulator to perform the numerical simulations, and (3) an economic model that automatically imports the simulation production data to evaluate the profitability of a particular design. Such a reservoir simulation framework is not currently available to the oil industry. The objectivesmore » of Task 1 are to develop three primary modules representing reservoir, chemical, and well data. The modules will be interfaced with an already available experimental design model. The objective of the Task 2 is to incorporate UTCHEM reservoir simulator and the modules with the strategic variables and developing the response surface maps to identify the significant variables from each module. The objective of the Task 3 is to develop the economic model designed specifically for the chemical processes targeted in this proposal and interface the economic model with UTCHEM production output. Task 4 is on the validation of the framework and performing simulations of oil reservoirs to screen, design and optimize the chemical processes.« less

A Framework to Design and Optimize Chemical Flooding Processes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mojdeh Delshad; Gary A. Pope; Kamy Sepehrnoori

2006-08-31

The goal of this proposed research is to provide an efficient and user friendly simulation framework for screening and optimizing chemical/microbial enhanced oil recovery processes. The framework will include (1) a user friendly interface to identify the variables that have the most impact on oil recovery using the concept of experimental design and response surface maps, (2) UTCHEM reservoir simulator to perform the numerical simulations, and (3) an economic model that automatically imports the simulation production data to evaluate the profitability of a particular design. Such a reservoir simulation framework is not currently available to the oil industry. The objectivesmore » of Task 1 are to develop three primary modules representing reservoir, chemical, and well data. The modules will be interfaced with an already available experimental design model. The objective of the Task 2 is to incorporate UTCHEM reservoir simulator and the modules with the strategic variables and developing the response surface maps to identify the significant variables from each module. The objective of the Task 3 is to develop the economic model designed specifically for the chemical processes targeted in this proposal and interface the economic model with UTCHEM production output. Task 4 is on the validation of the framework and performing simulations of oil reservoirs to screen, design and optimize the chemical processes.« less

A FRAMEWORK TO DESIGN AND OPTIMIZE CHEMICAL FLOODING PROCESSES

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mojdeh Delshad; Gary A. Pope; Kamy Sepehrnoori

2004-11-01

The goal of this proposed research is to provide an efficient and user friendly simulation framework for screening and optimizing chemical/microbial enhanced oil recovery processes. The framework will include (1) a user friendly interface to identify the variables that have the most impact on oil recovery using the concept of experimental design and response surface maps, (2) UTCHEM reservoir simulator to perform the numerical simulations, and (3) an economic model that automatically imports the simulation production data to evaluate the profitability of a particular design. Such a reservoir simulation framework is not currently available to the oil industry. The objectivesmore » of Task 1 are to develop three primary modules representing reservoir, chemical, and well data. The modules will be interfaced with an already available experimental design model. The objective of the Task 2 is to incorporate UTCHEM reservoir simulator and the modules with the strategic variables and developing the response surface maps to identify the significant variables from each module. The objective of the Task 3 is to develop the economic model designed specifically for the chemical processes targeted in this proposal and interface the economic model with UTCHEM production output. Task 4 is on the validation of the framework and performing simulations of oil reservoirs to screen, design and optimize the chemical processes.« less

Performance evaluation of DNA copy number segmentation methods.

PubMed

Pierre-Jean, Morgane; Rigaill, Guillem; Neuvial, Pierre

2015-07-01

A number of bioinformatic or biostatistical methods are available for analyzing DNA copy number profiles measured from microarray or sequencing technologies. In the absence of rich enough gold standard data sets, the performance of these methods is generally assessed using unrealistic simulation studies, or based on small real data analyses. To make an objective and reproducible performance assessment, we have designed and implemented a framework to generate realistic DNA copy number profiles of cancer samples with known truth. These profiles are generated by resampling publicly available SNP microarray data from genomic regions with known copy-number state. The original data have been extracted from dilutions series of tumor cell lines with matched blood samples at several concentrations. Therefore, the signal-to-noise ratio of the generated profiles can be controlled through the (known) percentage of tumor cells in the sample. This article describes this framework and its application to a comparison study between methods for segmenting DNA copy number profiles from SNP microarrays. This study indicates that no single method is uniformly better than all others. It also helps identifying pros and cons of the compared methods as a function of biologically informative parameters, such as the fraction of tumor cells in the sample and the proportion of heterozygous markers. This comparison study may be reproduced using the open source and cross-platform R package jointseg, which implements the proposed data generation and evaluation framework: http://r-forge.r-project.org/R/?group_id=1562. © The Author 2014. Published by Oxford University Press.

An Analytical Framework for Assessing the Efficacy of Small Satellites in Performing Novel Imaging Missions

NASA Astrophysics Data System (ADS)

Weaver, Oesa A.

In the last two decades, small satellites have opened up the use of space to groups other than governments and large corporations, allowing for increased participation and experimentation. This democratization of space was primarily enabled by two factors: improved technology and reduced launch costs. Improved technology allowed the miniaturization of components and reduced overall cost meaning many of the capabilities of larger satellites could be replicated at a fraction of the cost. In addition, new launcher systems that could host many small satellites as ride-shares on manifested vehicles lowered launch costs and simplified the process of getting a satellite into orbit. The potential of these smaller satellites to replace or augment existing systems has led to a flood of potential satellite and mission concepts, often with little rigorous study of whether the proposed satellite or mission is achievable or necessary. This work proposes an analytical framework to aid system designers in evaluating the ability of an existing concept or small satellite to perform a particular imaging mission, either replacing or augmenting existing capabilities. This framework was developed and then refined by application to the problem of using small satellites to perform a wide area search mission -- a mission not possible with existing imaging satellites, but one that would add to current capabilities. Requirements for a wide area search mission were developed, along with a list of factors that would affect image quality and system performance. Two existing small satellite concepts were evaluated for use by examining image quality from the systems, selecting an algorithm to perform the search function automatically, and then assessing mission feasibility by applying the algorithm to simulated imagery. Finally, a notional constellation design was developed to assess the number of satellites required to perform the mission. It was found that a constellation of 480 CubeSats producing 4 m spatial resolution panchromatic imagery and employing an on-board processing algorithm would be sufficient to perform a wide area search mission.

Land Surface Verification Toolkit (LVT) - A Generalized Framework for Land Surface Model Evaluation

NASA Technical Reports Server (NTRS)

Kumar, Sujay V.; Peters-Lidard, Christa D.; Santanello, Joseph; Harrison, Ken; Liu, Yuqiong; Shaw, Michael

2011-01-01

Model evaluation and verification are key in improving the usage and applicability of simulation models for real-world applications. In this article, the development and capabilities of a formal system for land surface model evaluation called the Land surface Verification Toolkit (LVT) is described. LVT is designed to provide an integrated environment for systematic land model evaluation and facilitates a range of verification approaches and analysis capabilities. LVT operates across multiple temporal and spatial scales and employs a large suite of in-situ, remotely sensed and other model and reanalysis datasets in their native formats. In addition to the traditional accuracy-based measures, LVT also includes uncertainty and ensemble diagnostics, information theory measures, spatial similarity metrics and scale decomposition techniques that provide novel ways for performing diagnostic model evaluations. Though LVT was originally designed to support the land surface modeling and data assimilation framework known as the Land Information System (LIS), it also supports hydrological data products from other, non-LIS environments. In addition, the analysis of diagnostics from various computational subsystems of LIS including data assimilation, optimization and uncertainty estimation are supported within LVT. Together, LIS and LVT provide a robust end-to-end environment for enabling the concepts of model data fusion for hydrological applications. The evolving capabilities of LVT framework are expected to facilitate rapid model evaluation efforts and aid the definition and refinement of formal evaluation procedures for the land surface modeling community.

Health services research evaluation principles. Broadening a general framework for evaluating health information technology.

PubMed

Sockolow, P S; Crawford, P R; Lehmann, H P

2012-01-01

Our forthcoming national experiment in increased health information technology (HIT) adoption funded by the American Recovery and Reinvestment Act of 2009 will require a comprehensive approach to evaluating HIT. The quality of evaluation studies of HIT to date reveals a need for broader evaluation frameworks that limits the generalizability of findings and the depth of lessons learned. Develop an informatics evaluation framework for health information technology (HIT) integrating components of health services research (HSR) evaluation and informatics evaluation to address identified shortcomings in available HIT evaluation frameworks. A systematic literature review updated and expanded the exhaustive review by Ammenwerth and deKeizer (AdK). From retained studies, criteria were elicited and organized into classes within a framework. The resulting Health Information Technology Research-based Evaluation Framework (HITREF) was used to guide clinician satisfaction survey construction, multi-dimensional analysis of data, and interpretation of findings in an evaluation of a vanguard community health care EHR. The updated review identified 128 electronic health record (EHR) evaluation studies and seven evaluation criteria not in AdK: EHR Selection/Development/Training; Patient Privacy Concerns; Unintended Consequences/ Benefits; Functionality; Patient Satisfaction with EHR; Barriers/Facilitators to Adoption; and Patient Satisfaction with Care. HITREF was used productively and was a complete evaluation framework which included all themes that emerged. We can recommend to future EHR evaluators that they consider adding a complete, research-based HIT evaluation framework, such as HITREF, to their evaluation tools suite to monitor HIT challenges as the federal government strives to increase HIT adoption.

Performance evaluation of Bragg coherent diffraction imaging

NASA Astrophysics Data System (ADS)

Öztürk, H.; Huang, X.; Yan, H.; Robinson, I. K.; Noyan, I. C.; Chu, Y. S.

2017-10-01

In this study, we present a numerical framework for modeling three-dimensional (3D) diffraction data in Bragg coherent diffraction imaging (Bragg CDI) experiments and evaluating the quality of obtained 3D complex-valued real-space images recovered by reconstruction algorithms under controlled conditions. The approach is used to systematically explore the performance and the detection limit of this phase-retrieval-based microscopy tool. The numerical investigation suggests that the superb performance of Bragg CDI is achieved with an oversampling ratio above 30 and a detection dynamic range above 6 orders. The observed performance degradation subject to the data binning processes is also studied. This numerical tool can be used to optimize experimental parameters and has the potential to significantly improve the throughput of Bragg CDI method.

In the Pen of the Author or Eye of the Beholder? A Measurement Framework for Understanding Peer Evaluation in Writing Assignments

ERIC Educational Resources Information Center

Branum-Martin, Lee; Patchan, Melissa M.

2016-01-01

Peer learning is often used in classrooms to help and support knowledge and skill acquisition. One form of peer learning, peer assessment, involves the quantitative (i.e., peer ratings) or qualitative (i.e., peer feedback) evaluation of a learner's performance by another learner among students. While we might be concerned about the quality of the…

An Investigation of Run-Time Operations in a Heterogeneous Desktop Grid Environment: The Texas Tech University Desktop Grid Case Study

ERIC Educational Resources Information Center

Perez, Jerry F.

2013-01-01

The goal of the dissertation study was to evaluate the existing DG scheduling algorithm. The evaluation was developed through previously explored simulated analyses of DGs performed by researchers in the field of DG scheduling optimization and to improve the current RT framework of the DG at TTU. The author analyzed the RT of an actual DG, thereby…

Framework for Evaluating the Impact of Advanced Practice Nursing Roles.

PubMed

Bryant-Lukosius, Denise; Spichiger, Elisabeth; Martin, Jacqueline; Stoll, Hansruedi; Kellerhals, Sabine Degen; Fliedner, Monica; Grossmann, Florian; Henry, Morag; Herrmann, Luzia; Koller, Antje; Schwendimann, René; Ulrich, Anja; Weibel, Lukas; Callens, Betty; De Geest, Sabina

2016-03-01

To address the gap in evidence-based information required to support the development of advanced practice nursing (APN) roles in Switzerland, stakeholders identified the need for guidance to generate strategic evaluation data. This article describes an evaluation framework developed to inform decisions about the effective utilization of APN roles across the country. A participatory approach was used by an international group of stakeholders. Published literature and an evidenced-based framework for introducing APN roles were analyzed and applied to define the purpose, target audiences, and essential elements of the evaluation framework. Through subsequent meetings and review by an expert panel, the framework was developed and refined. A framework to evaluate different types of APN roles as they evolve to meet dynamic population health, practice setting, and health system needs was created. It includes a matrix of key concepts to guide evaluations across three stages of APN role development: introduction, implementation, and long-term sustainability. For each stage, evaluation objectives and questions examining APN role structures, processes, and outcomes from different perspectives (e.g., patients, providers, managers, policy-makers) were identified. A practical, robust framework based on well-established evaluation concepts and current understanding of APN roles can be used to conduct systematic evaluations. The evaluation framework is sufficiently generic to allow application in developed countries globally, both for evaluation as well as research purposes. © 2016 Sigma Theta Tau International.

Towards more sustainable management of European food waste: Methodological approach and numerical application.

PubMed

Manfredi, Simone; Cristobal, Jorge

2016-09-01

Trying to respond to the latest policy needs, the work presented in this article aims at developing a life-cycle based framework methodology to quantitatively evaluate the environmental and economic sustainability of European food waste management options. The methodology is structured into six steps aimed at defining boundaries and scope of the evaluation, evaluating environmental and economic impacts and identifying best performing options. The methodology is able to accommodate additional assessment criteria, for example the social dimension of sustainability, thus moving towards a comprehensive sustainability assessment framework. A numerical case study is also developed to provide an example of application of the proposed methodology to an average European context. Different options for food waste treatment are compared, including landfilling, composting, anaerobic digestion and incineration. The environmental dimension is evaluated with the software EASETECH, while the economic assessment is conducted based on different indicators expressing the costs associated with food waste management. Results show that the proposed methodology allows for a straightforward identification of the most sustainable options for food waste, thus can provide factual support to decision/policy making. However, it was also observed that results markedly depend on a number of user-defined assumptions, for example on the choice of the indicators to express the environmental and economic performance. © The Author(s) 2016.

Fuzzy Traffic Control with Vehicle-to-Everything Communication.

PubMed

Salman, Muntaser A; Ozdemir, Suat; Celebi, Fatih V

2018-01-27

Traffic signal control (TSC) with vehicle-to everything (V2X) communication can be a very efficient solution to traffic congestion problem. Ratio of vehicles equipped with V2X communication capability in the traffic to the total number of vehicles (called penetration rate PR) is still low, thus V2X based TSC systems need to be supported by some other mechanisms. PR is the major factor that affects the quality of TSC process along with the evaluation interval. Quality of the TSC in each direction is a function of overall TSC quality of an intersection. Hence, quality evaluation of each direction should follow the evaluation of the overall intersection. Computational intelligence, more specifically swarm algorithm, has been recently used in this field in a European Framework Program FP7 supported project called COLOMBO. In this paper, using COLOMBO framework, further investigations have been done and two new methodologies using simple and fuzzy logic have been proposed. To evaluate the performance of our proposed methods, a comparison with COLOMBOs approach has been realized. The results reveal that TSC problem can be solved as a logical problem rather than an optimization problem. Performance of the proposed approaches is good enough to be suggested for future work under realistic scenarios even under low PR.

Fuzzy Traffic Control with Vehicle-to-Everything Communication

PubMed Central

Ozdemir, Suat; Celebi, Fatih V.

2018-01-01

Traffic signal control (TSC) with vehicle-to everything (V2X) communication can be a very efficient solution to traffic congestion problem. Ratio of vehicles equipped with V2X communication capability in the traffic to the total number of vehicles (called penetration rate PR) is still low, thus V2X based TSC systems need to be supported by some other mechanisms. PR is the major factor that affects the quality of TSC process along with the evaluation interval. Quality of the TSC in each direction is a function of overall TSC quality of an intersection. Hence, quality evaluation of each direction should follow the evaluation of the overall intersection. Computational intelligence, more specifically swarm algorithm, has been recently used in this field in a European Framework Program FP7 supported project called COLOMBO. In this paper, using COLOMBO framework, further investigations have been done and two new methodologies using simple and fuzzy logic have been proposed. To evaluate the performance of our proposed methods, a comparison with COLOMBOs approach has been realized. The results reveal that TSC problem can be solved as a logical problem rather than an optimization problem. Performance of the proposed approaches is good enough to be suggested for future work under realistic scenarios even under low PR. PMID:29382053

CommWalker: correctly evaluating modules in molecular networks in light of annotation bias.

PubMed

Luecken, M D; Page, M J T; Crosby, A J; Mason, S; Reinert, G; Deane, C M

2018-03-15

Detecting novel functional modules in molecular networks is an important step in biological research. In the absence of gold standard functional modules, functional annotations are often used to verify whether detected modules/communities have biological meaning. However, as we show, the uneven distribution of functional annotations means that such evaluation methods favor communities of well-studied proteins. We propose a novel framework for the evaluation of communities as functional modules. Our proposed framework, CommWalker, takes communities as inputs and evaluates them in their local network environment by performing short random walks. We test CommWalker's ability to overcome annotation bias using input communities from four community detection methods on two protein interaction networks. We find that modules accepted by CommWalker are similarly co-expressed as those accepted by current methods. Crucially, CommWalker performs well not only in well-annotated regions, but also in regions otherwise obscured by poor annotation. CommWalker community prioritization both faithfully captures well-validated communities and identifies functional modules that may correspond to more novel biology. The CommWalker algorithm is freely available at opig.stats.ox.ac.uk/resources or as a docker image on the Docker Hub at hub.docker.com/r/lueckenmd/commwalker/. deane@stats.ox.ac.uk. Supplementary data are available at Bioinformatics online.

77 FR 8217 - Evaluating the Usability of Electronic Health Record (EHR) Systems

Federal Register 2010, 2011, 2012, 2013, 2014

2012-02-14

... interface design guidelines for EHRs. Manufacturers interested in participating in this research will be... the usability of health information technology (HIT) systems. NIST research is designed to: (1... develop performance-oriented user interface design guidelines for EHRs, and a framework for assessing the...

Increasing the applicability of wind power projects via a multi-criteria approach: methodology and case study

NASA Astrophysics Data System (ADS)

Polatidis, Heracles; Morales, Jan Borràs

2016-11-01

In this paper a methodological framework for increasing the actual applicability of wind farms is developed and applied. The framework is based on multi-criteria decision aid techniques that perform an integrated technical and societal evaluation of a number of potential wind power projects that are a variation of a pre-existing actual proposal that faces implementation difficulties. A number of evaluation criteria are established and assessed via particular related software or are comparatively evaluated among each other on a semi-qualitative basis. The preference of a diverse audience of pertinent stakeholders can be also incorporated in the overall analysis. The result of the process is an identification of a new project that will exhibit increased actual implementation potential compared with the original proposal. The methodology is tested in a case study of a wind farm in the UK and relevant conclusions are drawn.

Effects of Impression Material, Impression Tray Type, and Type of Partial Edentulism on the Fit of Cobalt-Chromium Partial Denture Frameworks on Initial Clinical Insertion: A Retrospective Clinical Evaluation.

PubMed

Baig, Mirza Rustum; Akbar, Jaber Hussain; Qudeimat, Muawia; Omar, Ridwaan

2018-02-15

To evaluate the effects of impression material, impression tray type, and type of partial edentulism (ie, Kennedy class) on the accuracy of fit of cobalt-chromium (Co-Cr) partial removable dental prostheses (PRDP) in terms of the number of fabricated frameworks required until the attainment of adequate fit. Electronic case documentations of 120 partially edentulous patients provided with Co-Cr PRDP treatment for one or both arches were examined. Statistical analyses of data were performed using analysis of variance and Tukey honest significant difference test to compare the relationships between the different factors and the number of frameworks that needed to be fabricated for each patient (α = .05). Statistical analysis of data derived from 143 records (69 maxillary and 74 mandibular) revealed no significant correlation between impression material, tray type, or Kennedy class and the number of construction attempts for the pooled or individual arch data (P ≥ .05). In PRDP treatment, alginate can be chosen as a first-choice material, and metal stock trays can be a preferred option for making final impressions to fabricate Co-Cr frameworks.

Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video.

PubMed

Gunay, Osman; Toreyin, Behçet Ugur; Kose, Kivanc; Cetin, A Enis

2012-05-01

In this paper, an entropy-functional-based online adaptive decision fusion (EADF) framework is developed for image analysis and computer vision applications. In this framework, it is assumed that the compound algorithm consists of several subalgorithms, each of which yields its own decision as a real number centered around zero, representing the confidence level of that particular subalgorithm. Decision values are linearly combined with weights that are updated online according to an active fusion method based on performing entropic projections onto convex sets describing subalgorithms. It is assumed that there is an oracle, who is usually a human operator, providing feedback to the decision fusion method. A video-based wildfire detection system was developed to evaluate the performance of the decision fusion algorithm. In this case, image data arrive sequentially, and the oracle is the security guard of the forest lookout tower, verifying the decision of the combined algorithm. The simulation results are presented.

Efficient and Flexible Climate Analysis with Python in a Cloud-Based Distributed Computing Framework

NASA Astrophysics Data System (ADS)

Gannon, C.

2017-12-01

As climate models become progressively more advanced, and spatial resolution further improved through various downscaling projects, climate projections at a local level are increasingly insightful and valuable. However, the raw size of climate datasets presents numerous hurdles for analysts wishing to develop customized climate risk metrics or perform site-specific statistical analysis. Four Twenty Seven, a climate risk consultancy, has implemented a Python-based distributed framework to analyze large climate datasets in the cloud. With the freedom afforded by efficiently processing these datasets, we are able to customize and continually develop new climate risk metrics using the most up-to-date data. Here we outline our process for using Python packages such as XArray and Dask to evaluate netCDF files in a distributed framework, StarCluster to operate in a cluster-computing environment, cloud computing services to access publicly hosted datasets, and how this setup is particularly valuable for generating climate change indicators and performing localized statistical analysis.

Validation of the SWMF Magnetosphere: Fields and Particles

NASA Astrophysics Data System (ADS)

Welling, D. T.; Ridley, A. J.

2009-05-01

The Space Weather Modeling Framework has been developed at the University of Michigan to allow many independent space environment numerical models to be executed simultaneously and coupled together to create a more accurate, all-encompassing system. This work explores the capabilities of the framework when using the BATS-R-US MHD code, Rice Convection Model (RCM), the Ridley Ionosphere Model (RIM), and the Polar Wind Outflow Model (PWOM). Ten space weather events, ranging from quiet to extremely stormy periods, are modeled by the framework. All simulations are executed in a manner that mimics an operational environment where fewer resources are available and predictions are required in a timely manner. The results are compared against in-situ measurements of magnetic fields from GOES, Polar, Geotail, and Cluster satellites as well as MPA particle measurements from the LANL geosynchronous spacecraft. Various metrics are calculated to quantify performance. Results when using only two to all four components are compared to evaluate the increase in performance as new physics are included in the system.

ADEpedia: a scalable and standardized knowledge base of Adverse Drug Events using semantic web technology.

PubMed

Jiang, Guoqian; Solbrig, Harold R; Chute, Christopher G

2011-01-01

A source of semantically coded Adverse Drug Event (ADE) data can be useful for identifying common phenotypes related to ADEs. We proposed a comprehensive framework for building a standardized ADE knowledge base (called ADEpedia) through combining ontology-based approach with semantic web technology. The framework comprises four primary modules: 1) an XML2RDF transformation module; 2) a data normalization module based on NCBO Open Biomedical Annotator; 3) a RDF store based persistence module; and 4) a front-end module based on a Semantic Wiki for the review and curation. A prototype is successfully implemented to demonstrate the capability of the system to integrate multiple drug data and ontology resources and open web services for the ADE data standardization. A preliminary evaluation is performed to demonstrate the usefulness of the system, including the performance of the NCBO annotator. In conclusion, the semantic web technology provides a highly scalable framework for ADE data source integration and standard query service.

Integration of electro-anatomical and imaging data of the left ventricle: An evaluation framework.

PubMed

Soto-Iglesias, David; Butakoff, Constantine; Andreu, David; Fernández-Armenta, Juan; Berruezo, Antonio; Camara, Oscar

2016-08-01

Integration of electrical and structural information for scar characterization in the left ventricle (LV) is a crucial step to better guide radio-frequency ablation therapies, which are usually performed in complex ventricular tachycardia (VT) cases. This integration requires finding a common representation where to map the electrical information from the electro-anatomical map (EAM) surfaces and tissue viability information from delay-enhancement magnetic resonance images (DE-MRI). However, the development of a consistent integration method is still an open problem due to the lack of a proper evaluation framework to assess its accuracy. In this paper we present both: (i) an evaluation framework to assess the accuracy of EAM and imaging integration strategies with simulated EAM data and a set of global and local measures; and (ii) a new integration methodology based on a planar disk representation where the LV surface meshes are quasi-conformally mapped (QCM) by flattening, allowing for simultaneous visualization and joint analysis of the multi-modal data. The developed evaluation framework was applied to estimate the accuracy of the QCM-based integration strategy on a benchmark dataset of 128 synthetically generated ground-truth cases presenting different scar configurations and EAM characteristics. The obtained results demonstrate a significant reduction in global overlap errors (50-100%) with respect to state-of-the-art integration techniques, also better preserving the local topology of small structures such as conduction channels in scars. Data from seventeen VT patients were also used to study the feasibility of the QCM technique in a clinical setting, consistently outperforming the alternative integration techniques in the presence of sparse and noisy clinical data. The proposed evaluation framework has allowed a rigorous comparison of different EAM and imaging data integration strategies, providing useful information to better guide clinical practice in complex cardiac interventions. Copyright © 2016 Elsevier B.V. All rights reserved.

Providing a Science Base for the Evaluation of Tobacco Products

PubMed Central

Berman, Micah L.; Connolly, Greg; Cummings, K. Michael; Djordjevic, Mirjana V.; Hatsukami, Dorothy K.; Henningfield, Jack E.; Myers, Matthew; O'Connor, Richard J.; Parascandola, Mark; Rees, Vaughan; Rice, Jerry M.

2015-01-01

Objective Evidence-based tobacco regulation requires a comprehensive scientific framework to guide the evaluation of new tobacco products and health-related claims made by product manufacturers. Methods The Tobacco Product Assessment Consortium (TobPRAC) employed an iterative process involving consortia investigators, consultants, a workshop of independent scientists and public health experts, and written reviews in order to develop a conceptual framework for evaluating tobacco products. Results The consortium developed a four-phased framework for the scientific evaluation of tobacco products. The four phases addressed by the framework are: (1) pre-market evaluation, (2) pre-claims evaluation, (3) post-market activities, and (4) monitoring and re-evaluation. For each phase, the framework proposes the use of validated testing procedures that will evaluate potential harms at both the individual and population level. Conclusions While the validation of methods for evaluating tobacco products is an ongoing and necessary process, the proposed framework need not wait for fully validated methods to be used in guiding tobacco product regulation today. PMID:26665160

Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review.

PubMed

Rixon, Lorna; Baron, Justine; McGale, Nadine; Lorencatto, Fabiana; Francis, Jill; Davies, Anna

2016-11-18

The American Behaviour Change Consortium (BCC) framework acknowledges patients as active participants and supports the need to investigate the fidelity with which they receive interventions, i.e. receipt. According to this framework, addressing receipt consists in using strategies to assess or enhance participants' understanding and/or performance of intervention skills. This systematic review aims to establish the frequency with which receipt is addressed as defined in the BCC framework in health research, and to describe the methods used in papers informed by the BCC framework and in the wider literature. A forward citation search on papers presenting the BCC framework was performed to determine the frequency with which receipt as defined in this framework was addressed. A second electronic database search, including search terms pertaining to fidelity, receipt, health and process evaluations was performed to identify papers reporting on receipt in the wider literature and irrespective of the framework used. These results were combined with forward citation search results to review methods to assess receipt. Eligibility criteria and data extraction forms were developed and applied to papers. Results are described in a narrative synthesis. 19.6% of 33 studies identified from the forward citation search to report on fidelity were found to address receipt. In 60.6% of these, receipt was assessed in relation to understanding and in 42.4% in relation to performance of skill. Strategies to enhance these were present in 12.1% and 21.1% of studies, respectively. Fifty-five studies were included in the review of the wider literature. Several frameworks and operationalisations of receipt were reported, but the latter were not always consistent with the guiding framework. Receipt was most frequently operationalised in relation to intervention content (16.4%), satisfaction (14.5%), engagement (14.5%), and attendance (14.5%). The majority of studies (90.0%) included subjective assessments of receipt. These relied on quantitative (76.0%) rather than qualitative (42.0%) methods and studies collected data on intervention recipients (50.0%), intervention deliverers (28.0%), or both (22.0%). Few studies (26.0%) reported on the reliability or validity of methods used. Receipt is infrequently addressed in health research and improvements to methods of assessment and reporting are required.

Building an Evaluation Framework for the VIC Model in the NLDAS Testbed

NASA Astrophysics Data System (ADS)

Xia, Y.; Mocko, D. M.; Wang, S.; Pan, M.; Kumar, S.; Peters-Lidard, C. D.; Wei, H.; Ek, M. B.

2017-12-01

Since the second phase of North American Land Data Assimilation System (NLDAS-2) was operationally implemented at NCEP in August 2014, developing the third phase of NLDAS system (NLDAS-3) has been a key task for the NCEP and NASA NLDAS team. The Variable Infiltration Capacity (VIC) model is one major component of the NLDAS system. The current operational NLDAS-2 uses version 4.0.3 (VIC403), research NLDAS-2 uses version 4.0.5 (VIC405), and LIS-based (Land Information System) NLDAS uses version 4.1.2 (VIC412). The purpose of this study is to compressively evaluate three versions and document changes in model behavior towards VIC412 for NLDAS-3. To do that, we develop a relatively comprehensive framework including multiple variables and metrics to assess the performance of different versions. This framework is being incorporated into the NASA Land Verification Toolkit (LVT) for evaluation of other LSMs for NLDAS-3 development. The evaluation results show that there are large and significant improvements for VIC412 in southeastern United States when compared with VIC403 and VIC405. In the other regions, there are very limited improvements or even some degree of deteriorations. Potential reasons are due to: (1) few USGS streamflow observations for soil and hydrologic parameter calibration, (2) the lack of re-calibration of VIC412 in the NLDAS domain, and (3) changes in model physics from VIC403 to VIC412. Overall, the model version upgrade largely/significantly enhances model performance and skill score for all United States except for the Great Plains, suggesting a right direction for VIC model development. Some further efforts are needed for science understanding of land surface physical processes in GP and a re-calibration for VIC412 using reasonable reference datasets is suggested.

Disaster Metrics: A Comprehensive Framework for Disaster Evaluation Typologies.

PubMed

Wong, Diana F; Spencer, Caroline; Boyd, Lee; Burkle, Frederick M; Archer, Frank

2017-10-01

Introduction The frequency of disasters is increasing around the world with more people being at risk. There is a moral imperative to improve the way in which disaster evaluations are undertaken and reported with the aim of reducing preventable mortality and morbidity in future events. Disasters are complex events and undertaking disaster evaluations is a specialized area of study at an international level. Hypothesis/Problem While some frameworks have been developed to support consistent disaster research and evaluation, they lack validation, consistent terminology, and standards for reporting across the different phases of a disaster. There is yet to be an agreed, comprehensive framework to structure disaster evaluation typologies. The aim of this paper is to outline an evolving comprehensive framework for disaster evaluation typologies. It is anticipated that this new framework will facilitate an agreement on identifying, structuring, and relating the various evaluations found in the disaster setting with a view to better understand the process, outcomes, and impacts of the effectiveness and efficiency of interventions. Research was undertaken in two phases: (1) a scoping literature review (peer-reviewed and "grey literature") was undertaken to identify current evaluation frameworks and typologies used in the disaster setting; and (2) a structure was developed that included the range of typologies identified in Phase One and suggests possible relationships in the disaster setting. No core, unifying framework to structure disaster evaluation and research was identified in the literature. The authors propose a "Comprehensive Framework for Disaster Evaluation Typologies" that identifies, structures, and suggests relationships for the various typologies detected. The proposed Comprehensive Framework for Disaster Evaluation Typologies outlines the different typologies of disaster evaluations that were identified in this study and brings them together into a single framework. This unique, unifying framework has relevance at an international level and is expected to benefit the disaster, humanitarian, and development sectors. The next step is to undertake a validation process that will include international leaders with experience in evaluation, in general, and disasters specifically. This work promotes an environment for constructive dialogue on evaluations in the disaster setting to strengthen the evidence base for interventions across the disaster spectrum. It remains a work in progress. Wong DF , Spencer C , Boyd L , Burkle FM Jr. , Archer F . Disaster metrics: a comprehensive framework for disaster evaluation typologies. Prehosp Disaster Med. 2017;32(5):501-514.

Objective Fidelity Evaluation in Multisensory Virtual Environments: Auditory Cue Fidelity in Flight Simulation

PubMed Central

Meyer, Georg F.; Wong, Li Ting; Timson, Emma; Perfect, Philip; White, Mark D.

2012-01-01

We argue that objective fidelity evaluation of virtual environments, such as flight simulation, should be human-performance-centred and task-specific rather than measure the match between simulation and physical reality. We show how principled experimental paradigms and behavioural models to quantify human performance in simulated environments that have emerged from research in multisensory perception provide a framework for the objective evaluation of the contribution of individual cues to human performance measures of fidelity. We present three examples in a flight simulation environment as a case study: Experiment 1: Detection and categorisation of auditory and kinematic motion cues; Experiment 2: Performance evaluation in a target-tracking task; Experiment 3: Transferrable learning of auditory motion cues. We show how the contribution of individual cues to human performance can be robustly evaluated for each task and that the contribution is highly task dependent. The same auditory cues that can be discriminated and are optimally integrated in experiment 1, do not contribute to target-tracking performance in an in-flight refuelling simulation without training, experiment 2. In experiment 3, however, we demonstrate that the auditory cue leads to significant, transferrable, performance improvements with training. We conclude that objective fidelity evaluation requires a task-specific analysis of the contribution of individual cues. PMID:22957068

A framework for designing and analyzing binary decision-making strategies in cellular systems†

PubMed Central

Porter, Joshua R.; Andrews, Burton W.; Iglesias, Pablo A.

2015-01-01

Cells make many binary (all-or-nothing) decisions based on noisy signals gathered from their environment and processed through noisy decision-making pathways. Reducing the effect of noise to improve the fidelity of decision-making comes at the expense of increased complexity, creating a tradeoff between performance and metabolic cost. We present a framework based on rate distortion theory, a branch of information theory, to quantify this tradeoff and design binary decision-making strategies that balance low cost and accuracy in optimal ways. With this framework, we show that several observed behaviors of binary decision-making systems, including random strategies, hysteresis, and irreversibility, are optimal in an information-theoretic sense for various situations. This framework can also be used to quantify the goals around which a decision-making system is optimized and to evaluate the optimality of cellular decision-making systems by a fundamental information-theoretic criterion. As proof of concept, we use the framework to quantify the goals of the externally triggered apoptosis pathway. PMID:22370552

A Decision Support Framework for Feasibility Analysis of International Space Station (ISS) Research Capability Enhancing Options

NASA Technical Reports Server (NTRS)

Ortiz, James N.; Scott,Kelly; Smith, Harold

2004-01-01

The assembly and operation of the ISS has generated significant challenges that have ultimately impacted resources available to the program's primary mission: research. To address this, program personnel routinely perform trade-off studies on alternative options to enhance research. The approach, content level of analysis and resulting outputs of these studies vary due to many factors, however, complicating the Program Manager's job of selecting the best option. To address this, the program requested a framework be developed to evaluate multiple research-enhancing options in a thorough, disciplined and repeatable manner, and to identify the best option on the basis of cost, benefit and risk. The resulting framework consisted of a systematic methodology and a decision-support toolset. The framework provides quantifiable and repeatable means for ranking research-enhancing options for the complex and multiple-constraint domain of the space research laboratory. This paper describes the development, verification and validation of this framework and provides observations on its operational use.

Π4U: A high performance computing framework for Bayesian uncertainty quantification of complex models

NASA Astrophysics Data System (ADS)

Hadjidoukas, P. E.; Angelikopoulos, P.; Papadimitriou, C.; Koumoutsakos, P.

2015-03-01

We present Π4U, an extensible framework, for non-intrusive Bayesian Uncertainty Quantification and Propagation (UQ+P) of complex and computationally demanding physical models, that can exploit massively parallel computer architectures. The framework incorporates Laplace asymptotic approximations as well as stochastic algorithms, along with distributed numerical differentiation and task-based parallelism for heterogeneous clusters. Sampling is based on the Transitional Markov Chain Monte Carlo (TMCMC) algorithm and its variants. The optimization tasks associated with the asymptotic approximations are treated via the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). A modified subset simulation method is used for posterior reliability measurements of rare events. The framework accommodates scheduling of multiple physical model evaluations based on an adaptive load balancing library and shows excellent scalability. In addition to the software framework, we also provide guidelines as to the applicability and efficiency of Bayesian tools when applied to computationally demanding physical models. Theoretical and computational developments are demonstrated with applications drawn from molecular dynamics, structural dynamics and granular flow.

Research recruitment: A marketing framework to improve sample representativeness in health research.

PubMed

Howcutt, Sarah J; Barnett, Anna L; Barbosa-Boucas, Sofia; Smith, Lesley A

2018-04-01

This discussion paper proposes a five-part theoretical framework to inform recruitment strategies. The framework is based on a marketing model of consumer decision-making. Respondents in surveys are typically healthier than non-respondents, which has an impact on the availability of information about those most in need. Previous research has identified response patterns, provided theories about why people participate in research and evaluated different recruitment strategies. Social marketing has been applied successfully to recruitment and promotes focus on the needs of the participant, but little attention has been paid to the periods before and after participant-researcher contact (during advertising and following completion of studies). We propose a new model which conceptualises participation as a decision involving motivation, perception of information, attitude formation, integration of intention and action and finally evaluation and sharing of experience. Discussion paper. This discussion paper presents a critical review. No literature was excluded on date and the included citations span the years 1981-2017. The proposed framework suggests that researchers could engage a broader demographic if they shape research design and advertising to perform functions that participants are seeking to achieve. The framework provides a novel and useful conceptualisation of recruitment which could help to inform public engagement in research design, researcher training and research policy. This framework challenges researchers to investigate the goals of the potential participants when designing a study's advertising and procedures. © 2017 John Wiley & Sons Ltd.

Reframing Evaluation: Defining an Indigenous Evaluation Framework

ERIC Educational Resources Information Center

LaFrance, Joan; Nichols, Richard

2008-01-01

The American Indian Higher Education Consortium (AIHEC), comprising 34 American Indian tribally controlled colleges and universities, has undertaken a comprehensive effort to develop an "Indigenous Framework for Evaluation" that synthesizes Indigenous ways of knowing and Western evaluation practice. To ground the framework, AIHEC engaged…

Electronic immunization data collection systems: application of an evaluation framework.

PubMed

Heidebrecht, Christine L; Kwong, Jeffrey C; Finkelstein, Michael; Quan, Sherman D; Pereira, Jennifer A; Quach, Susan; Deeks, Shelley L

2014-01-14

Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units. The Centers for Disease Control and Prevention's Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations. Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers' immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform. Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.

A Stochastic Framework for Evaluating Seizure Prediction Algorithms Using Hidden Markov Models

PubMed Central

Wong, Stephen; Gardner, Andrew B.; Krieger, Abba M.; Litt, Brian

2007-01-01

Responsive, implantable stimulation devices to treat epilepsy are now in clinical trials. New evidence suggests that these devices may be more effective when they deliver therapy before seizure onset. Despite years of effort, prospective seizure prediction, which could improve device performance, remains elusive. In large part, this is explained by lack of agreement on a statistical framework for modeling seizure generation and a method for validating algorithm performance. We present a novel stochastic framework based on a three-state hidden Markov model (HMM) (representing interictal, preictal, and seizure states) with the feature that periods of increased seizure probability can transition back to the interictal state. This notion reflects clinical experience and may enhance interpretation of published seizure prediction studies. Our model accommodates clipped EEG segments and formalizes intuitive notions regarding statistical validation. We derive equations for type I and type II errors as a function of the number of seizures, duration of interictal data, and prediction horizon length and we demonstrate the model’s utility with a novel seizure detection algorithm that appeared to predicted seizure onset. We propose this framework as a vital tool for designing and validating prediction algorithms and for facilitating collaborative research in this area. PMID:17021032

Real-time tracking of visually attended objects in virtual environments and its application to LOD.

PubMed

Lee, Sungkil; Kim, Gerard Jounghyun; Choi, Seungmoon

2009-01-01

This paper presents a real-time framework for computationally tracking objects visually attended by the user while navigating in interactive virtual environments. In addition to the conventional bottom-up (stimulus-driven) saliency map, the proposed framework uses top-down (goal-directed) contexts inferred from the user's spatial and temporal behaviors, and identifies the most plausibly attended objects among candidates in the object saliency map. The computational framework was implemented using GPU, exhibiting high computational performance adequate for interactive virtual environments. A user experiment was also conducted to evaluate the prediction accuracy of the tracking framework by comparing objects regarded as visually attended by the framework to actual human gaze collected with an eye tracker. The results indicated that the accuracy was in the level well supported by the theory of human cognition for visually identifying single and multiple attentive targets, especially owing to the addition of top-down contextual information. Finally, we demonstrate how the visual attention tracking framework can be applied to managing the level of details in virtual environments, without any hardware for head or eye tracking.

Modeling Urban Scenarios & Experiments: Fort Indiantown Gap Data Collections Summary and Analysis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Archer, Daniel E.; Bandstra, Mark S.; Davidson, Gregory G.

This report summarizes experimental radiation detector, contextual sensor, weather, and global positioning system (GPS) data collected to inform and validate a comprehensive, operational radiation transport modeling framework to evaluate radiation detector system and algorithm performance. This framework will be used to study the influence of systematic effects (such as geometry, background activity, background variability, environmental shielding, etc.) on detector responses and algorithm performance using synthetic time series data. This work consists of performing data collection campaigns at a canonical, controlled environment for complete radiological characterization to help construct and benchmark a high-fidelity model with quantified system geometries, detector response functions,more » and source terms for background and threat objects. This data also provides an archival, benchmark dataset that can be used by the radiation detection community. The data reported here spans four data collection campaigns conducted between May 2015 and September 2016.« less

Invention, design and performance of coconut agrowaste fiberboards for ecologically efficacious buildings

NASA Astrophysics Data System (ADS)

Lokko, Mae-ling Jovenes

As global quantities of waste by-products from food production as well as the range of their applications increase, researchers are realizing critical opportunities to transform the burden of underutilized wastes into ecological profits. Within the tropical hot-humid region, where half the world's current and projected future population growth is concentrated, there is a dire demand for building materials to meet ambitious development schemes and rising housing deficits. However, the building sector has largely overlooked the potential of local agricultural wastes to serve as alternatives to energy-intensive, imported building technologies. Industrial ecologists have recently investigated the use of agrowaste biocomposites to replace conventional wood products that use harmful urea-formaldehyde, phenolic and isocyanate resins. Furthermore, developments in the performance of building material systems with respect to cost, energy, air quality management and construction innovation have evolved metrics about what constitutes material 'upcycling' within building life cycle. While these developments have largely been focused on technical and cost performance, much less attention has been paid to addressing deeply-seated social and cultural barriers to adoption that have sedimented over decades of importation. This dissertation evaluates the development coconut agricultural building material systems in four phases: (i) non-toxic, low-energy production of medium-high density boards (500-1200 kg/m3) from coconut fibers and emerging biobinders; (ii) characterization and evaluation of coconut agricultural building materials hygrothermal performance (iii) scaled-up design development of coconut modular building material systems and (iv) development of a value translation framework for the bottom-up distribution of value to stakeholders within the upcycling framework. This integrated design methodological approach is significant to develop ecological thinking around agrowaste building materials, influence social and cultural acceptability and create value translation frameworks that sufficiently characterize the composite value proposition of upcycled building systems.

A Framework for Quality Assurance in Child Welfare.

ERIC Educational Resources Information Center

O'Brien, Mary; Watson, Peter

In their search for new ways to assess their agencies' success in working with children and families, child welfare administrators and senior managers are increasingly seeking regular and reliable sources of information that help them evaluate agency performance, make ongoing decisions, and provide an accurate picture for agency staff and external…

A Comparative Assessment of Greek Universities' Efficiency Using Quantitative Analysis

ERIC Educational Resources Information Center

Katharaki, Maria; Katharakis, George

2010-01-01

In part due to the increased demand for higher education, typical evaluation frameworks for universities often address the key issue of available resource utilisation. This study seeks to estimate the efficiency of 20 public universities in Greece through quantitative analysis (including performance indicators, data envelopment analysis (DEA) and…

An analytical framework to assist decision makers in the use of forest ecosystem model predictions

USDA-ARS?s Scientific Manuscript database

The predictions of most terrestrial ecosystem models originate from deterministic simulations. Relatively few uncertainty evaluation exercises in model outputs are performed by either model developers or users. This issue has important consequences for decision makers who rely on models to develop n...

SWAT ungauged: Hydrological budget and crop yield predictions in the Upper Mississippi River Basin

USDA-ARS?s Scientific Manuscript database

Physically based, distributed hydrologic models are increasingly used in assessments of water resources, best management practices, and climate and land use changes. Model performance evaluation in ungauged basins is an important research topic. In this study, we propose a framework for developing S...

MOOC Learning Experience Design: Issues and Challenges

ERIC Educational Resources Information Center

Fournier, Hélène; Kop, Rita

2015-01-01

This paper will present current work on various frameworks that are aimed at guiding the research, development, and evaluation efforts around Massive Open Online Courses (MOOCs). Initiatives and activities, including current work by the National Research Council (NRC) in the context of Learning and Performance Support Systems and MOOCs, will be…

A unified framework for gesture recognition and spatiotemporal gesture segmentation.

PubMed

Alon, Jonathan; Athitsos, Vassilis; Yuan, Quan; Sclaroff, Stan

2009-09-01

Within the context of hand gesture recognition, spatiotemporal gesture segmentation is the task of determining, in a video sequence, where the gesturing hand is located and when the gesture starts and ends. Existing gesture recognition methods typically assume either known spatial segmentation or known temporal segmentation, or both. This paper introduces a unified framework for simultaneously performing spatial segmentation, temporal segmentation, and recognition. In the proposed framework, information flows both bottom-up and top-down. A gesture can be recognized even when the hand location is highly ambiguous and when information about when the gesture begins and ends is unavailable. Thus, the method can be applied to continuous image streams where gestures are performed in front of moving, cluttered backgrounds. The proposed method consists of three novel contributions: a spatiotemporal matching algorithm that can accommodate multiple candidate hand detections in every frame, a classifier-based pruning framework that enables accurate and early rejection of poor matches to gesture models, and a subgesture reasoning algorithm that learns which gesture models can falsely match parts of other longer gestures. The performance of the approach is evaluated on two challenging applications: recognition of hand-signed digits gestured by users wearing short-sleeved shirts, in front of a cluttered background, and retrieval of occurrences of signs of interest in a video database containing continuous, unsegmented signing in American Sign Language (ASL).

Applicability Analysis of Validation Evidence for Biomedical Computational Models

DOE PAGES

Pathmanathan, Pras; Gray, Richard A.; Romero, Vicente J.; ...

2017-09-07

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibilitymore » of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

Applicability Analysis of Validation Evidence for Biomedical Computational Models

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pathmanathan, Pras; Gray, Richard A.; Romero, Vicente J.

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibilitymore » of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

Value Driven Outcomes (VDO): a pragmatic, modular, and extensible software framework for understanding and improving health care costs and outcomes

PubMed Central

Kawamoto, Kensaku; Martin, Cary J; Williams, Kip; Tu, Ming-Chieh; Park, Charlton G; Hunter, Cheri; Staes, Catherine J; Bray, Bruce E; Deshmukh, Vikrant G; Holbrook, Reid A; Morris, Scott J; Fedderson, Matthew B; Sletta, Amy; Turnbull, James; Mulvihill, Sean J; Crabtree, Gordon L; Entwistle, David E; McKenna, Quinn L; Strong, Michael B; Pendleton, Robert C; Lee, Vivian S

2015-01-01

Objective To develop expeditiously a pragmatic, modular, and extensible software framework for understanding and improving healthcare value (costs relative to outcomes). Materials and methods In 2012, a multidisciplinary team was assembled by the leadership of the University of Utah Health Sciences Center and charged with rapidly developing a pragmatic and actionable analytics framework for understanding and enhancing healthcare value. Based on an analysis of relevant prior work, a value analytics framework known as Value Driven Outcomes (VDO) was developed using an agile methodology. Evaluation consisted of measurement against project objectives, including implementation timeliness, system performance, completeness, accuracy, extensibility, adoption, satisfaction, and the ability to support value improvement. Results A modular, extensible framework was developed to allocate clinical care costs to individual patient encounters. For example, labor costs in a hospital unit are allocated to patients based on the hours they spent in the unit; actual medication acquisition costs are allocated to patients based on utilization; and radiology costs are allocated based on the minutes required for study performance. Relevant process and outcome measures are also available. A visualization layer facilitates the identification of value improvement opportunities, such as high-volume, high-cost case types with high variability in costs across providers. Initial implementation was completed within 6 months, and all project objectives were fulfilled. The framework has been improved iteratively and is now a foundational tool for delivering high-value care. Conclusions The framework described can be expeditiously implemented to provide a pragmatic, modular, and extensible approach to understanding and improving healthcare value. PMID:25324556

MEPROCS framework for Craniofacial Superimposition: Validation study.

PubMed

Ibáñez, O; Vicente, R; Navega, D; Campomanes-Álvarez, C; Cattaneo, C; Jankauskas, R; Huete, M I; Navarro, F; Hardiman, R; Ruiz, E; Imaizumi, K; Cavalli, F; Veselovskaya, E; Humpire, D; Cardoso, J; Collini, F; Mazzarelli, D; Gibelli, D; Damas, S

2016-11-01

Craniofacial Superimposition (CFS) involves the process of overlaying a skull with a number of ante-mortem images of an individual and the analysis of their morphological correspondence. The lack of unified working protocols and the absence of commonly accepted standards, led to contradictory consensus regarding its reliability. One of the more important aims of 'New Methodologies and Protocols of Forensic Identification by Craniofacial Superimposition (MEPROCS)' project was to propose a common framework for CFS, what can be considered the first international standard in the field. The framework aimed to serve as a roadmap for avoiding particular assumptions that could bias the process. At the same time, it provides some empirical support to certain practices, technological means, and morphological criteria expected to facilitate the application of the CFS task and to improve its reliability. In order to confirm the utility and potential benefits of the framework use, there is a need to empirically evaluate it in CFS identification scenarios as close as possible to the reality. Thus, the purpose of this study is to validate the CFS framework developed. For that aim 12 participants were asked to report about a variable number of CFS following all the recommendations of the framework. The results are analysed and discussed according to the framework understanding and fulfilment, the participants' performance, and the correlation between expected decisions and those given by the participants. In view of the quantitative results and qualitative examination criteria we can conclude that those who follow the MEPROCS recommendations improve their performance. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Translating Radiometric Requirements for Satellite Sensors to Match International Standards.

PubMed

Pearlman, Aaron; Datla, Raju; Kacker, Raghu; Cao, Changyong

2014-01-01

International scientific standards organizations created standards on evaluating uncertainty in the early 1990s. Although scientists from many fields use these standards, they are not consistently implemented in the remote sensing community, where traditional error analysis framework persists. For a satellite instrument under development, this can create confusion in showing whether requirements are met. We aim to create a methodology for translating requirements from the error analysis framework to the modern uncertainty approach using the product level requirements of the Advanced Baseline Imager (ABI) that will fly on the Geostationary Operational Environmental Satellite R-Series (GOES-R). In this paper we prescribe a method to combine several measurement performance requirements, written using a traditional error analysis framework, into a single specification using the propagation of uncertainties formula. By using this approach, scientists can communicate requirements in a consistent uncertainty framework leading to uniform interpretation throughout the development and operation of any satellite instrument.

Translating Radiometric Requirements for Satellite Sensors to Match International Standards

PubMed Central

Pearlman, Aaron; Datla, Raju; Kacker, Raghu; Cao, Changyong

2014-01-01

International scientific standards organizations created standards on evaluating uncertainty in the early 1990s. Although scientists from many fields use these standards, they are not consistently implemented in the remote sensing community, where traditional error analysis framework persists. For a satellite instrument under development, this can create confusion in showing whether requirements are met. We aim to create a methodology for translating requirements from the error analysis framework to the modern uncertainty approach using the product level requirements of the Advanced Baseline Imager (ABI) that will fly on the Geostationary Operational Environmental Satellite R-Series (GOES-R). In this paper we prescribe a method to combine several measurement performance requirements, written using a traditional error analysis framework, into a single specification using the propagation of uncertainties formula. By using this approach, scientists can communicate requirements in a consistent uncertainty framework leading to uniform interpretation throughout the development and operation of any satellite instrument. PMID:26601032

Process change evaluation framework for allogeneic cell therapies: impact on drug development and commercialization.

PubMed

Hassan, Sally; Huang, Hsini; Warren, Kim; Mahdavi, Behzad; Smith, David; Jong, Simcha; Farid, Suzanne S

2016-04-01

Some allogeneic cell therapies requiring a high dose of cells for large indication groups demand a change in cell expansion technology, from planar units to microcarriers in single-use bioreactors for the market phase. The aim was to model the optimal timing for making this change. A development lifecycle cash flow framework was created to examine the implications of process changes to microcarrier cultures at different stages of a cell therapy's lifecycle. The analysis performed under assumptions used in the framework predicted that making this switch earlier in development is optimal from a total expected out-of-pocket cost perspective. From a risk-adjusted net present value view, switching at Phase I is economically competitive but a post-approval switch can offer the highest risk-adjusted net present value as the cost of switching is offset by initial market penetration with planar technologies. The framework can facilitate early decision-making during process development.

Fuel Cycle Analysis Framework Base Cases for the IAEA/INPRO GAINS Collaborative Project

DOE Office of Scientific and Technical Information (OSTI.GOV)

Brent Dixon

Thirteen countries participated in the Collaborative Project GAINS “Global Architecture of Innovative Nuclear Energy Systems Based on Thermal and Fast Reactors Including a Closed Fuel Cycle”, which was the primary activity within the IAEA/INPRO Program Area B: “Global Vision on Sustainable Nuclear Energy” for the last three years. The overall objective of GAINS was to develop a standard framework for assessing future nuclear energy systems taking into account sustainable development, and to validate results through sample analyses. This paper details the eight scenarios that constitute the GAINS framework base cases for analysis of the transition to future innovative nuclear energymore » systems. The framework base cases provide a reference for users of the framework to start from in developing and assessing their own alternate systems. Each base case is described along with performance results against the GAINS sustainability evaluation metrics. The eight cases include four using a moderate growth projection and four using a high growth projection for global nuclear electricity generation through 2100. The cases are divided into two sets, addressing homogeneous and heterogeneous scenarios developed by GAINS to model global fuel cycle strategies. The heterogeneous world scenario considers three separate nuclear groups based on their fuel cycle strategies, with non-synergistic and synergistic cases. The framework base case analyses results show the impact of these different fuel cycle strategies while providing references for future users of the GAINS framework. A large number of scenario alterations are possible and can be used to assess different strategies, different technologies, and different assumptions about possible futures of nuclear power. Results can be compared to the framework base cases to assess where these alternate cases perform differently versus the sustainability indicators.« less

Evaluation of CMAQ and CAMx Ensemble Air Quality Forecasts during the 2015 MAPS-Seoul Field Campaign

NASA Astrophysics Data System (ADS)

Kim, E.; Kim, S.; Bae, C.; Kim, H. C.; Kim, B. U.

2015-12-01

The performance of Air quality forecasts during the 2015 MAPS-Seoul Field Campaign was evaluated. An forecast system has been operated to support the campaign's daily aircraft route decisions for airborne measurements to observe long-range transporting plume. We utilized two real-time ensemble systems based on the Weather Research and Forecasting (WRF)-Sparse Matrix Operator Kernel Emissions (SMOKE)-Comprehensive Air quality Model with extensions (CAMx) modeling framework and WRF-SMOKE- Community Multi_scale Air Quality (CMAQ) framework over northeastern Asia to simulate PM10 concentrations. Global Forecast System (GFS) from National Centers for Environmental Prediction (NCEP) was used to provide meteorological inputs for the forecasts. For an additional set of retrospective simulations, ERA Interim Reanalysis from European Centre for Medium-Range Weather Forecasts (ECMWF) was also utilized to access forecast uncertainties from the meteorological data used. Model Inter-Comparison Study for Asia (MICS-Asia) and National Institute of Environment Research (NIER) Clean Air Policy Support System (CAPSS) emission inventories are used for foreign and domestic emissions, respectively. In the study, we evaluate the CMAQ and CAMx model performance during the campaign by comparing the results to the airborne and surface measurements. Contributions of foreign and domestic emissions are estimated using a brute force method. Analyses on model performance and emissions will be utilized to improve air quality forecasts for the upcoming KORUS-AQ field campaign planned in 2016.

Task-specific image partitioning.

PubMed

Kim, Sungwoong; Nowozin, Sebastian; Kohli, Pushmeet; Yoo, Chang D

2013-02-01

Image partitioning is an important preprocessing step for many of the state-of-the-art algorithms used for performing high-level computer vision tasks. Typically, partitioning is conducted without regard to the task in hand. We propose a task-specific image partitioning framework to produce a region-based image representation that will lead to a higher task performance than that reached using any task-oblivious partitioning framework and existing supervised partitioning framework, albeit few in number. The proposed method partitions the image by means of correlation clustering, maximizing a linear discriminant function defined over a superpixel graph. The parameters of the discriminant function that define task-specific similarity/dissimilarity among superpixels are estimated based on structured support vector machine (S-SVM) using task-specific training data. The S-SVM learning leads to a better generalization ability while the construction of the superpixel graph used to define the discriminant function allows a rich set of features to be incorporated to improve discriminability and robustness. We evaluate the learned task-aware partitioning algorithms on three benchmark datasets. Results show that task-aware partitioning leads to better labeling performance than the partitioning computed by the state-of-the-art general-purpose and supervised partitioning algorithms. We believe that the task-specific image partitioning paradigm is widely applicable to improving performance in high-level image understanding tasks.

Analysis of key technologies for virtual instruments metrology

NASA Astrophysics Data System (ADS)

Liu, Guixiong; Xu, Qingui; Gao, Furong; Guan, Qiuju; Fang, Qiang

2008-12-01

Virtual instruments (VIs) require metrological verification when applied as measuring instruments. Owing to the software-centered architecture, metrological evaluation of VIs includes two aspects: measurement functions and software characteristics. Complexity of software imposes difficulties on metrological testing of VIs. Key approaches and technologies for metrology evaluation of virtual instruments are investigated and analyzed in this paper. The principal issue is evaluation of measurement uncertainty. The nature and regularity of measurement uncertainty caused by software and algorithms can be evaluated by modeling, simulation, analysis, testing and statistics with support of powerful computing capability of PC. Another concern is evaluation of software features like correctness, reliability, stability, security and real-time of VIs. Technologies from software engineering, software testing and computer security domain can be used for these purposes. For example, a variety of black-box testing, white-box testing and modeling approaches can be used to evaluate the reliability of modules, components, applications and the whole VI software. The security of a VI can be assessed by methods like vulnerability scanning and penetration analysis. In order to facilitate metrology institutions to perform metrological verification of VIs efficiently, an automatic metrological tool for the above validation is essential. Based on technologies of numerical simulation, software testing and system benchmarking, a framework for the automatic tool is proposed in this paper. Investigation on implementation of existing automatic tools that perform calculation of measurement uncertainty, software testing and security assessment demonstrates the feasibility of the automatic framework advanced.

Renewing focus on family planning service quality globally.

PubMed

Hancock, Nancy L; Stuart, Gretchen S; Tang, Jennifer H; Chibwesha, Carla J; Stringer, Jeffrey S A; Chi, Benjamin H

2016-01-01

Reducing the global unmet need for contraception is currently a priority for many governments, multi-lateral initiatives, non-governmental organizations, and donors. Evidence strongly suggests that the provision of quality family planning services can increase uptake, prevalence, and continuation of contraception. While an accepted framework to define the components of family planning service quality exists, translating this framework into assessment tools that are accessible, easily utilized, and valid for service providers has remained a challenge. We propose new approaches to improve the standardization and accessibility of family planning service quality assessment tools to simplify family planning service quality evaluation. With easier approaches to program evaluation, quality improvements can be performed more swiftly to help increase uptake and continuation of contraception to improve the health of women and their families.

Developing biodiversity indicators on a stakeholders' opinions basis: the gypsum industry Key Performance Indicators framework.

PubMed

Pitz, Carline; Mahy, Grégory; Vermeulen, Cédric; Marlet, Christine; Séleck, Maxime

2016-07-01

This study aims to establish a common Key Performance Indicators (KPIs) framework for reporting about the gypsum industry biodiversity at the European level. In order to integrate different opinions and to reach a consensus framework, an original participatory process approach has been developed among different stakeholder groups: Eurogypsum, European and regional authorities, university scientists, consulting offices, European and regional associations for the conservation of nature, and the extractive industry. The strategy is developed around four main steps: (1) building of a maximum set of indicators to be submitted to stakeholders based on the literature (Focus Group method); (2) evaluating the consensus about indicators through a policy Delphi survey aiming at the prioritization of indicator classes using the Analytic Hierarchy Process method (AHP) and of individual indicators; (3) testing acceptability and feasibility through analysis of Environmental Impact Assessments (EIAs) and visits to three European quarries; (4) Eurogypsum final decision and communication. The resulting framework contains a set of 11 indicators considered the most suitable for all the stakeholders. Our KPIs respond to European legislation and strategies for biodiversity. The framework aims at improving sustainability in quarries and at helping to manage biodiversity as well as to allow the creation of coherent reporting systems. The final goal is to allow for the definition of the actual biodiversity status of gypsum quarries and allow for enhancing it. The framework is adaptable to the local context of each gypsum quarry.

Evolution of a multilevel framework for health program evaluation.

PubMed

Masso, Malcolm; Quinsey, Karen; Fildes, Dave

2017-07-01

A well-conceived evaluation framework increases understanding of a program's goals and objectives, facilitates the identification of outcomes and can be used as a planning tool during program development. Herein we describe the origins and development of an evaluation framework that recognises that implementation is influenced by the setting in which it takes place, the individuals involved and the processes by which implementation is accomplished. The framework includes an evaluation hierarchy that focuses on outcomes for consumers, providers and the care delivery system, and is structured according to six domains: program delivery, impact, sustainability, capacity building, generalisability and dissemination. These components of the evaluation framework fit into a matrix structure, and cells within the matrix are supported by relevant evaluation tools. The development of the framework has been influenced by feedback from various stakeholders, existing knowledge of the evaluators and the literature on health promotion and implementation science. Over the years, the framework has matured and is generic enough to be useful in a wide variety of circumstances, yet specific enough to focus data collection, data analysis and the presentation of findings.

Goal orientation and self-efficacy in relation to memory in adulthood

PubMed Central

Hastings, Erin C.; West, Robin L.

2011-01-01

The achievement goal framework (Dweck, 1986) has been well-established in children and college-students, but has rarely been examined empirically with older adults. The current study, including younger and older adults, examined the effects of memory self-efficacy, learning goals (focusing on skill mastery over time) and performance goals (focusing on performance outcome evaluations) on memory performance. Questionnaires measured memory self-efficacy and general orientation toward learning and performance goals; free and cued recall was assessed in a subsequent telephone interview. As expected, age was negatively related and education was positively related to memory self-efficacy, and memory self-efficacy was positively related to memory, in a structural equation model. Age was also negatively related to memory performance. Results supported the positive impact of learning goals and the negative impact of performance goals on memory self-efficacy. There was no significant direct effect of learning or performance goals on memory performance; their impact occurred via their effect on memory self-efficacy. The present study supports past research suggesting that learning goals are beneficial, and performance goals are maladaptive, for self-efficacy and learning, and validates the achievement goal framework in a sample including older adults. PMID:21728891

Improving Quality and Reducing Waste in Allied Health Workplace Education Programs: A Pragmatic Operational Education Framework Approach.

PubMed

Golder, Janet; Farlie, Melanie K; Sevenhuysen, Samantha

2016-01-01

Efficient utilisation of education resources is required for the delivery of effective learning opportunities for allied health professionals. This study aimed to develop an education framework to support delivery of high-quality education within existing education resources. This study was conducted in a large metropolitan health service. Homogenous and purposive sampling methods were utilised in Phase 1 (n=43) and 2 (n=14) consultation stages. Participants included 25 allied health professionals, 22 managers, 1 educator, and 3 executives. Field notes taken during 43 semi-structured interviews and 4 focus groups were member-checked, and semantic thematic analysis methods were utilised. Framework design was informed by existing published framework development guides. The framework model contains governance, planning, delivery, and evaluation and research elements and identifies performance indicators, practice examples, and support tools for a range of stakeholders. Themes integrated into framework content include improving quality of education and training provided and delivery efficiency, greater understanding of education role requirements, and workforce support for education-specific knowledge and skill development. This framework supports efficient delivery of allied health workforce education and training to the highest standard, whilst pragmatically considering current allied health education workforce demands.

Conceptual Privacy Framework for Health Information on Wearable Device

PubMed Central

Safavi, Seyedmostafa; Shukur, Zarina

2014-01-01

Wearable health tech provides doctors with the ability to remotely supervise their patients' wellness. It also makes it much easier to authorize someone else to take appropriate actions to ensure the person's wellness than ever before. Information Technology may soon change the way medicine is practiced, improving the performance, while reducing the price of healthcare. We analyzed the secrecy demands of wearable devices, including Smartphone, smart watch and their computing techniques, that can soon change the way healthcare is provided. However, before this is adopted in practice, all devices must be equipped with sufficient privacy capabilities related to healthcare service. In this paper, we formulated a new improved conceptual framework for wearable healthcare systems. This framework consists of ten principles and nine checklists, capable of providing complete privacy protection package to wearable device owners. We constructed this framework based on the analysis of existing mobile technology, the results of which are combined with the existing security standards. The approach also incorporates the market share percentage level of every app and its respective OS. This framework is evaluated based on the stringent CIA and HIPAA principles for information security. This evaluation is followed by testing the capability to revoke rights of subjects to access objects and ability to determine the set of available permissions for a particular subject for all models Finally, as the last step, we examine the complexity of the required initial setup. PMID:25478915

Autonomic Management of Application Workflows on Hybrid Computing Infrastructure

DOE PAGES

Kim, Hyunjoo; el-Khamra, Yaakoub; Rodero, Ivan; ...

2011-01-01

In this paper, we present a programming and runtime framework that enables the autonomic management of complex application workflows on hybrid computing infrastructures. The framework is designed to address system and application heterogeneity and dynamics to ensure that application objectives and constraints are satisfied. The need for such autonomic system and application management is becoming critical as computing infrastructures become increasingly heterogeneous, integrating different classes of resources from high-end HPC systems to commodity clusters and clouds. For example, the framework presented in this paper can be used to provision the appropriate mix of resources based on application requirements and constraints.more » The framework also monitors the system/application state and adapts the application and/or resources to respond to changing requirements or environment. To demonstrate the operation of the framework and to evaluate its ability, we employ a workflow used to characterize an oil reservoir executing on a hybrid infrastructure composed of TeraGrid nodes and Amazon EC2 instances of various types. Specifically, we show how different applications objectives such as acceleration, conservation and resilience can be effectively achieved while satisfying deadline and budget constraints, using an appropriate mix of dynamically provisioned resources. Our evaluations also demonstrate that public clouds can be used to complement and reinforce the scheduling and usage of traditional high performance computing infrastructure.« less

Conceptual privacy framework for health information on wearable device.

PubMed

Safavi, Seyedmostafa; Shukur, Zarina

2014-01-01

Wearable health tech provides doctors with the ability to remotely supervise their patients' wellness. It also makes it much easier to authorize someone else to take appropriate actions to ensure the person's wellness than ever before. Information Technology may soon change the way medicine is practiced, improving the performance, while reducing the price of healthcare. We analyzed the secrecy demands of wearable devices, including Smartphone, smart watch and their computing techniques, that can soon change the way healthcare is provided. However, before this is adopted in practice, all devices must be equipped with sufficient privacy capabilities related to healthcare service. In this paper, we formulated a new improved conceptual framework for wearable healthcare systems. This framework consists of ten principles and nine checklists, capable of providing complete privacy protection package to wearable device owners. We constructed this framework based on the analysis of existing mobile technology, the results of which are combined with the existing security standards. The approach also incorporates the market share percentage level of every app and its respective OS. This framework is evaluated based on the stringent CIA and HIPAA principles for information security. This evaluation is followed by testing the capability to revoke rights of subjects to access objects and ability to determine the set of available permissions for a particular subject for all models Finally, as the last step, we examine the complexity of the required initial setup.

Supporting the Evaluation and Implementation of Musculoskeletal Models of Care: A Globally Informed Framework for Judging Readiness and Success.

PubMed

Briggs, Andrew M; Jordan, Joanne E; Jennings, Matthew; Speerin, Robyn; Bragge, Peter; Chua, Jason; Woolf, Anthony D; Slater, Helen

2017-04-01

To develop a globally informed framework to evaluate readiness for implementation and success after implementation of musculoskeletal models of care (MOCs). Three phases were undertaken: 1) a qualitative study with 27 Australian subject matter experts (SMEs) to develop a draft framework; 2) an eDelphi study with an international panel of 93 SMEs across 30 nations to evaluate face validity, and refine and establish consensus on the framework components; and 3) translation of the framework into a user-focused resource and evaluation of its acceptability with the eDelphi panel. A comprehensive evaluation framework was developed for judging the readiness and success of musculoskeletal MOCs. The framework consists of 9 domains, with each domain containing a number of themes underpinned by detailed elements. In the first Delphi round, scores of "partly agree" or "completely agree" with the draft framework ranged 96.7%-100%. In the second round, "essential" scores ranged 58.6%-98.9%, resulting in 14 of 34 themes being classified as essential. SMEs strongly agreed or agreed that the final framework was useful (98.8%), usable (95.1%), credible (100%) and appealing (93.9%). Overall, 96.3% strongly supported or supported the final structure of the framework as it was presented, while 100%, 96.3%, and 100% strongly supported or supported the content within the readiness, initiating implementation, and success streams, respectively. An empirically derived framework to evaluate the readiness and success of musculoskeletal MOCs was strongly supported by an international panel of SMEs. The framework provides an important internationally applicable benchmark for the development, implementation, and evaluation of musculoskeletal MOCs. © 2016, American College of Rheumatology.

Framework for measuring adaptive knowledge-rich systems performance.

PubMed

Bushko, Renata G

2005-01-01

The universe is non repeatable in nature--most of events cannot be prestated and do not repeat themselves. The only way to create systems that are truly useful is to make them adaptive (able to reason by analogy and learn) and rich in knowledge (including common sense knowledge). Adaptive and knowledge-rich health management could get us closer to errorless health care where small incremental adjustments happen all the time preventing occurrence of an error. In the era of adaptive systems we need to have a way to evaluate their performance. Are they truly adaptive? How adaptive are they? Are they accurate enough? Are they fast enough? Are they cost effective? This chapter presents general framework for measuring adaptive knowledge-rich systems' performance and includes among others definitions of adaptiveness factor, britt (a unit of brittleness) and uso-quant (unit of usefulness of a piece of knowledge). Measuring adaptive knowledge-rich systems performance is one of the most important research areas that can have a big pay-off in healthcare now and in the future.

Comparative Analysis of Two Industries for Validating Green Manufacturing (GM) Framework: An Indian Scenario

NASA Astrophysics Data System (ADS)

Rehman, Minhaj Ahemad Abdul; Shrivastava, Rakesh Lakshmikumar; Shrivastava, Rashmi Rakesh

2017-04-01

Green Manufacturing (GM) deals with manufacturing practices that reduces or eliminates the adverse environmental impact during any of its phases. It emphasizes the use of processes that do not contaminate the environment or hurt consumers, employees, or other stakeholders. This paper presents the comparative analysis of two Indian industries representing different sectors for validating GM framework. It also highlights the road map of the companies for achieving performance improvement through GM implementation and its impact on organisational performance. The case studies helps in evaluating the companies GM implementation and overall business performance. For this, a developed diagnostic instrument in the form of questionnaire was administered amongst employees in the companies respectively and their responses were analysed. In order to have a better understanding of the impact of GM implementation, the information about overall business performance was obtained over the last 3 years. The diagnostic instrument developed here may be used by manufacturing organisations to prioritise their management efforts to assess and implement GM.

Audio-based performance evaluation of squash players

PubMed Central

Hajdú-Szücs, Katalin; Fenyvesi, Nóra; Vattay, Gábor

2018-01-01

In competitive sports it is often very hard to quantify the performance. A player to score or overtake may depend on only millesimal of seconds or millimeters. In racquet sports like tennis, table tennis and squash many events will occur in a short time duration, whose recording and analysis can help reveal the differences in performance. In this paper we show that it is possible to architect a framework that utilizes the characteristic sound patterns to precisely classify the types of and localize the positions of these events. From these basic information the shot types and the ball speed along the trajectories can be estimated. Comparing these estimates with the optimal speed and target the precision of the shot can be defined. The detailed shot statistics and precision information significantly enriches and improves data available today. Feeding them back to the players and the coaches facilitates to describe playing performance objectively and to improve strategy skills. The framework is implemented, its hardware and software components are installed and tested in a squash court. PMID:29579067

Occlusion handling framework for tracking in smart camera networks by per-target assistance task assignment

NASA Astrophysics Data System (ADS)

Bo, Nyan Bo; Deboeverie, Francis; Veelaert, Peter; Philips, Wilfried

2017-09-01

Occlusion is one of the most difficult challenges in the area of visual tracking. We propose an occlusion handling framework to improve the performance of local tracking in a smart camera view in a multicamera network. We formulate an extensible energy function to quantify the quality of a camera's observation of a particular target by taking into account both person-person and object-person occlusion. Using this energy function, a smart camera assesses the quality of observations over all targets being tracked. When it cannot adequately observe of a target, a smart camera estimates the quality of observation of the target from view points of other assisting cameras. If a camera with better observation of the target is found, the tracking task of the target is carried out with the assistance of that camera. In our framework, only positions of persons being tracked are exchanged between smart cameras. Thus, communication bandwidth requirement is very low. Performance evaluation of our method on challenging video sequences with frequent and severe occlusions shows that the accuracy of a baseline tracker is considerably improved. We also report the performance comparison to the state-of-the-art trackers in which our method outperforms.

An efficient framework for Java data processing systems in HPC environments

NASA Astrophysics Data System (ADS)

Fries, Aidan; Castañeda, Javier; Isasi, Yago; Taboada, Guillermo L.; Portell de Mora, Jordi; Sirvent, Raül

2011-11-01

Java is a commonly used programming language, although its use in High Performance Computing (HPC) remains relatively low. One of the reasons is a lack of libraries offering specific HPC functions to Java applications. In this paper we present a Java-based framework, called DpcbTools, designed to provide a set of functions that fill this gap. It includes a set of efficient data communication functions based on message-passing, thus providing, when a low latency network such as Myrinet is available, higher throughputs and lower latencies than standard solutions used by Java. DpcbTools also includes routines for the launching, monitoring and management of Java applications on several computing nodes by making use of JMX to communicate with remote Java VMs. The Gaia Data Processing and Analysis Consortium (DPAC) is a real case where scientific data from the ESA Gaia astrometric satellite will be entirely processed using Java. In this paper we describe the main elements of DPAC and its usage of the DpcbTools framework. We also assess the usefulness and performance of DpcbTools through its performance evaluation and the analysis of its impact on some DPAC systems deployed in the MareNostrum supercomputer (Barcelona Supercomputing Center).

Evaluation of non-negative matrix factorization of grey matter in age prediction.

PubMed

Varikuti, Deepthi P; Genon, Sarah; Sotiras, Aristeidis; Schwender, Holger; Hoffstaedter, Felix; Patil, Kaustubh R; Jockwitz, Christiane; Caspers, Svenja; Moebus, Susanne; Amunts, Katrin; Davatzikos, Christos; Eickhoff, Simon B

2018-06-01

The relationship between grey matter volume (GMV) patterns and age can be captured by multivariate pattern analysis, allowing prediction of individuals' age based on structural imaging. Raw data, voxel-wise GMV and non-sparse factorization (with Principal Component Analysis, PCA) show good performance but do not promote relatively localized brain components for post-hoc examinations. Here we evaluated a non-negative matrix factorization (NNMF) approach to provide a reduced, but also interpretable representation of GMV data in age prediction frameworks in healthy and clinical populations. This examination was performed using three datasets: a multi-site cohort of life-span healthy adults, a single site cohort of older adults and clinical samples from the ADNI dataset with healthy subjects, participants with Mild Cognitive Impairment and patients with Alzheimer's disease (AD) subsamples. T1-weighted images were preprocessed with VBM8 standard settings to compute GMV values after normalization, segmentation and modulation for non-linear transformations only. Non-negative matrix factorization was computed on the GM voxel-wise values for a range of granularities (50-690 components) and LASSO (Least Absolute Shrinkage and Selection Operator) regression were used for age prediction. First, we compared the performance of our data compression procedure (i.e., NNMF) to various other approaches (i.e., uncompressed VBM data, PCA-based factorization and parcellation-based compression). We then investigated the impact of the granularity on the accuracy of age prediction, as well as the transferability of the factorization and model generalization across datasets. We finally validated our framework by examining age prediction in ADNI samples. Our results showed that our framework favorably compares with other approaches. They also demonstrated that the NNMF based factorization derived from one dataset could be efficiently applied to compress VBM data of another dataset and that granularities between 300 and 500 components give an optimal representation for age prediction. In addition to the good performance in healthy subjects our framework provided relatively localized brain regions as the features contributing to the prediction, thereby offering further insights into structural changes due to brain aging. Finally, our validation in clinical populations showed that our framework is sensitive to deviance from normal structural variations in pathological aging. Copyright © 2018 Elsevier Inc. All rights reserved.

How to improve patient retention in an antiretroviral treatment program in Ethiopia: a mixed-methods study

PubMed Central

2014-01-01

Background Patient retention, defined as continuous engagement of patients in care, is one of the crucial indicators for monitoring and evaluating the performance of antiretroviral treatment (ART) programs. It has been identified that suboptimal patient retention in care is one of the challenges of ART programs in many settings. ART programs have, therefore, been striving hard to identify and implement interventions that improve their suboptimal levels of retention. The objective of this study was to develop a framework for improving patient retention in care based on interventions implemented in health facilities that have achieved higher levels of retention in care. Methods A mixed-methods study, based on the positive deviance approach, was conducted in Ethiopia in 2011/12. Quantitative data were collected to estimate and compare the levels of retention in care in nine health facilities. Key informant interviews and focus group discussions were conducted to identify a package of interventions implemented in the health facilities with relatively higher or improving levels of retention. Results Retention in care in the Ethiopian ART program was found to be variable across health facilities. Among hospitals, the poorest performer had 0.46 (0.35, 0.60) times less retention than the reference; among health centers, the poorest performers had 0.44 (0.28, 0.70) times less retention than the reference. Health facilities with higher and improving patient retention were found to implement a comprehensive package of interventions: (1) retention promoting activities by health facilities, (2) retention promoting activities by community-based organizations, (3) coordination of these activities by case manager(s), and (4) patient information systems by data clerk(s). On the contrary, such interventions were either poorly implemented or did not exist in health facilities with lower retention in care. A framework to improve retention in care was developed based on the evidence found by applying the positive deviance approach. Conclusion A framework for improving retention in care of patients on ART was developed. We recommend that health facilities implement the framework, monitor and evaluate their levels of retention in care, and, if necessary, adapt the framework to their own contexts. PMID:24475889

Using developmental evaluation as a system of organizational learning: An example from San Francisco.

PubMed

Shea, Jennifer; Taylor, Tory

2017-12-01

In the last 20 years, developmental evaluation has emerged as a promising approach to support organizational learning in emergent social programs. Through a continuous system of inquiry, reflection, and application of knowledge, developmental evaluation serves as a system of tools, methods, and guiding principles intended to support constructive organizational learning. However, missing from the developmental evaluation literature is a nuanced framework to guide evaluators in how to elevate the organizational practices and concepts most relevant for emergent programs. In this article, we describe and reflect on work we did to develop, pilot, and refine an integrated pilot framework. Drawing on established developmental evaluation inquiry frameworks and incorporating lessons learned from applying the pilot framework, we put forward the Evaluation-led Learning framework to help fill that gap and encourage others to implement and refine it. We posit that without explicitly incorporating the assessments at the foundation of the Evaluation-led Learning framework, developmental evaluation's ability to affect organizational learning in productive ways will likely be haphazard and limited. Copyright © 2017 Elsevier Ltd. All rights reserved.

A protocol for evaluating video trackers under real-world conditions.

PubMed

Nawaz, Tahir; Cavallaro, Andrea

2013-04-01

The absence of a commonly adopted performance evaluation framework is hampering advances in the design of effective video trackers. In this paper, we present a single-score evaluation measure and a protocol to objectively compare trackers. The proposed measure evaluates tracking accuracy and failure, and combines them for both summative and formative performance assessment. The proposed protocol is composed of a set of trials that evaluate the robustness of trackers on a range of test scenarios representing several real-world conditions. The protocol is validated on a set of sequences with a diversity of targets (head, vehicle and person) and challenges (occlusions, background clutter, pose changes and scale changes) using six state-of-the-art trackers, highlighting their strengths and weaknesses on more than 187000 frames. The software implementing the protocol and the evaluation results are made available online and new results can be included, thus facilitating the comparison of trackers.

Plant Species Identification by Bi-channel Deep Convolutional Networks

NASA Astrophysics Data System (ADS)

He, Guiqing; Xia, Zhaoqiang; Zhang, Qiqi; Zhang, Haixi; Fan, Jianping

2018-04-01

Plant species identification achieves much attention recently as it has potential application in the environmental protection and human life. Although deep learning techniques can be directly applied for plant species identification, it still needs to be designed for this specific task to obtain the state-of-art performance. In this paper, a bi-channel deep learning framework is developed for identifying plant species. In the framework, two different sub-networks are fine-tuned over their pretrained models respectively. And then a stacking layer is used to fuse the output of two different sub-networks. We construct a plant dataset of Orchidaceae family for algorithm evaluation. Our experimental results have demonstrated that our bi-channel deep network can achieve very competitive performance on accuracy rates compared to the existing deep learning algorithm.

Performance evaluation of Bragg coherent diffraction imaging

DOE PAGES

Ozturk, Hande; Huang, X.; Yan, H.; ...

2017-10-03

In this study, we present a numerical framework for modeling three-dimensional (3D) diffraction data in Bragg coherent diffraction imaging (Bragg CDI) experiments and evaluating the quality of obtained 3D complex-valued real-space images recovered by reconstruction algorithms under controlled conditions. The approach is used to systematically explore the performance and the detection limit of this phase-retrieval-based microscopy tool. The numerical investigation suggests that the superb performance of Bragg CDI is achieved with an oversampling ratio above 30 and a detection dynamic range above 6 orders. The observed performance degradation subject to the data binning processes is also studied. Furthermore, this numericalmore » tool can be used to optimize experimental parameters and has the potential to significantly improve the throughput of Bragg CDI method.« less

Levels of Autonomy and Autonomous System Performance Assessment for Intelligent Unmanned Systems

DTIC Science & Technology

2014-04-01

LIDAR and camera sensors that is driven entirely by teleoperation would be AL 0. If that same robot used its LIDAR and camera data to generate a...obstacle detection, mapping, path planning 3 CMMAD semi- autonomous counter- mine system (Few 2010) Talon UGV, camera, LIDAR , metal detector...NCAP framework are performed on individual UMS components and do not require mission level evaluations. For example, bench testing of camera, LIDAR

An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit).

PubMed

Yusof, Maryati Mohd; Kuljis, Jasna; Papazafeiropoulou, Anastasia; Stergioulas, Lampros K

2008-06-01

The realization of Health Information Systems (HIS) requires rigorous evaluation that addresses technology, human and organization issues. Our review indicates that current evaluation methods evaluate different aspects of HIS and they can be improved upon. A new evaluation framework, human, organization and technology-fit (HOT-fit) was developed after having conducted a critical appraisal of the findings of existing HIS evaluation studies. HOT-fit builds on previous models of IS evaluation--in particular, the IS Success Model and the IT-Organization Fit Model. This paper introduces the new framework for HIS evaluation that incorporates comprehensive dimensions and measures of HIS and provides a technological, human and organizational fit. Literature review on HIS and IS evaluation studies and pilot testing of developed framework. The framework was used to evaluate a Fundus Imaging System (FIS) of a primary care organization in the UK. The case study was conducted through observation, interview and document analysis. The main findings show that having the right user attitude and skills base together with good leadership, IT-friendly environment and good communication can have positive influence on the system adoption. Comprehensive, specific evaluation factors, dimensions and measures in the new framework (HOT-fit) are applicable in HIS evaluation. The use of such a framework is argued to be useful not only for comprehensive evaluation of the particular FIS system under investigation, but potentially also for any Health Information System in general.

A natural language processing and geospatial clustering framework for harvesting local place names from geotagged housing advertisements

DOE PAGES

Hu, Yingjie; Mao, Huina; Mckenzie, Grant

2018-04-13

We report that local place names are frequently used by residents living in a geographic region. Such place names may not be recorded in existing gazetteers, due to their vernacular nature, relative insignificance to a gazetteer covering a large area (e.g. the entire world), recent establishment (e.g. the name of a newly-opened shopping center) or other reasons. While not always recorded, local place names play important roles in many applications, from supporting public participation in urban planning to locating victims in disaster response. In this paper, we propose a computational framework for harvesting local place names from geotagged housing advertisements.more » We make use of those advertisements posted on local-oriented websites, such as Craigslist, where local place names are often mentioned. The proposed framework consists of two stages: natural language processing (NLP) and geospatial clustering. The NLP stage examines the textual content of housing advertisements and extracts place name candidates. The geospatial stage focuses on the coordinates associated with the extracted place name candidates and performs multiscale geospatial clustering to filter out the non-place names. We evaluate our framework by comparing its performance with those of six baselines. Finally, we also compare our result with four existing gazetteers to demonstrate the not-yet-recorded local place names discovered by our framework.« less

Outcome-based ventilation: A framework for assessing performance, health, and energy impacts to inform office building ventilation decisions.

PubMed

Rackes, A; Ben-David, T; Waring, M S

2018-07-01

This article presents an outcome-based ventilation (OBV) framework, which combines competing ventilation impacts into a monetized loss function ($/occ/h) used to inform ventilation rate decisions. The OBV framework, developed for U.S. offices, considers six outcomes of increasing ventilation: profitable outcomes realized from improvements in occupant work performance and sick leave absenteeism; health outcomes from occupant exposure to outdoor fine particles and ozone; and energy outcomes from electricity and natural gas usage. We used the literature to set low, medium, and high reference values for OBV loss function parameters, and evaluated the framework and outcome-based ventilation rates using a simulated U.S. office stock dataset and a case study in New York City. With parameters for all outcomes set at medium values derived from literature-based central estimates, higher ventilation rates' profitable benefits dominated negative health and energy impacts, and the OBV framework suggested ventilation should be ≥45 L/s/occ, much higher than the baseline ~8.5 L/s/occ rate prescribed by ASHRAE 62.1. Only when combining very low parameter estimates for profitable impacts with very high ones for health and energy impacts were all outcomes on the same order. Even then, however, outcome-based ventilation rates were often twice the baseline rate or more. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

IntelliHealth: A medical decision support application using a novel weighted multi-layer classifier ensemble framework.

PubMed

Bashir, Saba; Qamar, Usman; Khan, Farhan Hassan

2016-02-01

Accuracy plays a vital role in the medical field as it concerns with the life of an individual. Extensive research has been conducted on disease classification and prediction using machine learning techniques. However, there is no agreement on which classifier produces the best results. A specific classifier may be better than others for a specific dataset, but another classifier could perform better for some other dataset. Ensemble of classifiers has been proved to be an effective way to improve classification accuracy. In this research we present an ensemble framework with multi-layer classification using enhanced bagging and optimized weighting. The proposed model called "HM-BagMoov" overcomes the limitations of conventional performance bottlenecks by utilizing an ensemble of seven heterogeneous classifiers. The framework is evaluated on five different heart disease datasets, four breast cancer datasets, two diabetes datasets, two liver disease datasets and one hepatitis dataset obtained from public repositories. The analysis of the results show that ensemble framework achieved the highest accuracy, sensitivity and F-Measure when compared with individual classifiers for all the diseases. In addition to this, the ensemble framework also achieved the highest accuracy when compared with the state of the art techniques. An application named "IntelliHealth" is also developed based on proposed model that may be used by hospitals/doctors for diagnostic advice. Copyright © 2015 Elsevier Inc. All rights reserved.

Implant-supported fixed dental prostheses with CAD/CAM-fabricated porcelain crown and zirconia-based framework.

PubMed

Takaba, Masayuki; Tanaka, Shinpei; Ishiura, Yuichi; Baba, Kazuyoshi

2013-07-01

Recently, fixed dental prostheses (FDPs) with a hybrid structure of CAD/CAM porcelain crowns adhered to a CAD/CAM zirconia framework (PAZ) have been developed. The aim of this report was to describe the clinical application of a newly developed implant-supported FDP fabrication system, which uses PAZ, and to evaluate the outcome after a maximum application period of 36 months. Implants were placed in three patients with edentulous areas in either the maxilla or mandible. After the implant fixtures had successfully integrated with bone, gold-platinum alloy or zirconia custom abutments were first fabricated. Zirconia framework wax-up was performed on the custom abutments, and the CAD/CAM zirconia framework was prepared using the CAD/CAM system. Next, wax-up was performed on working models for porcelain crown fabrication, and CAD/CAM porcelain crowns were fabricated. The CAD/CAM zirconia frameworks and CAD/CAM porcelain crowns were bonded using adhesive resin cement, and the PAZ was cemented. Cementation of the implant superstructure improved the esthetics and masticatory efficiency in all patients. No undesirable outcomes, such as superstructure chipping, stomatognathic dysfunction, or periimplant bone resorption, were observed in any of the patients. PAZ may be a potential solution for ceramic-related clinical problems such as chipping and fracture and associated complicated repair procedures in implant-supported FDPs. © 2012 by the American College of Prosthodontists.

A natural language processing and geospatial clustering framework for harvesting local place names from geotagged housing advertisements

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hu, Yingjie; Mao, Huina; Mckenzie, Grant

We report that local place names are frequently used by residents living in a geographic region. Such place names may not be recorded in existing gazetteers, due to their vernacular nature, relative insignificance to a gazetteer covering a large area (e.g. the entire world), recent establishment (e.g. the name of a newly-opened shopping center) or other reasons. While not always recorded, local place names play important roles in many applications, from supporting public participation in urban planning to locating victims in disaster response. In this paper, we propose a computational framework for harvesting local place names from geotagged housing advertisements.more » We make use of those advertisements posted on local-oriented websites, such as Craigslist, where local place names are often mentioned. The proposed framework consists of two stages: natural language processing (NLP) and geospatial clustering. The NLP stage examines the textual content of housing advertisements and extracts place name candidates. The geospatial stage focuses on the coordinates associated with the extracted place name candidates and performs multiscale geospatial clustering to filter out the non-place names. We evaluate our framework by comparing its performance with those of six baselines. Finally, we also compare our result with four existing gazetteers to demonstrate the not-yet-recorded local place names discovered by our framework.« less

Process evaluation of discharge planning implementation in healthcare using normalization process theory.

PubMed

Nordmark, Sofi; Zingmark, Karin; Lindberg, Inger

2016-04-27

Discharge planning is a care process that aims to secure the transfer of care for the patient at transition from home to the hospital and back home. Information exchange and collaboration between care providers are essential, but deficits are common. A wide range of initiatives to improve the discharge planning process have been developed and implemented for the past three decades. However, there are still high rates of reported medical errors and adverse events related to failures in the discharge planning. Using theoretical frameworks such as Normalization Process Theory (NPT) can support evaluations of complex interventions and processes in healthcare. The aim of this study was to explore the embedding and integration of the DPP from the perspective of registered nurses, district nurses and homecare organizers. The study design was explorative, using the NPT as a framework to explore the embedding and integration of the DPP. Data consisted of written documentation from; workshops with staff, registered adverse events and system failures, web based survey and individual interviews with staff. Using the NPT as a framework to explore the embedding and integration of discharge planning after 10 years in use showed that the staff had reached a consensus of opinion of what the process was (coherence) and how they evaluated the process (reflexive monitoring). However, they had not reached a consensus of opinion of who performed the process (cognitive participation) and how it was performed (collective action). This could be interpreted as the process had not become normalized in daily practice. The result shows necessity to observe the implementation of old practices to better understand the needs of new ones before developing and implementing new practices or supportive tools within healthcare to reach the aim of development and to accomplish sustainable implementation. The NPT offers a generalizable framework for analysis, which can explain and shape the implementation process of old practices, before further development of new practices or supportive tools.

Unified Bayesian Estimator of EEG Reference at Infinity: rREST (Regularized Reference Electrode Standardization Technique).

PubMed

Hu, Shiang; Yao, Dezhong; Valdes-Sosa, Pedro A

2018-01-01

The choice of reference for the electroencephalogram (EEG) is a long-lasting unsolved issue resulting in inconsistent usages and endless debates. Currently, both the average reference (AR) and the reference electrode standardization technique (REST) are two primary, apparently irreconcilable contenders. We propose a theoretical framework to resolve this reference issue by formulating both (a) estimation of potentials at infinity, and (b) determination of the reference, as a unified Bayesian linear inverse problem, which can be solved by maximum a posterior estimation. We find that AR and REST are very particular cases of this unified framework: AR results from biophysically non-informative prior; while REST utilizes the prior based on the EEG generative model. To allow for simultaneous denoising and reference estimation, we develop the regularized versions of AR and REST, named rAR and rREST, respectively. Both depend on a regularization parameter that is the noise to signal variance ratio. Traditional and new estimators are evaluated with this framework, by both simulations and analysis of real resting EEGs. Toward this end, we leverage the MRI and EEG data from 89 subjects which participated in the Cuban Human Brain Mapping Project. Generated artificial EEGs-with a known ground truth, show that relative error in estimating the EEG potentials at infinity is lowest for rREST. It also reveals that realistic volume conductor models improve the performances of REST and rREST. Importantly, for practical applications, it is shown that an average lead field gives the results comparable to the individual lead field. Finally, it is shown that the selection of the regularization parameter with Generalized Cross-Validation (GCV) is close to the "oracle" choice based on the ground truth. When evaluated with the real 89 resting state EEGs, rREST consistently yields the lowest GCV. This study provides a novel perspective to the EEG reference problem by means of a unified inverse solution framework. It may allow additional principled theoretical formulations and numerical evaluation of performance.

An integrative and applicable phylogenetic footprinting framework for cis-regulatory motifs identification in prokaryotic genomes.

PubMed

Liu, Bingqiang; Zhang, Hanyuan; Zhou, Chuan; Li, Guojun; Fennell, Anne; Wang, Guanghui; Kang, Yu; Liu, Qi; Ma, Qin

2016-08-09

Phylogenetic footprinting is an important computational technique for identifying cis-regulatory motifs in orthologous regulatory regions from multiple genomes, as motifs tend to evolve slower than their surrounding non-functional sequences. Its application, however, has several difficulties for optimizing the selection of orthologous data and reducing the false positives in motif prediction. Here we present an integrative phylogenetic footprinting framework for accurate motif predictions in prokaryotic genomes (MP(3)). The framework includes a new orthologous data preparation procedure, an additional promoter scoring and pruning method and an integration of six existing motif finding algorithms as basic motif search engines. Specifically, we collected orthologous genes from available prokaryotic genomes and built the orthologous regulatory regions based on sequence similarity of promoter regions. This procedure made full use of the large-scale genomic data and taxonomy information and filtered out the promoters with limited contribution to produce a high quality orthologous promoter set. The promoter scoring and pruning is implemented through motif voting by a set of complementary predicting tools that mine as many motif candidates as possible and simultaneously eliminate the effect of random noise. We have applied the framework to Escherichia coli k12 genome and evaluated the prediction performance through comparison with seven existing programs. This evaluation was systematically carried out at the nucleotide and binding site level, and the results showed that MP(3) consistently outperformed other popular motif finding tools. We have integrated MP(3) into our motif identification and analysis server DMINDA, allowing users to efficiently identify and analyze motifs in 2,072 completely sequenced prokaryotic genomes. The performance evaluation indicated that MP(3) is effective for predicting regulatory motifs in prokaryotic genomes. Its application may enhance progress in elucidating transcription regulation mechanism, thus provide benefit to the genomic research community and prokaryotic genome researchers in particular.

An official American thoracic society workshop report: developing performance measures from clinical practice guidelines.

PubMed

Kahn, Jeremy M; Gould, Michael K; Krishnan, Jerry A; Wilson, Kevin C; Au, David H; Cooke, Colin R; Douglas, Ivor S; Feemster, Laura C; Mularski, Richard A; Slatore, Christopher G; Wiener, Renda Soylemez

2014-05-01

Many health care performance measures are either not based on high-quality clinical evidence or not tightly linked to patient-centered outcomes, limiting their usefulness in quality improvement. In this report we summarize the proceedings of an American Thoracic Society workshop convened to address this problem by reviewing current approaches to performance measure development and creating a framework for developing high-quality performance measures by basing them directly on recommendations from well-constructed clinical practice guidelines. Workshop participants concluded that ideally performance measures addressing care processes should be linked to clinical practice guidelines that explicitly rate the quality of evidence and the strength of recommendations, such as the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) process. Under this framework, process-based performance measures would only be developed from strong recommendations based on high- or moderate-quality evidence. This approach would help ensure that clinical processes specified in performance measures are both of clear benefit to patients and supported by strong evidence. Although this approach may result in fewer performance measures, it would substantially increase the likelihood that quality-improvement programs based on these measures actually improve patient care.

Do senior management cultures affect performance? Evidence from Italian public healthcare organizations.

PubMed

Prenestini, Anna; Lega, Federico

2013-01-01

Healthcare organizations are often characterized by diffuse power, ambiguous goals, and a plurality of actors. In this complex and pluralistic context, senior healthcare managers are expected to provide strategic direction and lead their organizations toward their goals and performance targets. The present work explores the relationship between senior management team culture and performance by investigating Italian public healthcare organizations in the Tuscany region. Our assessment of senior management culture was accomplished through the use of an established framework and a corresponding tool, the competing values framework, which supports the idea that specific aspects of performance are related to a dominant management culture. Organizational performance was assessed using a wide range of measures collected by a multidimensional performance evaluation system, which was developed in Tuscany to measure the performance of its 12 local health authorities (LHAs) and four teaching hospitals (THs). Usable responses were received from 80 senior managers of 11 different healthcare organizations (two THs and nine LHAs). Our findings show that Tuscan healthcare organizations are characterized by various dominant cultures: developmental, clan, rational, and hierarchical. These variations in dominant culture were associated with performance measures. The implications for management theory, professional practice, and public policy are discussed.

Replica-exchange Wang Landau sampling: pushing the limits of Monte Carlo simulations in materials sciences

DOE Office of Scientific and Technical Information (OSTI.GOV)

Perera, Meewanage Dilina N; Li, Ying Wai; Eisenbach, Markus

We describe the study of thermodynamics of materials using replica-exchange Wang Landau (REWL) sampling, a generic framework for massively parallel implementations of the Wang Landau Monte Carlo method. To evaluate the performance and scalability of the method, we investigate the magnetic phase transition in body-centered cubic (bcc) iron using the classical Heisenberg model parameterized with first principles calculations. We demonstrate that our framework leads to a significant speedup without compromising the accuracy and precision and facilitates the study of much larger systems than is possible with its serial counterpart.

Bureaucracy blights funding in Europe

NASA Astrophysics Data System (ADS)

Banks, Michael

2009-04-01

The process of applying for money from the Framework programme of the European Union (EU) is a "stain" on the EU's reputation and a "radical overhaul" of the administration is needed. That is the view of an independent panel of scientists charged with evaluating the €18bn Sixth Framework programme, which lasted from 2002 to 2006 and funded collaborative research performed within more than one country. The panel says that grants handed out by the programme take, on average, over a year to get approved, with a quarter of proposals taking more than 450 days before being signed.

Integrated modeling for parametric evaluation of smart x-ray optics

NASA Astrophysics Data System (ADS)

Dell'Agostino, S.; Riva, M.; Spiga, D.; Basso, S.; Civitani, Marta

2014-08-01

This work is developed in the framework of AXYOM project, which proposes to study the application of a system of piezoelectric actuators to grazing-incidence X-ray telescope optic prototypes: thin glass or plastic foils, in order to increase their angular resolution. An integrated optomechanical model has been set up to evaluate the performances of X-ray optics under deformation induced by Piezo Actuators. Parametric evaluation has been done looking at different number and position of actuators to optimize the outcome. Different evaluations have also been done over the actuator types, considering Flexible Piezoceramic, Multi Fiber Composites piezo actuators, and PVDF.

Developing an evaluation framework for clinical redesign programs: lessons learnt.

PubMed

Samaranayake, Premaratne; Dadich, Ann; Fitzgerald, Anneke; Zeitz, Kathryn

2016-09-19

Purpose The purpose of this paper is to present lessons learnt through the development of an evaluation framework for a clinical redesign programme - the aim of which was to improve the patient journey through improved discharge practices within an Australian public hospital. Design/methodology/approach The development of the evaluation framework involved three stages - namely, the analysis of secondary data relating to the discharge planning pathway; the analysis of primary data including field-notes and interview transcripts on hospital processes; and the triangulation of these data sets to devise the framework. The evaluation framework ensured that resource use, process management, patient satisfaction, and staff well-being and productivity were each connected with measures, targets, and the aim of clinical redesign programme. Findings The application of business process management and a balanced scorecard enabled a different way of framing the evaluation, ensuring measurable outcomes were connected to inputs and outputs. Lessons learnt include: first, the importance of mixed-methods research to devise the framework and evaluate the redesigned processes; second, the need for appropriate tools and resources to adequately capture change across the different domains of the redesign programme; and third, the value of developing and applying an evaluative framework progressively. Research limitations/implications The evaluation framework is limited by its retrospective application to a clinical process redesign programme. Originality/value This research supports benchmarking with national and international practices in relation to best practice healthcare redesign processes. Additionally, it provides a theoretical contribution on evaluating health services improvement and redesign initiatives.

The Macro- and Micropolitics of Personnel Evaluation: A Framework.

ERIC Educational Resources Information Center

Bridges, Edwin M.; Groves, Barry R.

1999-01-01

Explicates a conceptual framework for analyzing the politics of personnel evaluation in an educational context. Using several elements of the framework, discusses the politics of teacher evaluation in California in relation to the types of personnel evaluation decisions, the actors, their access to these decisions, sources and levels of power, and…

Alternatives Assessment Frameworks: Research Needs for the Informed Substitution of Hazardous Chemicals

PubMed Central

Jacobs, Molly M.; Malloy, Timothy F.; Tickner, Joel A.; Edwards, Sally

2015-01-01

Background Given increasing pressures for hazardous chemical replacement, there is growing interest in alternatives assessment to avoid substituting a toxic chemical with another of equal or greater concern. Alternatives assessment is a process for identifying, comparing, and selecting safer alternatives to chemicals of concern (including those used in materials, processes, or technologies) on the basis of their hazards, performance, and economic viability. Objectives The purposes of this substantive review of alternatives assessment frameworks are to identify consistencies and differences in methods and to outline needs for research and collaboration to advance science policy practice. Methods This review compares methods used in six core components of these frameworks: hazard assessment, exposure characterization, life-cycle impacts, technical feasibility evaluation, economic feasibility assessment, and decision making. Alternatives assessment frameworks published from 1990 to 2014 were included. Results Twenty frameworks were reviewed. The frameworks were consistent in terms of general process steps, but some differences were identified in the end points addressed. Methodological gaps were identified in the exposure characterization, life-cycle assessment, and decision–analysis components. Methods for addressing data gaps remain an issue. Discussion Greater consistency in methods and evaluation metrics is needed but with sufficient flexibility to allow the process to be adapted to different decision contexts. Conclusion Although alternatives assessment is becoming an important science policy field, there is a need for increased cross-disciplinary collaboration to refine methodologies in support of the informed substitution and design of safer chemicals, materials, and products. Case studies can provide concrete lessons to improve alternatives assessment. Citation Jacobs MM, Malloy TF, Tickner JA, Edwards S. 2016. Alternatives assessment frameworks: research needs for the informed substitution of hazardous chemicals. Environ Health Perspect 124:265–280; http://dx.doi.org/10.1289/ehp.1409581 PMID:26339778

A Robust Decision-Making Technique for Water Management under Decadal Scale Climate Variability

NASA Astrophysics Data System (ADS)

Callihan, L.; Zagona, E. A.; Rajagopalan, B.

2013-12-01

Robust decision making, a flexible and dynamic approach to managing water resources in light of deep uncertainties associated with climate variability at inter-annual to decadal time scales, is an analytical framework that detects when a system is in or approaching a vulnerable state. It provides decision makers the opportunity to implement strategies that both address the vulnerabilities and perform well over a wide range of plausible future scenarios. A strategy that performs acceptably over a wide range of possible future states is not likely to be optimal with respect to the actual future state. The degree of success--the ability to avoid vulnerable states and operate efficiently--thus depends on the skill in projecting future states and the ability to select the most efficient strategies to address vulnerabilities. This research develops a robust decision making framework that incorporates new methods of decadal scale projections with selection of efficient strategies. Previous approaches to water resources planning under inter-annual climate variability combining skillful seasonal flow forecasts with climatology for subsequent years are not skillful for medium term (i.e. decadal scale) projections as decision makers are not able to plan adequately to avoid vulnerabilities. We address this need by integrating skillful decadal scale streamflow projections into the robust decision making framework and making the probability distribution of this projection available to the decision making logic. The range of possible future hydrologic scenarios can be defined using a variety of nonparametric methods. Once defined, an ensemble projection of decadal flow scenarios are generated from a wavelet-based spectral K-nearest-neighbor resampling approach using historical and paleo-reconstructed data. This method has been shown to generate skillful medium term projections with a rich variety of natural variability. The current state of the system in combination with the probability distribution of the projected flow ensembles enables the selection of appropriate decision options. This process is repeated for each year of the planning horizon--resulting in system outcomes that can be evaluated on their performance and resiliency. The research utilizes the RiverSMART suite of software modeling and analysis tools developed under the Bureau of Reclamation's WaterSMART initiative and built around the RiverWare modeling environment. A case study is developed for the Gunnison and Upper Colorado River Basins. The ability to mitigate vulnerability using the framework is gauged by system performance indicators that measure the ability of the system to meet various water demands (i.e. agriculture, environmental flows, hydropower etc.). Options and strategies for addressing vulnerabilities include measures such as conservation, reallocation and adjustments to operational policy. In addition to being able to mitigate vulnerabilities, options and strategies are evaluated based on benefits, costs and reliability. Flow ensembles are also simulated to incorporate mean and variance from climate change projections for the planning horizon and the above robust decision-making framework is applied to evaluate its performance under changing climate.

Classification framework for partially observed dynamical systems

NASA Astrophysics Data System (ADS)

Shen, Yuan; Tino, Peter; Tsaneva-Atanasova, Krasimira

2017-04-01

We present a general framework for classifying partially observed dynamical systems based on the idea of learning in the model space. In contrast to the existing approaches using point estimates of model parameters to represent individual data items, we employ posterior distributions over model parameters, thus taking into account in a principled manner the uncertainty due to both the generative (observational and/or dynamic noise) and observation (sampling in time) processes. We evaluate the framework on two test beds: a biological pathway model and a stochastic double-well system. Crucially, we show that the classification performance is not impaired when the model structure used for inferring posterior distributions is much more simple than the observation-generating model structure, provided the reduced-complexity inferential model structure captures the essential characteristics needed for the given classification task.

Improving effectiveness of protection efforts in tiger source sites: Developing a framework for law enforcement monitoring using MIST.

PubMed

Stokes, Emma J

2010-12-01

Wild tigers are in a critical state with an estimated population decline of more than 95% over the past century. Improving the capacity and effectiveness of law enforcement in reducing poaching of tigers is an immediate priority to secure remaining wild populations in source sites. From 2008-2010, standardized patrol-based law enforcement monitoring (LEM) was established under the Tigers Forever Program across 8 key tiger sites in order to improve and evaluate law enforcement interventions. Patrol-based monitoring has the distinct advantage of providing regular and rapid information on illegal activities and ranger performance, although, until recently, it has received relatively little scrutiny from the conservation community. The present paper outlines a framework for implementation of LEM in tiger source sites using MIST, a computerized management information system for ranger-based data collection. The framework addresses many of the technical, practical and institutional challenges involved in the design, implementation, sustainability and evaluation of LEM. Adoption of such a framework for LEM is a cost-effective strategy to improve the efficiency of law enforcement efforts, to increase the motivation of enforcement staff and to promote the accountability of law enforcement agencies in addressing threats to tigers. When combined with independent, systematic and science-based monitoring of tigers and their prey, LEM has great potential for evaluating the effectiveness of protection-based conservation investments. © 2010 ISZS, Blackwell Publishing and IOZ/CAS.

The Use of Techniques of Sensory Evaluation as a Framework for Teaching Experimental Methods.

ERIC Educational Resources Information Center

Bennett, R.; Hamilton, M.

1981-01-01

Describes sensory assessment techniques and conditions for their satisfactory performance, including how they can provide open-ended exercises and advantages as relatively inexpensive and simple methods of teaching experimentation. Experiments described focus on diffusion of salt into potatoes after being cooked in boiled salted water. (Author/JN)

Navigating the Turbulent Waters of School Reform Guided by Complexity Theory

ERIC Educational Resources Information Center

White, David G.; Levin, James A.

2016-01-01

The goal of this research study has been to develop, implement, and evaluate a school reform design experiment at a continuation high school with low-income, low-performing underrepresented minority students. The complexity sciences served as a theoretical framework for this design experiment. Treating an innovative college preparatory program as…

Evaluating and communicating options for harvesting young-growth douglas-fir forests

Treesearch

Dean S. DeBell; Jeffery D. DeBell; Robert O. Curtis; Nancy K. Allison

1997-01-01

A cooperative project, developed by Washington State Department of Natural Resources (DNR) and the Pacific Northwest Research Station (PNW), provides a framework for managers and scientists to (1) obtain experience with a range of silvicultural options; (2) develop information about public response to visual appearance, economic performance, and biological aspects...

A Framework for Implementing Individualized Self-Regulated Learning Strategies in the Classroom

ERIC Educational Resources Information Center

Ness, Bryan M.; Middleton, Michael J.

2012-01-01

Self-regulated learning (SRL) is a conceptual model that can be used to design and implement individualized learning strategies for students with learning disabilities. Students who self-regulate their learning engage in planning, performance, and self-evaluation during academic tasks. This article highlights one approach for teaching SRL skills…

Applying the Multilevel Framework of Discourse Comprehension to Evaluate the Text Characteristics of General Chemistry Textbooks

ERIC Educational Resources Information Center

Pyburn, Daniel T.; Pazicni, Samuel

2014-01-01

Prior chemistry education research has demonstrated a relationship between student reading skill and general chemistry course performance. In addition to student characteristics, however, the qualities of the learning materials with which students interact also impact student learning. For example, low-knowledge students benefit from texts that…

Evaluation Criterion for Quality Assessment of E-Learning Content

ERIC Educational Resources Information Center

Al-Alwani, Abdulkareem

2014-01-01

Research trends related to e-learning systems are oriented towards increasing the efficiency and capacity of the systems, thus they reflect a large variance in performance when considering content conformity and quality standards. The Framework related to standardisation of digital content for e-learning systems is likely to play a significant…

Taiwan HRD Practitioner Competencies: An Application of the ASTD WLP Competency Model

ERIC Educational Resources Information Center

Chen, Angela Shin-yih; Bian, Min-dau; Hom, Yi-ming

2005-01-01

This paper aims to identify Taiwan HRD practitioners' perceived competency levels, and to evaluate the importance of 52 workplace learning and performance (WLP) competencies. The conceptual framework is adapted from the American Society of Training and Development (ASTD) models for WLP conducted by Rothwell, Sanders and Soper (1999). The survey…

Benchmarking Diagnostic Algorithms on an Electrical Power System Testbed

NASA Technical Reports Server (NTRS)

Kurtoglu, Tolga; Narasimhan, Sriram; Poll, Scott; Garcia, David; Wright, Stephanie

2009-01-01

Diagnostic algorithms (DAs) are key to enabling automated health management. These algorithms are designed to detect and isolate anomalies of either a component or the whole system based on observations received from sensors. In recent years a wide range of algorithms, both model-based and data-driven, have been developed to increase autonomy and improve system reliability and affordability. However, the lack of support to perform systematic benchmarking of these algorithms continues to create barriers for effective development and deployment of diagnostic technologies. In this paper, we present our efforts to benchmark a set of DAs on a common platform using a framework that was developed to evaluate and compare various performance metrics for diagnostic technologies. The diagnosed system is an electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The paper presents the fundamentals of the benchmarking framework, the ADAPT system, description of faults and data sets, the metrics used for evaluation, and an in-depth analysis of benchmarking results obtained from testing ten diagnostic algorithms on the ADAPT electrical power system testbed.

Development and Test Evaluations for Ni-DOBDC Metal Organic Framework (MOF) Engineered Forms

DOE Office of Scientific and Technical Information (OSTI.GOV)

Troy G. Garn; Mitchell Greenhalgh

2013-07-01

A joint effort to prepare engineered forms of a Ni-DOBDC metal organic framework (MOF) was completed with contributions from PNNL, SNL and the INL. Two independent methods were used at INL and SNL to prepare engineered form (EF) sorbents from Ni-DOBDC MOF powder developed and prepared at PNNL. Xe and Kr capacity test evaluations were performed at ambient temperature with the cryostat experimental setup at INL. The initial INL EF MOF test results indicated a Xe capacity of 1.6 mmol/kg sorbent and no Kr capacity. A large loss of surface area also occurred during minimal testing rendering the INL EFmore » MOF unusable. Four capacity tests were completed using the SNL EF MOF at ambient temperature and resulted in Xe capacities of 1.4, 4.2, 5.0 and 3.8 mmol/kg sorbent with no Kr capacity observed in any ambient temperature tests. Two additional capacity tests were performed at 240 K to further evaluate SNL EF MOF performance. Xe capacities of 50.7 and 49.3 mmol/kg of sorbent and Kr capacities of 0.77 and 0.69 mmol/kg of sorbent were obtained, respectively. Following the adsorption evaluations, the SNL EF MOF material had lost about 40 % of the initial mass and 40 % of the initial surface area. In general, the Xe capacity results at ambient temperature for the INL and SNL EF Ni-DOBDC MOF’s were lower than 9.8 mmol Xe/kg sorbent test results reported by INL in FY-12 using PNNL’s inital EF supplied material.« less

Multilevel microvibration test for performance predictions of a space optical load platform

NASA Astrophysics Data System (ADS)

Li, Shiqi; Zhang, Heng; Liu, Shiping; Wang, Yue

2018-05-01

This paper presents a framework for the multilevel microvibration analysis and test of a space optical load platform. The test framework is conducted on three levels, including instrument, subsystem, and system level. Disturbance source experimental investigations are performed to evaluate the vibration amplitude and study vibration mechanism. Transfer characteristics of space camera are validated by a subsystem test, which allows the calculation of transfer functions from various disturbance sources to optical performance outputs. In order to identify the influence of the source on the spacecraft performance, a system level microvibration measurement test has been performed on the ground. From the time domain analysis and spectrum analysis of multilevel microvibration tests, we concluded that the disturbance source has a significant effect on its installation position. After transmitted through mechanical links, the residual vibration reduces to a background noise level. In addition, the angular microvibration of the platform jitter is mainly concentrated in the rotation of y-axes. This work is applied to a real practical application involving the high resolution satellite camera system.

Multidimensional evaluation of performance with experimental application of balanced scorecard: a two year experience

PubMed Central

2011-01-01

Background In today's dynamic health-care system, organizations such as hospitals are required to improve their performance for multiple stakeholders and deliver an integrated care that means to work effectively, be innovative and organize efficiently. Achieved goals and levels of quality can be successfully measured by a multidimensional approach like Balanced Scorecard (BSC). The aim of the study was to verify the opportunity to introduce BSC framework to measure performance in St. Anna University Hospital of Ferrara, applying it to the Clinical Laboratory Operative Unit in order to compare over time performance results and achievements of assigned targets. Methods In the first experience with BSC we distinguished four perspectives, according to Kaplan and Norton, identified Key Performance Areas and Key Performance Indicators, set standards and weights for each objective, collected data for all indicators, recognized cause-and-effect relationships in a strategic map. One year later we proceeded with the next data collection and analysed the preservation of framework aptitude to measure Operative Unit performance. In addition, we verified the ability to underline links between strategic actions belonging to different perspectives in producing outcomes changes. Results The BSC was found to be effective for underlining existing problems and identifying opportunities for improvements. The BSC also revealed the specific perspective contribution to overall performance enhancement. After time results comparison was possible depending on the selection of feasible and appropriate key performance indicators, which was occasionally limited by data collection problems. Conclusions The first use of BSC to compare performance at Operative Unit level, in course of time, suggested this framework can be successfully adopted for results measuring and revealing effective health factors, allowing health-care quality improvements. PMID:21586111

Multidimensional evaluation of performance with experimental application of balanced scorecard: a two year experience.

PubMed

Lupi, Silvia; Verzola, Adriano; Carandina, Gianni; Salani, Manuela; Antonioli, Paola; Gregorio, Pasquale

2011-05-17

In today's dynamic health-care system, organizations such as hospitals are required to improve their performance for multiple stakeholders and deliver an integrated care that means to work effectively, be innovative and organize efficiently. Achieved goals and levels of quality can be successfully measured by a multidimensional approach like Balanced Scorecard (BSC). The aim of the study was to verify the opportunity to introduce BSC framework to measure performance in St. Anna University Hospital of Ferrara, applying it to the Clinical Laboratory Operative Unit in order to compare over time performance results and achievements of assigned targets. In the first experience with BSC we distinguished four perspectives, according to Kaplan and Norton, identified Key Performance Areas and Key Performance Indicators, set standards and weights for each objective, collected data for all indicators, recognized cause-and-effect relationships in a strategic map. One year later we proceeded with the next data collection and analysed the preservation of framework aptitude to measure Operative Unit performance. In addition, we verified the ability to underline links between strategic actions belonging to different perspectives in producing outcomes changes. The BSC was found to be effective for underlining existing problems and identifying opportunities for improvements. The BSC also revealed the specific perspective contribution to overall performance enhancement. After time results comparison was possible depending on the selection of feasible and appropriate key performance indicators, which was occasionally limited by data collection problems. The first use of BSC to compare performance at Operative Unit level, in course of time, suggested this framework can be successfully adopted for results measuring and revealing effective health factors, allowing health-care quality improvements.