Application of system thinking concepts in health system strengthening in low-income settings: a proposed conceptual framework for the evaluation of a complex health system intervention: the case of the BHOMA intervention in Zambia.

PubMed

Mutale, Wilbroad; Balabanova, Dina; Chintu, Namwinga; Mwanamwenge, Margaret Tembo; Ayles, Helen

2016-02-01

The current drive to strengthen health systems provides an opportunity to develop new strategies that will enable countries to achieve targets for millennium development goals. In this paper, we present a proposed framework for evaluating a new health system strengthening intervention in Zambia known as Better Health Outcomes through Mentoring and Assessment. We briefly describe the intervention design and focus on the proposed evaluation approach through the lens of systems thinking. In this paper, we present a proposed framework to evaluate a complex health system intervention applying systems thinking concepts. We hope that lessons learnt from this process will help to adapt the intervention and limit unintended negative consequences while promoting positive effects. Emphasis will be paid to interaction and interdependence between health system building blocks, context and the community. © 2014 The Authors. Journal of Evaluation in Clinical Practice published by John Wiley & Sons, Ltd.

Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC Working Group.

PubMed

Buehler, James W; Hopkins, Richard S; Overhage, J Marc; Sosin, Daniel M; Tong, Van

2004-05-07

The threat of terrorism and high-profile disease outbreaks has drawn attention to public health surveillance systems for early detection of outbreaks. State and local health departments are enhancing existing surveillance systems and developing new systems to better detect outbreaks through public health surveillance. However, information is limited about the usefulness of surveillance systems for outbreak detection or the best ways to support this function. This report supplements previous guidelines for evaluating public health surveillance systems. Use of this framework is intended to improve decision-making regarding the implementation of surveillance for outbreak detection. Use of a standardized evaluation methodology, including description of system design and operation, also will enhance the exchange of information regarding methods to improve early detection of outbreaks. The framework directs particular attention to the measurement of timeliness and validity for outbreak detection. The evaluation framework is designed to support assessment and description of all surveillance approaches to early detection, whether through traditional disease reporting, specialized analytic routines for aberration detection, or surveillance using early indicators of disease outbreaks, such as syndromic surveillance.

Proposed evaluation framework for assessing operator performance with multisensor displays

NASA Technical Reports Server (NTRS)

Foyle, David C.

1992-01-01

Despite aggressive work on the development of sensor fusion algorithms and techniques, no formal evaluation procedures have been proposed. Based on existing integration models in the literature, an evaluation framework is developed to assess an operator's ability to use multisensor, or sensor fusion, displays. The proposed evaluation framework for evaluating the operator's ability to use such systems is a normative approach: The operator's performance with the sensor fusion display can be compared to the models' predictions based on the operator's performance when viewing the original sensor displays prior to fusion. This allows for the determination as to when a sensor fusion system leads to: 1) poorer performance than one of the original sensor displays (clearly an undesirable system in which the fused sensor system causes some distortion or interference); 2) better performance than with either single sensor system alone, but at a sub-optimal (compared to the model predictions) level; 3) optimal performance (compared to model predictions); or, 4) super-optimal performance, which may occur if the operator were able to use some highly diagnostic 'emergent features' in the sensor fusion display, which were unavailable in the original sensor displays. An experiment demonstrating the usefulness of the proposed evaluation framework is discussed.

A Cost Comparison Framework for Use in Optimizing Ground Water Pump and Treat Systems

EPA Pesticide Factsheets

This fact sheet has been prepared to provide a framework for conducting cost comparisons to evaluate whether or not to pursue potential opportunities from an optimization evaluation for improving, replacing, or supplementing the P&T system.

Evaluating Health Information Systems Using Ontologies.

PubMed

Eivazzadeh, Shahryar; Anderberg, Peter; Larsson, Tobias C; Fricker, Samuel A; Berglund, Johan

2016-06-16

There are several frameworks that attempt to address the challenges of evaluation of health information systems by offering models, methods, and guidelines about what to evaluate, how to evaluate, and how to report the evaluation results. Model-based evaluation frameworks usually suggest universally applicable evaluation aspects but do not consider case-specific aspects. On the other hand, evaluation frameworks that are case specific, by eliciting user requirements, limit their output to the evaluation aspects suggested by the users in the early phases of system development. In addition, these case-specific approaches extract different sets of evaluation aspects from each case, making it challenging to collectively compare, unify, or aggregate the evaluation of a set of heterogeneous health information systems. The aim of this paper is to find a method capable of suggesting evaluation aspects for a set of one or more health information systems-whether similar or heterogeneous-by organizing, unifying, and aggregating the quality attributes extracted from those systems and from an external evaluation framework. On the basis of the available literature in semantic networks and ontologies, a method (called Unified eValuation using Ontology; UVON) was developed that can organize, unify, and aggregate the quality attributes of several health information systems into a tree-style ontology structure. The method was extended to integrate its generated ontology with the evaluation aspects suggested by model-based evaluation frameworks. An approach was developed to extract evaluation aspects from the ontology that also considers evaluation case practicalities such as the maximum number of evaluation aspects to be measured or their required degree of specificity. The method was applied and tested in Future Internet Social and Technological Alignment Research (FI-STAR), a project of 7 cloud-based eHealth applications that were developed and deployed across European Union countries. The relevance of the evaluation aspects created by the UVON method for the FI-STAR project was validated by the corresponding stakeholders of each case. These evaluation aspects were extracted from a UVON-generated ontology structure that reflects both the internally declared required quality attributes in the 7 eHealth applications of the FI-STAR project and the evaluation aspects recommended by the Model for ASsessment of Telemedicine applications (MAST) evaluation framework. The extracted evaluation aspects were used to create questionnaires (for the corresponding patients and health professionals) to evaluate each individual case and the whole of the FI-STAR project. The UVON method can provide a relevant set of evaluation aspects for a heterogeneous set of health information systems by organizing, unifying, and aggregating the quality attributes through ontological structures. Those quality attributes can be either suggested by evaluation models or elicited from the stakeholders of those systems in the form of system requirements. The method continues to be systematic, context sensitive, and relevant across a heterogeneous set of health information systems.

A Framework for Usability Evaluation in EHR Procurement.

PubMed

Tyllinen, Mari; Kaipio, Johanna; Lääveri, Tinja

2018-01-01

Usability should be considered already by the procuring organizations when selecting future systems. In this paper, we present a framework for usability evaluation during electronic health record (EHR) system procurement. We describe the objectives of the evaluation, the procedure, selected usability attributes and the evaluation methods to measure them. We also present the emphasis usability had in the selection process. We do not elaborate on the details of the results, the application of methods or gathering of data. Instead we focus on the components of the framework to inform and give an example to other similar procurement projects.

A proposed ITS evaluation framework for Texas

DOT National Transportation Integrated Search

1999-03-01

This report presents a proposed intelligent transportation system (ITS) evaluation framework that can be used by the Texas Department of Transportation (TxDOT) in developing evaluation plans for specific ITS applications and deployments in Texas. The...

Models for evaluating the performability of degradable computing systems

NASA Technical Reports Server (NTRS)

Wu, L. T.

1982-01-01

Recent advances in multiprocessor technology established the need for unified methods to evaluate computing systems performance and reliability. In response to this modeling need, a general modeling framework that permits the modeling, analysis and evaluation of degradable computing systems is considered. Within this framework, several user oriented performance variables are identified and shown to be proper generalizations of the traditional notions of system performance and reliability. Furthermore, a time varying version of the model is developed to generalize the traditional fault tree reliability evaluation methods of phased missions.

Evaluating Health Information Systems Using Ontologies

PubMed Central

Anderberg, Peter; Larsson, Tobias C; Fricker, Samuel A; Berglund, Johan

2016-01-01

Background There are several frameworks that attempt to address the challenges of evaluation of health information systems by offering models, methods, and guidelines about what to evaluate, how to evaluate, and how to report the evaluation results. Model-based evaluation frameworks usually suggest universally applicable evaluation aspects but do not consider case-specific aspects. On the other hand, evaluation frameworks that are case specific, by eliciting user requirements, limit their output to the evaluation aspects suggested by the users in the early phases of system development. In addition, these case-specific approaches extract different sets of evaluation aspects from each case, making it challenging to collectively compare, unify, or aggregate the evaluation of a set of heterogeneous health information systems. Objectives The aim of this paper is to find a method capable of suggesting evaluation aspects for a set of one or more health information systems—whether similar or heterogeneous—by organizing, unifying, and aggregating the quality attributes extracted from those systems and from an external evaluation framework. Methods On the basis of the available literature in semantic networks and ontologies, a method (called Unified eValuation using Ontology; UVON) was developed that can organize, unify, and aggregate the quality attributes of several health information systems into a tree-style ontology structure. The method was extended to integrate its generated ontology with the evaluation aspects suggested by model-based evaluation frameworks. An approach was developed to extract evaluation aspects from the ontology that also considers evaluation case practicalities such as the maximum number of evaluation aspects to be measured or their required degree of specificity. The method was applied and tested in Future Internet Social and Technological Alignment Research (FI-STAR), a project of 7 cloud-based eHealth applications that were developed and deployed across European Union countries. Results The relevance of the evaluation aspects created by the UVON method for the FI-STAR project was validated by the corresponding stakeholders of each case. These evaluation aspects were extracted from a UVON-generated ontology structure that reflects both the internally declared required quality attributes in the 7 eHealth applications of the FI-STAR project and the evaluation aspects recommended by the Model for ASsessment of Telemedicine applications (MAST) evaluation framework. The extracted evaluation aspects were used to create questionnaires (for the corresponding patients and health professionals) to evaluate each individual case and the whole of the FI-STAR project. Conclusions The UVON method can provide a relevant set of evaluation aspects for a heterogeneous set of health information systems by organizing, unifying, and aggregating the quality attributes through ontological structures. Those quality attributes can be either suggested by evaluation models or elicited from the stakeholders of those systems in the form of system requirements. The method continues to be systematic, context sensitive, and relevant across a heterogeneous set of health information systems. PMID:27311735

A Patient-Centered Framework for Evaluating Digital Maturity of Health Services: A Systematic Review

PubMed Central

Callahan, Ryan; Darzi, Ara; Mayer, Erik

2016-01-01

Background Digital maturity is the extent to which digital technologies are used as enablers to deliver a high-quality health service. Extensive literature exists about how to assess the components of digital maturity, but it has not been used to design a comprehensive framework for evaluation. Consequently, the measurement systems that do exist are limited to evaluating digital programs within one service or care setting, meaning that digital maturity evaluation is not accounting for the needs of patients across their care pathways. Objective The objective of our study was to identify the best methods and metrics for evaluating digital maturity and to create a novel, evidence-based tool for evaluating digital maturity across patient care pathways. Methods We systematically reviewed the literature to find the best methods and metrics for evaluating digital maturity. We searched the PubMed database for all papers relevant to digital maturity evaluation. Papers were selected if they provided insight into how to appraise digital systems within the health service and if they indicated the factors that constitute or facilitate digital maturity. Papers were analyzed to identify methodology for evaluating digital maturity and indicators of digitally mature systems. We then used the resulting information about methodology to design an evaluation framework. Following that, the indicators of digital maturity were extracted and grouped into increasing levels of maturity and operationalized as metrics within the evaluation framework. Results We identified 28 papers as relevant to evaluating digital maturity, from which we derived 5 themes. The first theme concerned general evaluation methodology for constructing the framework (7 papers). The following 4 themes were the increasing levels of digital maturity: resources and ability (6 papers), usage (7 papers), interoperability (3 papers), and impact (5 papers). The framework includes metrics for each of these levels at each stage of the typical patient care pathway. Conclusions The framework uses a patient-centric model that departs from traditional service-specific measurements and allows for novel insights into how digital programs benefit patients across the health system. Trial Registration N/A PMID:27080852

SERVAL: a new framework for the evaluation of animal health surveillance.

PubMed

Drewe, J A; Hoinville, L J; Cook, A J C; Floyd, T; Gunn, G; Stärk, K D C

2015-02-01

Animal health surveillance programmes may change in response to altering requirements or perceived weaknesses but are seldom subjected to any formal evaluation to ensure that they provide valuable information in an efficient manner. The literature on the evaluation of animal health surveillance systems is sparse, and those that are published may be unstructured and therefore incomplete. To address this gap, we have developed SERVAL, a SuRveillance EVALuation framework, which is novel and aims to be generic and therefore suitable for the evaluation of any animal health surveillance system. The inclusion of socio-economic criteria ensures that economic evaluation is an integral part of this framework. SERVAL was developed with input from a technical workshop of international experts followed by a consultation process involving providers and users of surveillance and evaluation data. It has been applied to a range of case studies encompassing different surveillance and evaluation objectives. Here, we describe the development, structure and application of the SERVAL framework. We discuss users' experiences in applying SERVAL to evaluate animal health surveillance systems in Great Britain. © 2013 Crown Copyright. This article is published with the permission of the Controller of HMSO and the Queen's Printer for Scotland.

Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram

PubMed Central

2015-01-01

Objectives Technological advances using telemedicine and telehealth are growing in healthcare fields, but the evaluation framework for them is inconsistent and limited. This paper suggests a comprehensive evaluation framework for telemedicine system implementation and will support related stakeholders' decision-making by promoting general understanding, and resolving arguments and controversies. Methods This study focused on developing a comprehensive evaluation framework by summarizing themes across the range of evaluation techniques and organized foundational evaluation frameworks generally applicable through studies and cases of diverse telemedicine. Evaluation factors related to aspects of information technology; the evaluation of satisfaction of service providers and consumers, cost, quality, and information security are organized using the fishbone diagram. Results It was not easy to develop a monitoring and evaluation framework for telemedicine since evaluation frameworks for telemedicine are very complex with many potential inputs, activities, outputs, outcomes, and stakeholders. A conceptual framework was developed that incorporates the key dimensions that need to be considered in the evaluation of telehealth implementation for a formal structured approach to the evaluation of a service. The suggested framework consists of six major dimensions and the subsequent branches for each dimension. Conclusions To implement telemedicine and telehealth services, stakeholders should make decisions based on sufficient evidence in quality and safety measured by the comprehensive evaluation framework. Further work would be valuable in applying more comprehensive evaluations to verify and improve the comprehensive framework across a variety of contexts with more factors and participant group dimensions. PMID:26618028

Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram.

PubMed

Chang, Hyejung

2015-10-01

Technological advances using telemedicine and telehealth are growing in healthcare fields, but the evaluation framework for them is inconsistent and limited. This paper suggests a comprehensive evaluation framework for telemedicine system implementation and will support related stakeholders' decision-making by promoting general understanding, and resolving arguments and controversies. This study focused on developing a comprehensive evaluation framework by summarizing themes across the range of evaluation techniques and organized foundational evaluation frameworks generally applicable through studies and cases of diverse telemedicine. Evaluation factors related to aspects of information technology; the evaluation of satisfaction of service providers and consumers, cost, quality, and information security are organized using the fishbone diagram. It was not easy to develop a monitoring and evaluation framework for telemedicine since evaluation frameworks for telemedicine are very complex with many potential inputs, activities, outputs, outcomes, and stakeholders. A conceptual framework was developed that incorporates the key dimensions that need to be considered in the evaluation of telehealth implementation for a formal structured approach to the evaluation of a service. The suggested framework consists of six major dimensions and the subsequent branches for each dimension. To implement telemedicine and telehealth services, stakeholders should make decisions based on sufficient evidence in quality and safety measured by the comprehensive evaluation framework. Further work would be valuable in applying more comprehensive evaluations to verify and improve the comprehensive framework across a variety of contexts with more factors and participant group dimensions.

A framework for outcome-level evaluation of in-service training of health care workers.

PubMed

O'Malley, Gabrielle; Perdue, Thomas; Petracca, Frances

2013-10-01

In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President's Emergency Plan for AIDS Relief's Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the ability of evaluators to determine training outcomes. In addition, a group of user-friendly resources, the Training Evaluation Framework and Tools (TEFT) were created to help evaluators and stakeholders understand and apply the framework. Feedback from pilot users suggests that using the framework and accompanying tools may support outcome evaluation planning. Further assessment will assist in strengthening guidelines and tools for operationalization.

A framework for outcome-level evaluation of in-service training of health care workers

PubMed Central

2013-01-01

Background In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President’s Emergency Plan for AIDS Relief’s Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. Methods A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. Results The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the ability of evaluators to determine training outcomes. In addition, a group of user-friendly resources, the Training Evaluation Framework and Tools (TEFT) were created to help evaluators and stakeholders understand and apply the framework. Conclusions Feedback from pilot users suggests that using the framework and accompanying tools may support outcome evaluation planning. Further assessment will assist in strengthening guidelines and tools for operationalization. PMID:24083635

An Integrated Approach to Disability Policy Development, Implementation, and Evaluation

ERIC Educational Resources Information Center

Shogren, Karrie A.; Luckasson, Ruth; Schalock, Robert L.

2017-01-01

This article provides a framework for an integrated approach to disability policy development, implementation, and evaluation. The article discusses how a framework that combines systems thinking and valued outcomes can be used by coalition partners across ecological systems to implement disability policy, promote the effective use of resources,…

The combination of an Environmental Management System and Life Cycle Assessment at the territorial level

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mazzi, Anna; Toniolo, Sara; Catto, Stella

A framework to include a Life Cycle Assessment in the significance evaluation of the environmental aspects of an Environmental Management System has been studied for some industrial sectors, but there is a literature gap at the territorial level, where the indirect impact assessment is crucial. To overcome this criticality, our research proposes the Life Cycle Assessment as a framework to assess environmental aspects of public administration within an Environmental Management System applied at the territorial level. This research is structured in two parts: the design of a new methodological framework and the pilot application for an Italian municipality. The methodologicalmore » framework designed supports Initial Environmental Analysis at the territorial level thanks to the results derived from the impact assessment phase. The pilot application in an Italian municipality EMAS registered demonstrates the applicability of the framework and its effectiveness in evaluating the environmental impact assessment for direct and indirect aspects. Through the discussion of the results, we underline the growing knowledge derived by this research in terms of the reproducibility and consistency of the criteria to define the significance of the direct and indirect environmental aspects for a local public administration. - Highlights: • The combination between Environmental Management System and LCA is studied. • A methodological framework is elaborated and tested at the territorial level. • Life Cycle Impact Assessment supports the evaluation of aspects significance. • The framework assures consistency of evaluation criteria on the studied territory.« less

The development of an evaluation framework for injury surveillance systems

PubMed Central

Mitchell, Rebecca J; Williamson, Ann M; O'Connor, Rod

2009-01-01

Background Access to good quality information from injury surveillance is essential to develop and monitor injury prevention activities. To determine if information obtained from surveillance is of high quality, the limitations and strengths of a surveillance system are often examined. Guidelines have been developed to assist in evaluating certain types of surveillance systems. However, to date, no standard guidelines have been developed to specifically evaluate an injury surveillance system. The aim of this research is to develop a framework to guide the evaluation of injury surveillance systems. Methods The development of an Evaluation Framework for Injury Surveillance Systems (EFISS) involved a four stage process. First, a literature review was conducted to identify an initial set of characteristics that were recognised as important and/or had been recommended to be assessed in an evaluation of a surveillance system. Second, this set of characteristics was assessed using SMART criteria. Third, those surviving were presented to an expert panel using a two round modified-Delphi study to gain an alternative perspective on characteristic definitions, practicality of assessment, and characteristic importance. Finally, a rating system was created for the EFISS characteristics. Results The resulting EFISS consisted of 18 characteristics that assess three areas of an injury surveillance system – five characteristics assess data quality, nine characteristics assess the system's operation, and four characteristics assess the practical capability of an injury surveillance system. A rating system assesses the performance of each characteristic. Conclusion The development of the EFISS builds upon existing evaluation guidelines for surveillance systems and provides a framework tailored to evaluate an injury surveillance system. Ultimately, information obtained through an evaluation of an injury data collection using the EFISS would be useful for agencies to recommend how a collection could be improved to increase its usefulness for injury surveillance and in the long-term injury prevention. PMID:19627617

Multidisciplinary eHealth Survey Evaluation Methods

ERIC Educational Resources Information Center

Karras, Bryant T.; Tufano, James T.

2006-01-01

This paper describes the development process of an evaluation framework for describing and comparing web survey tools. We believe that this approach will help shape the design, development, deployment, and evaluation of population-based health interventions. A conceptual framework for describing and evaluating web survey systems will enable the…

Towards an evaluation framework for Laboratory Information Systems.

PubMed

Yusof, Maryati M; Arifin, Azila

Laboratory testing and reporting are error-prone and redundant due to repeated, unnecessary requests and delayed or missed reactions to laboratory reports. Occurring errors may negatively affect the patient treatment process and clinical decision making. Evaluation on laboratory testing and Laboratory Information System (LIS) may explain the root cause to improve the testing process and enhance LIS in supporting the process. This paper discusses a new evaluation framework for LIS that encompasses the laboratory testing cycle and the socio-technical part of LIS. Literature review on discourses, dimensions and evaluation methods of laboratory testing and LIS. A critical appraisal of the Total Testing Process (TTP) and the human, organization, technology-fit factors (HOT-fit) evaluation frameworks was undertaken in order to identify error incident, its contributing factors and preventive action pertinent to laboratory testing process and LIS. A new evaluation framework for LIS using a comprehensive and socio-technical approach is outlined. Positive relationship between laboratory and clinical staff resulted in a smooth laboratory testing process, reduced errors and increased process efficiency whilst effective use of LIS streamlined the testing processes. The TTP-LIS framework could serve as an assessment as well as a problem-solving tool for the laboratory testing process and system. Copyright © 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Ltd. All rights reserved.

A Transparent Framework for Evaluating the Effects of DGPV on Distribution System Costs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Horowitz, Kelsey A; Mather, Barry A; Ding, Fei

Assessing the costs and benefits of distributed photovoltaic generators (DGPV) to the power system and electricity consumers is key to determining appropriate policies, tariff designs, and power system upgrades for the modern grid. We advance understanding of this topic by providing a transparent framework, terminology, and data set for evaluating distribution system upgrade costs, line losses, and interconnection costs as a function of DGPV penetration level.

Evaluation Framework and Analyses for Thermal Energy Storage Integrated with Packaged Air Conditioning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kung, F.; Deru, M.; Bonnema, E.

2013-10-01

Few third-party guidance documents or tools are available for evaluating thermal energy storage (TES) integrated with packaged air conditioning (AC), as this type of TES is relatively new compared to TES integrated with chillers or hot water systems. To address this gap, researchers at the National Renewable Energy Laboratory conducted a project to improve the ability of potential technology adopters to evaluate TES technologies. Major project outcomes included: development of an evaluation framework to describe key metrics, methodologies, and issues to consider when assessing the performance of TES systems integrated with packaged AC; application of multiple concepts from the evaluationmore » framework to analyze performance data from four demonstration sites; and production of a new simulation capability that enables modeling of TES integrated with packaged AC in EnergyPlus. This report includes the evaluation framework and analysis results from the project.« less

PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems

PubMed Central

Aqil, Anwer; Lippeveld, Theo; Hozumi, Dairiku

2009-01-01

The utility and effectiveness of routine health information systems (RHIS) in improving health system performance in developing countries has been questioned. This paper argues that the health system needs internal mechanisms to develop performance targets, track progress, and create and manage knowledge for continuous improvement. Based on documented RHIS weaknesses, we have developed the Performance of Routine Information System Management (PRISM) framework, an innovative approach to design, strengthen and evaluate RHIS. The PRISM framework offers a paradigm shift by putting emphasis on RHIS performance and incorporating the organizational, technical and behavioural determinants of performance. By describing causal pathways of these determinants, the PRISM framework encourages and guides the development of interventions for strengthening or reforming RHIS. Furthermore, it conceptualizes and proposes a methodology for measuring the impact of RHIS on health system performance. Ultimately, the PRISM framework, in spite of its challenges and competing paradigms, proposes a new agenda for building and sustaining information systems, for the promotion of an information culture, and for encouraging accountability in health systems. PMID:19304786

'CHEATS': a generic information communication technology (ICT) evaluation framework.

PubMed

Shaw, Nicola T

2002-05-01

This paper describes a generic framework for the evaluation of information communication technologies. This framework, CHEATS, utilises both qualitative and quantitative research methods and has proved appropriate in multiple clinical settings including telepsychiatry, teledermatology and teleeducation. The paper demonstrates how a multidisciplinary approach is essential when evaluating new and emerging technologies, particularly when such systems are implemented in real service as opposed to a research setting.

A framework to observe and evaluate the sustainability of human-natural systems in a complex dynamic context.

PubMed

Satanarachchi, Niranji; Mino, Takashi

2014-01-01

This paper aims to explore the prominent implications of the process of observing complex dynamics linked to sustainability in human-natural systems and to propose a framework for sustainability evaluation by introducing the concept of sustainability boundaries. Arguing that both observing and evaluating sustainability should engage awareness of complex dynamics from the outset, we try to embody this idea in the framework by two complementary methods, namely, the layer view- and dimensional view-based methods, which support the understanding of a reflexive and iterative sustainability process. The framework enables the observation of complex dynamic sustainability contexts, which we call observation metastructures, and enable us to map the contexts to sustainability boundaries.

A proposed analytic framework for determining the impact of an antimicrobial resistance intervention.

PubMed

Grohn, Yrjo T; Carson, Carolee; Lanzas, Cristina; Pullum, Laura; Stanhope, Michael; Volkova, Victoriya

2017-06-01

Antimicrobial use (AMU) is increasingly threatened by antimicrobial resistance (AMR). The FDA is implementing risk mitigation measures promoting prudent AMU in food animals. Their evaluation is crucial: the AMU/AMR relationship is complex; a suitable framework to analyze interventions is unavailable. Systems science analysis, depicting variables and their associations, would help integrate mathematics/epidemiology to evaluate the relationship. This would identify informative data and models to evaluate interventions. This National Institute for Mathematical and Biological Synthesis AMR Working Group's report proposes a system framework to address the methodological gap linking livestock AMU and AMR in foodborne bacteria. It could evaluate how AMU (and interventions) impact AMR. We will evaluate pharmacokinetic/dynamic modeling techniques for projecting AMR selection pressure on enteric bacteria. We study two methods to model phenotypic AMR changes in bacteria in the food supply and evolutionary genotypic analyses determining molecular changes in phenotypic AMR. Systems science analysis integrates the methods, showing how resistance in the food supply is explained by AMU and concurrent factors influencing the whole system. This process is updated with data and techniques to improve prediction and inform improvements for AMU/AMR surveillance. Our proposed framework reflects both the AMR system's complexity, and desire for simple, reliable conclusions.

Overcoming structural constraints to patient utilization of electronic medical records: a critical review and proposal for an evaluation framework.

PubMed

Winkelman, Warren J; Leonard, Kevin J

2004-01-01

There are constraints embedded in medical record structure that limit use by patients in self-directed disease management. Through systematic review of the literature from a critical perspective, four characteristics that either enhance or mitigate the influence of medical record structure on patient utilization of an electronic patient record (EPR) system have been identified: environmental pressures, physician centeredness, collaborative organizational culture, and patient centeredness. An evaluation framework is proposed for use when considering adaptation of existing EPR systems for online patient access. Exemplars of patient-accessible EPR systems from the literature are evaluated utilizing the framework. From this study, it appears that traditional information system research and development methods may not wholly capture many pertinent social issues that arise when expanding access of EPR systems to patients. Critically rooted methods such as action research can directly inform development strategies so that these systems may positively influence health outcomes.

Beyond Effectiveness: A Pragmatic Evaluation Framework for Learning and Continuous Quality Improvement of e-Learning Interventions in Healthcare.

PubMed

Dafalla, Tarig Dafalla Mohamed; Kushniruk, Andre W; Borycki, Elizabeth M

2015-01-01

A pragmatic evaluation framework for evaluating the usability and usefulness of an e-learning intervention for a patient clinical information scheduling system is presented in this paper. The framework was conceptualized based on two different but related concepts (usability and usefulness) and selection of appropriate and valid methods of data collection and analysis that included: (1) Low-Cost Rapid Usability Engineering (LCRUE), (2) Cognitive Task Analysis (CTA), (3) Heuristic Evaluation (HE) criteria for web-based learning, and (4) Software Usability Measurement Inventory (SUMI). The results of the analysis showed some areas where usability that were related to General Interface Usability (GIU), instructional design and content was problematic; some of which might account for the poorly rated aspects of usability when subjectively measured. This paper shows that using a pragmatic framework can be a useful way, not only for measuring the usability and usefulness, but also for providing a practical objective evidences for learning and continuous quality improvement of e-learning systems. The findings should be of interest to educators, developers, designers, researchers, and usability practitioners involved in the development of e-learning systems in healthcare. This framework could be an appropriate method for assessing the usability, usefulness and safety of health information systems both in the laboratory and in the clinical context.

Towards a framework for selection of supervisory control for commercial buildings: HVAC system energy efficiency

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ramachandran, Thiagarajan; Kundu, Soumya; Chen, Yan

This paper develops and utilizes an optimization based framework to investigate the maximal energy efficiency potentially attainable by HVAC system operation in a non-predictive context. Performance is evaluated relative to the existing state of the art set point reset strategies. The expected efficiency increase driven by operation constraints relaxations is evaluated.

Towards a framework for selection of supervisory control for commercial buildings: HVAC system energy efficiency

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ramachandran, Thiagarajan; Kundu, Soumya; Chen, Yan

This paper develops and utilizes an optimization based framework to investigate the maximal energy efficiency potentially attainable by HVAC system operation in a non-predictive context. Performance is evaluated relative to the existing state of the art set-point reset strategies. The expected efficiency increase driven by operation constraints relaxations is evaluated.

Disability Policy Evaluation: Combining Logic Models and Systems Thinking.

PubMed

Claes, Claudia; Ferket, Neelke; Vandevelde, Stijn; Verlet, Dries; De Maeyer, Jessica

2017-07-01

Policy evaluation focuses on the assessment of policy-related personal, family, and societal changes or benefits that follow as a result of the interventions, services, and supports provided to those persons to whom the policy is directed. This article describes a systematic approach to policy evaluation based on an evaluation framework and an evaluation process that combine the use of logic models and systems thinking. The article also includes an example of how the framework and process have recently been used in policy development and evaluation in Flanders (Belgium), as well as four policy evaluation guidelines based on relevant published literature.

The training for health equity network evaluation framework: a pilot study at five health professional schools.

PubMed

Ross, Simone J; Preston, Robyn; Lindemann, Iris C; Matte, Marie C; Samson, Rex; Tandinco, Filedito D; Larkins, Sarah L; Palsdottir, Bjorg; Neusy, Andre-Jacques

2014-01-01

The Training for Health Equity Network (THEnet), a group of diverse health professional schools aspiring toward social accountability, developed and pilot tested a comprehensive evaluation framework to assess progress toward socially accountable health professions education. The evaluation framework provides criteria for schools to assess their level of social accountability within their organization and planning; education, research and service delivery; and the direct and indirect impacts of the school and its graduates, on the community and health system. This paper describes the pilot implementation of testing the evaluation framework across five THEnet schools, and examines whether the evaluation framework was practical and feasible across contexts for the purposes of critical reflection and continuous improvement in terms of progress towards social accountability. In this pilot study, schools utilized the evaluation framework using a mixed method approach of data collection comprising of workshops, qualitative interviews and focus group discussions, document review and collation and analysis of existing quantitative data. The evaluation framework allowed each school to contextually gather evidence on how it was meeting the aspirational goals of social accountability across a range of school activities, and to identify strengths and areas for improvement and development. The evaluation framework pilot study demonstrated how social accountability can be assessed through a critically reflective and comprehensive process. As social accountability focuses on the relationship between health professions schools and health system and health population outcomes, each school was able to demonstrate to students, health professionals, governments, accrediting bodies, communities and other stakeholders how current and future health care needs of populations are addressed in terms of education, research, and service learning.

From expert-derived user needs to user-perceived ease of use and usefulness: a two-phase mixed-methods evaluation framework.

PubMed

Boland, Mary Regina; Rusanov, Alexander; So, Yat; Lopez-Jimenez, Carlos; Busacca, Linda; Steinman, Richard C; Bakken, Suzanne; Bigger, J Thomas; Weng, Chunhua

2014-12-01

Underspecified user needs and frequent lack of a gold standard reference are typical barriers to technology evaluation. To address this problem, this paper presents a two-phase evaluation framework involving usability experts (phase 1) and end-users (phase 2). In phase 1, a cross-system functionality alignment between expert-derived user needs and system functions was performed to inform the choice of "the best available" comparison system to enable a cognitive walkthrough in phase 1 and a comparative effectiveness evaluation in phase 2. During phase 2, five quantitative and qualitative evaluation methods are mixed to assess usability: time-motion analysis, software log, questionnaires - System Usability Scale and the Unified Theory of Acceptance of Use of Technology, think-aloud protocols, and unstructured interviews. Each method contributes data for a unique measure (e.g., time motion analysis contributes task-completion-time; software log contributes action transition frequency). The measures are triangulated to yield complementary insights regarding user-perceived ease-of-use, functionality integration, anxiety during use, and workflow impact. To illustrate its use, we applied this framework in a formative evaluation of a software called Integrated Model for Patient Care and Clinical Trials (IMPACT). We conclude that this mixed-methods evaluation framework enables an integrated assessment of user needs satisfaction and user-perceived usefulness and usability of a novel design. This evaluation framework effectively bridges the gap between co-evolving user needs and technology designs during iterative prototyping and is particularly useful when it is difficult for users to articulate their needs for technology support due to the lack of a baseline. Copyright © 2013 Elsevier Inc. All rights reserved.

Analyzing Electronic Question/Answer Services: Framework and Evaluations of Selected Services.

ERIC Educational Resources Information Center

White, Marilyn Domas, Ed.

This report develops an analytical framework based on systems analysis for evaluating electronic question/answer or AskA services operated by a wide range of types of organizations, including libraries. Version 1.0 of this framework was applied in June 1999 to a selective sample of 11 electronic question/answer services, which cover a range of…

Understanding Illinois Principals' Concerns Implementing Charlotte Danielson's Framework for Teaching as a Model for Evaluation

ERIC Educational Resources Information Center

Mckenna, George Tucker

2017-01-01

The purpose of this study is to determine the levels of concern of Illinois principals regarding the adoption of an evaluation system modeled after Charlotte Danielson's Framework for Teaching. Principal demographics and involvement in the use of and professional development surrounding Charlotte Danielson's Framework for Teaching were studied for…

Evolution of a multilevel framework for health program evaluation.

PubMed

Masso, Malcolm; Quinsey, Karen; Fildes, Dave

2017-07-01

A well-conceived evaluation framework increases understanding of a program's goals and objectives, facilitates the identification of outcomes and can be used as a planning tool during program development. Herein we describe the origins and development of an evaluation framework that recognises that implementation is influenced by the setting in which it takes place, the individuals involved and the processes by which implementation is accomplished. The framework includes an evaluation hierarchy that focuses on outcomes for consumers, providers and the care delivery system, and is structured according to six domains: program delivery, impact, sustainability, capacity building, generalisability and dissemination. These components of the evaluation framework fit into a matrix structure, and cells within the matrix are supported by relevant evaluation tools. The development of the framework has been influenced by feedback from various stakeholders, existing knowledge of the evaluators and the literature on health promotion and implementation science. Over the years, the framework has matured and is generic enough to be useful in a wide variety of circumstances, yet specific enough to focus data collection, data analysis and the presentation of findings.

Efficient evaluation of wireless real-time control networks.

PubMed

Horvath, Peter; Yampolskiy, Mark; Koutsoukos, Xenofon

2015-02-11

In this paper, we present a system simulation framework for the design and performance evaluation of complex wireless cyber-physical systems. We describe the simulator architecture and the specific developments that are required to simulate cyber-physical systems relying on multi-channel, multihop mesh networks. We introduce realistic and efficient physical layer models and a system simulation methodology, which provides statistically significant performance evaluation results with low computational complexity. The capabilities of the proposed framework are illustrated in the example of WirelessHART, a centralized, real-time, multi-hop mesh network designed for industrial control and monitor applications.

A Systematic Approach for Quantitative Analysis of Multidisciplinary Design Optimization Framework

NASA Astrophysics Data System (ADS)

Kim, Sangho; Park, Jungkeun; Lee, Jeong-Oog; Lee, Jae-Woo

An efficient Multidisciplinary Design and Optimization (MDO) framework for an aerospace engineering system should use and integrate distributed resources such as various analysis codes, optimization codes, Computer Aided Design (CAD) tools, Data Base Management Systems (DBMS), etc. in a heterogeneous environment, and need to provide user-friendly graphical user interfaces. In this paper, we propose a systematic approach for determining a reference MDO framework and for evaluating MDO frameworks. The proposed approach incorporates two well-known methods, Analytic Hierarchy Process (AHP) and Quality Function Deployment (QFD), in order to provide a quantitative analysis of the qualitative criteria of MDO frameworks. Identification and hierarchy of the framework requirements and the corresponding solutions for the reference MDO frameworks, the general one and the aircraft oriented one were carefully investigated. The reference frameworks were also quantitatively identified using AHP and QFD. An assessment of three in-house frameworks was then performed. The results produced clear and useful guidelines for improvement of the in-house MDO frameworks and showed the feasibility of the proposed approach for evaluating an MDO framework without a human interference.

Reusable rocket engine intelligent control system framework design, phase 2

NASA Technical Reports Server (NTRS)

Nemeth, ED; Anderson, Ron; Ols, Joe; Olsasky, Mark

1991-01-01

Elements of an advanced functional framework for reusable rocket engine propulsion system control are presented for the Space Shuttle Main Engine (SSME) demonstration case. Functional elements of the baseline functional framework are defined in detail. The SSME failure modes are evaluated and specific failure modes identified for inclusion in the advanced functional framework diagnostic system. Active control of the SSME start transient is investigated, leading to the identification of a promising approach to mitigating start transient excursions. Key elements of the functional framework are simulated and demonstration cases are provided. Finally, the advanced function framework for control of reusable rocket engines is presented.

Software Engineering Support of the Third Round of Scientific Grand Challenge Investigations: Earth System Modeling Software Framework Survey

NASA Technical Reports Server (NTRS)

Talbot, Bryan; Zhou, Shu-Jia; Higgins, Glenn; Zukor, Dorothy (Technical Monitor)

2002-01-01

One of the most significant challenges in large-scale climate modeling, as well as in high-performance computing in other scientific fields, is that of effectively integrating many software models from multiple contributors. A software framework facilitates the integration task, both in the development and runtime stages of the simulation. Effective software frameworks reduce the programming burden for the investigators, freeing them to focus more on the science and less on the parallel communication implementation. while maintaining high performance across numerous supercomputer and workstation architectures. This document surveys numerous software frameworks for potential use in Earth science modeling. Several frameworks are evaluated in depth, including Parallel Object-Oriented Methods and Applications (POOMA), Cactus (from (he relativistic physics community), Overture, Goddard Earth Modeling System (GEMS), the National Center for Atmospheric Research Flux Coupler, and UCLA/UCB Distributed Data Broker (DDB). Frameworks evaluated in less detail include ROOT, Parallel Application Workspace (PAWS), and Advanced Large-Scale Integrated Computational Environment (ALICE). A host of other frameworks and related tools are referenced in this context. The frameworks are evaluated individually and also compared with each other.

U.S. Army Human Capital Enterprise (HCE) ARFORGEN Data Management, Correlation, Integration and Synchronization Analysis

DTIC Science & Technology

2011-08-15

system must, at a minimum, include  design and configuration  framework  supporting:  Part 1.   Net  Ready. The system must support  net ‐ centric operations...Analyze, evaluate and incorporate relevant DoD Architecture Framework . 5) Document standards for each task / condition combination. 6) Prepare final FAA...task  Analyze, evaluate and incorporate relevant Army Architecture Framework  Document standards for each task/condition combination forming

Framework for Evaluating the Impact of Advanced Practice Nursing Roles.

PubMed

Bryant-Lukosius, Denise; Spichiger, Elisabeth; Martin, Jacqueline; Stoll, Hansruedi; Kellerhals, Sabine Degen; Fliedner, Monica; Grossmann, Florian; Henry, Morag; Herrmann, Luzia; Koller, Antje; Schwendimann, René; Ulrich, Anja; Weibel, Lukas; Callens, Betty; De Geest, Sabina

2016-03-01

To address the gap in evidence-based information required to support the development of advanced practice nursing (APN) roles in Switzerland, stakeholders identified the need for guidance to generate strategic evaluation data. This article describes an evaluation framework developed to inform decisions about the effective utilization of APN roles across the country. A participatory approach was used by an international group of stakeholders. Published literature and an evidenced-based framework for introducing APN roles were analyzed and applied to define the purpose, target audiences, and essential elements of the evaluation framework. Through subsequent meetings and review by an expert panel, the framework was developed and refined. A framework to evaluate different types of APN roles as they evolve to meet dynamic population health, practice setting, and health system needs was created. It includes a matrix of key concepts to guide evaluations across three stages of APN role development: introduction, implementation, and long-term sustainability. For each stage, evaluation objectives and questions examining APN role structures, processes, and outcomes from different perspectives (e.g., patients, providers, managers, policy-makers) were identified. A practical, robust framework based on well-established evaluation concepts and current understanding of APN roles can be used to conduct systematic evaluations. The evaluation framework is sufficiently generic to allow application in developed countries globally, both for evaluation as well as research purposes. © 2016 Sigma Theta Tau International.

Real-time long term measurement using integrated framework for ubiquitous smart monitoring

NASA Astrophysics Data System (ADS)

Heo, Gwanghee; Lee, Giu; Lee, Woosang; Jeon, Joonryong; Kim, Pil-Joong

2007-04-01

Ubiquitous monitoring combining internet technologies and wireless communication is one of the most promising technologies of infrastructure health monitoring against the natural of man-made hazards. In this paper, an integrated framework of the ubiquitous monitoring is developed for real-time long term measurement in internet environment. This framework develops a wireless sensor system based on Bluetooth technology and sends measured acceleration data to the host computer through TCP/IP protocol. And it is also designed to respond to the request of web user on real time basis. In order to verify this system, real time monitoring tests are carried out on a prototype self-anchored suspension bridge. Also, wireless measurement system is analyzed to estimate its sensing capacity and evaluate its performance for monitoring purpose. Based on the evaluation, this paper proposes the effective strategies for integrated framework in order to detect structural deficiencies and to design an early warning system.

Overcoming Structural Constraints to Patient Utilization of Electronic Medical Records: A Critical Review and Proposal for an Evaluation Framework

PubMed Central

Winkelman, Warren J.; Leonard, Kevin J.

2004-01-01

There are constraints embedded in medical record structure that limit use by patients in self-directed disease management. Through systematic review of the literature from a critical perspective, four characteristics that either enhance or mitigate the influence of medical record structure on patient utilization of an electronic patient record (EPR) system have been identified: environmental pressures, physician centeredness, collaborative organizational culture, and patient centeredness. An evaluation framework is proposed for use when considering adaptation of existing EPR systems for online patient access. Exemplars of patient-accessible EPR systems from the literature are evaluated utilizing the framework. From this study, it appears that traditional information system research and development methods may not wholly capture many pertinent social issues that arise when expanding access of EPR systems to patients. Critically rooted methods such as action research can directly inform development strategies so that these systems may positively influence health outcomes. PMID:14633932

System Theoretic Frameworks for Mitigating Risk Complexity in the Nuclear Fuel Cycle

DOE Office of Scientific and Technical Information (OSTI.GOV)

Williams, Adam David; Mohagheghi, Amir H.; Cohn, Brian

In response to the expansion of nuclear fuel cycle (NFC) activities -- and the associated suite of risks -- around the world, this project evaluated systems-based solutions for managing such risk complexity in multimodal and multi-jurisdictional international spent nuclear fuel (SNF) transportation. By better understanding systemic risks in SNF transportation, developing SNF transportation risk assessment frameworks, and evaluating these systems-based risk assessment frameworks, this research illustrated interdependency between safety, security, and safeguards risks is inherent in NFC activities and can go unidentified when each "S" is independently evaluated. Two novel system-theoretic analysis techniques -- dynamic probabilistic risk assessment (DPRA) andmore » system-theoretic process analysis (STPA) -- provide integrated "3S" analysis to address these interdependencies and the research results suggest a need -- and provide a way -- to reprioritize United States engagement efforts to reduce global nuclear risks. Lastly, this research identifies areas where Sandia National Laboratories can spearhead technical advances to reduce global nuclear dangers.« less

Beyond the plot: technology extrapolation domains for scaling out agronomic science

NASA Astrophysics Data System (ADS)

Rattalino Edreira, Juan I.; Cassman, Kenneth G.; Hochman, Zvi; van Ittersum, Martin K.; van Bussel, Lenny; Claessens, Lieven; Grassini, Patricio

2018-05-01

Ensuring an adequate food supply in systems that protect environmental quality and conserve natural resources requires productive and resource-efficient cropping systems on existing farmland. Meeting this challenge will be difficult without a robust spatial framework that facilitates rapid evaluation and scaling-out of currently available and emerging technologies. Here we develop a global spatial framework to delineate ‘technology extrapolation domains’ based on key climate and soil factors that govern crop yields and yield stability in rainfed crop production. The proposed framework adequately represents the spatial pattern of crop yields and stability when evaluated over the data-rich US Corn Belt. It also facilitates evaluation of cropping system performance across continents, which can improve efficiency of agricultural research that seeks to intensify production on existing farmland. Populating this biophysical spatial framework with appropriate socio-economic attributes provides the potential to amplify the return on investments in agricultural research and development by improving the effectiveness of research prioritization and impact assessment.

Towards an evaluation framework for information quality management (IQM) practices for health information systems--evaluation criteria for effective IQM practices.

PubMed

Mohammed, Siti Asma; Yusof, Maryati Mohd

2013-04-01

Poor information quality (IQ) must be understood as a business problem rather than systems problem. In health care organization, what is required is an effective quality management that continuously manages and reviews the factors influencing IQ in health information systems (HIS) so as to achieve the desired outcomes. Hence, in order to understand the issues of information quality management (IQM) practices in health care organizations, a more holistic evaluation study should be undertaken to investigate the IQM practices in health care organizations. It is the aim of this paper to identify the significant evaluation criteria that influence the production of good IQ in HIS. Six selected frameworks and best practices both from health informatics and information systems literature have been reviewed to identify the evaluation criteria from the perspective of human, organizational and technological factors. From the review, it was found that human and organization factors are of greater significance in influencing HIS IQ. Our review depicts that there is still shortage in finding a comprehensive IQM evaluation framework. Thus, the criteria from the frameworks reviewed can be used in combination for more comprehensive evaluation criteria. Integrated IQM evaluation criteria for HIS are then proposed in this study. Poor IQ is the result of complex interdependency within sociotechnical factors in health care organization and lack of formal and structured IQM practices. Thus, a feedback mechanism such as evaluation is needed to understand the issues in depth in the future. © 2012 Blackwell Publishing Ltd.

Health Information Technology Evaluation Framework (HITREF) Comprehensiveness as Assessed in Electronic Point-of-Care Documentation Systems Evaluations.

PubMed

Sockolow, Paulina S; Bowles, Kathryn H; Rogers, Michelle

2015-01-01

We assessed the Health Information Technology (HIT) Reference-based Evaluation Framework (HITREF) comprehensiveness in two HIT evaluations in settings different from that in which the HITREF was developed. Clinician satisfaction themes that emerged from clinician interviews in the home care and the hospital studies were compared to the framework components. Across both studies, respondents commented on 12 of the 20 HITREF components within 5 of the 6 HITREF concepts. No new components emerged that were missing from the HITREF providing evidence that the HITREF is a comprehensive framework. HITREF use in a range of HIT evaluations by researchers new to the HITREF demonstrates that it can be used as intended. Therefore, we continue to recommend the HITREF as a comprehensive, research-based HIT evaluation framework to increase the capacity of informatics evaluators' use of best practice and evidence-based practice to support the credibility of their findings for fulfilling the purpose of program evaluation.

A framework for human-hydrologic system model development integrating hydrology and water management: application to the Cutzamala water system in Mexico

NASA Astrophysics Data System (ADS)

Wi, S.; Freeman, S.; Brown, C.

2017-12-01

This study presents a general approach to developing computational models of human-hydrologic systems where human modification of hydrologic surface processes are significant or dominant. A river basin system is represented by a network of human-hydrologic response units (HHRUs) identified based on locations where river regulations happen (e.g., reservoir operation and diversions). Natural and human processes in HHRUs are simulated in a holistic framework that integrates component models representing rainfall-runoff, river routing, reservoir operation, flow diversion and water use processes. We illustrate the approach in a case study of the Cutzamala water system (CWS) in Mexico, a complex inter-basin water transfer system supplying the Mexico City Metropolitan Area (MCMA). The human-hydrologic system model for CWS (CUTZSIM) is evaluated in terms of streamflow and reservoir storages measured across the CWS and to water supplied for MCMA. The CUTZSIM improves the representation of hydrology and river-operation interaction and, in so doing, advances evaluation of system-wide water management consequences under altered climatic and demand regimes. The integrated modeling framework enables evaluation and simulation of model errors throughout the river basin, including errors in representation of the human component processes. Heretofore, model error evaluation, predictive error intervals and the resultant improved understanding have been limited to hydrologic processes. The general framework represents an initial step towards fuller understanding and prediction of the many and varied processes that determine the hydrologic fluxes and state variables in real river basins.

First Destination Transportation Study

DTIC Science & Technology

1978-03-03

avoidance potentials on a daily procurement basis. A "total" sys- tematic cost evaluation program, including specific de - cision rules needed on an...implementation priority :x needed. The rec- ommended priority is as follows: [ (1) Develop improved system management concepts that will "form the framework ...priority as it provides the overall framework for better system management. Cost evaluation and CCSS Enhancement Programs deserve next consideration, as

Quality Attribute Techniques Framework

NASA Astrophysics Data System (ADS)

Chiam, Yin Kia; Zhu, Liming; Staples, Mark

The quality of software is achieved during its development. Development teams use various techniques to investigate, evaluate and control potential quality problems in their systems. These “Quality Attribute Techniques” target specific product qualities such as safety or security. This paper proposes a framework to capture important characteristics of these techniques. The framework is intended to support process tailoring, by facilitating the selection of techniques for inclusion into process models that target specific product qualities. We use risk management as a theory to accommodate techniques for many product qualities and lifecycle phases. Safety techniques have motivated the framework, and safety and performance techniques have been used to evaluate the framework. The evaluation demonstrates the ability of quality risk management to cover the development lifecycle and to accommodate two different product qualities. We identify advantages and limitations of the framework, and discuss future research on the framework.

THE EPA MULTIMEDIA INTEGRATED MODELING SYSTEM SOFTWARE SUITE

EPA Science Inventory

The U.S. EPA is developing a Multimedia Integrated Modeling System (MIMS) framework that will provide a software infrastructure or environment to support constructing, composing, executing, and evaluating complex modeling studies. The framework will include (1) common software ...

New model framework and structure and the commonality evaluation model. [concerning unmanned spacecraft projects

NASA Technical Reports Server (NTRS)

1977-01-01

The development of a framework and structure for shuttle era unmanned spacecraft projects and the development of a commonality evaluation model is documented. The methodology developed for model utilization in performing cost trades and comparative evaluations for commonality studies is discussed. The model framework consists of categories of activities associated with the spacecraft system's development process. The model structure describes the physical elements to be treated as separate identifiable entities. Cost estimating relationships for subsystem and program-level components were calculated.

The participatory design of a performance oriented monitoring and evaluation system in an international development environment.

PubMed

Guerra-López, Ingrid; Hicks, Karen

2015-02-01

This article illustrates the application of the impact monitoring and evaluation process for the design and development of a performance monitoring and evaluation framework in the context of human and institutional capacity development. This participative process facilitated stakeholder ownership in several areas including the design, development, and use of a new monitoring and evaluation system, as well their targeted results and accomplishments through the use of timely performance data gathered through ongoing monitoring and evaluation. The process produced a performance indicator map, a comprehensive monitoring and evaluation framework, and data collection templates to promote the development, implementation, and sustainability of the monitoring and evaluation system of a farmer's trade union in an African country. Copyright © 2014 Elsevier Ltd. All rights reserved.

Electronic immunization data collection systems: application of an evaluation framework.

PubMed

Heidebrecht, Christine L; Kwong, Jeffrey C; Finkelstein, Michael; Quan, Sherman D; Pereira, Jennifer A; Quach, Susan; Deeks, Shelley L

2014-01-14

Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units. The Centers for Disease Control and Prevention's Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations. Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers' immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform. Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.

Evaluating a federated medical search engine: tailoring the methodology and reporting the evaluation outcomes.

PubMed

Saparova, D; Belden, J; Williams, J; Richardson, B; Schuster, K

2014-01-01

Federated medical search engines are health information systems that provide a single access point to different types of information. Their efficiency as clinical decision support tools has been demonstrated through numerous evaluations. Despite their rigor, very few of these studies report holistic evaluations of medical search engines and even fewer base their evaluations on existing evaluation frameworks. To evaluate a federated medical search engine, MedSocket, for its potential net benefits in an established clinical setting. This study applied the Human, Organization, and Technology (HOT-fit) evaluation framework in order to evaluate MedSocket. The hierarchical structure of the HOT-factors allowed for identification of a combination of efficiency metrics. Human fit was evaluated through user satisfaction and patterns of system use; technology fit was evaluated through the measurements of time-on-task and the accuracy of the found answers; and organization fit was evaluated from the perspective of system fit to the existing organizational structure. Evaluations produced mixed results and suggested several opportunities for system improvement. On average, participants were satisfied with MedSocket searches and confident in the accuracy of retrieved answers. However, MedSocket did not meet participants' expectations in terms of download speed, access to information, and relevance of the search results. These mixed results made it necessary to conclude that in the case of MedSocket, technology fit had a significant influence on the human and organization fit. Hence, improving technological capabilities of the system is critical before its net benefits can become noticeable. The HOT-fit evaluation framework was instrumental in tailoring the methodology for conducting a comprehensive evaluation of the search engine. Such multidimensional evaluation of the search engine resulted in recommendations for system improvement.

A framework for the evaluation of the benefits of intelligent transportation systems

DOT National Transportation Integrated Search

2000-11-30

This report discusses methods to assess the benefits of Intelligent Transportation Systems. It provides a comprehensive review of methods currently available in the United States. It develops a framework for benefits assessment using benefit trees an...

Applying a Framework to Evaluate Assignment Marking Software: A Case Study on Lightwork

ERIC Educational Resources Information Center

Heinrich, Eva; Milne, John

2012-01-01

This article presents the findings of a qualitative evaluation on the effect of a specialised software tool on the efficiency and quality of assignment marking. The software, Lightwork, combines with the Moodle learning management system and provides support through marking rubrics and marker allocations. To enable the evaluation a framework has…

A novel framework for analyzing conservation impacts: evaluation, theory, and marine protected areas.

PubMed

Mascia, Michael B; Fox, Helen E; Glew, Louise; Ahmadia, Gabby N; Agrawal, Arun; Barnes, Megan; Basurto, Xavier; Craigie, Ian; Darling, Emily; Geldmann, Jonas; Gill, David; Holst Rice, Susie; Jensen, Olaf P; Lester, Sarah E; McConney, Patrick; Mumby, Peter J; Nenadovic, Mateja; Parks, John E; Pomeroy, Robert S; White, Alan T

2017-07-01

Environmental conservation initiatives, including marine protected areas (MPAs), have proliferated in recent decades. Designed to conserve marine biodiversity, many MPAs also seek to foster sustainable development. As is the case for many other environmental policies and programs, the impacts of MPAs are poorly understood. Social-ecological systems, impact evaluation, and common-pool resource governance are three complementary scientific frameworks for documenting and explaining the ecological and social impacts of conservation interventions. We review key components of these three frameworks and their implications for the study of conservation policy, program, and project outcomes. Using MPAs as an illustrative example, we then draw upon these three frameworks to describe an integrated approach for rigorous empirical documentation and causal explanation of conservation impacts. This integrated three-framework approach for impact evaluation of governance in social-ecological systems (3FIGS) accounts for alternative explanations, builds upon and advances social theory, and provides novel policy insights in ways that no single approach affords. Despite the inherent complexity of social-ecological systems and the difficulty of causal inference, the 3FIGS approach can dramatically advance our understanding of, and the evidentiary basis for, effective MPAs and other conservation initiatives. © 2017 New York Academy of Sciences.

Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation.

PubMed

Witter, Sophie; Toonen, Jurrien; Meessen, Bruno; Kagubare, Jean; Fritsche, György; Vaughan, Kelsey

2013-09-29

Performance-based financing is increasingly being applied in a variety of contexts, with the expectation that it can improve the performance of health systems. However, while there is a growing literature on implementation issues and effects on outputs, there has been relatively little focus on interactions between PBF and health systems and how these should be studied. This paper aims to contribute to filling that gap by developing a framework for assessing the interactions between PBF and health systems, focusing on low and middle income countries. In doing so, it elaborates a general framework for monitoring and evaluating health system reforms in general. This paper is based on an exploratory literature review and on the work of a group of academics and PBF practitioners. The group developed ideas for the monitoring and evaluation framework through exchange of emails and working documents. Ideas were further refined through discussion at the Health Systems Research symposium in Beijing in October 2012, through comments from members of the online PBF Community of Practice and Beijing participants, and through discussion with PBF experts in Bergen in June 2013. The paper starts with a discussion of definitions, to clarify the core concept of PBF and how the different terms are used. It then develops a framework for monitoring its interactions with the health system, structured around five domains of context, the development process, design, implementation and effects. Some of the key questions for monitoring and evaluation are highlighted, and a systematic approach to monitoring effects proposed, structured according to the health system pillars, but also according to inputs, processes and outputs. The paper lays out a broad framework within which indicators can be prioritised for monitoring and evaluation of PBF or other health system reforms. It highlights the dynamic linkages between the domains and the different pillars. All of these are also framed within inter-sectoral and wider societal contexts. It highlights the importance of differentiating short term and long term effects, and also effects (intended and unintended) at different levels of the health system, and for different sectors and areas of the country. Outstanding work will include using and refining the framework and agreeing on the most important hypotheses to test using it, in relation to PBF but also other purchasing and provider payment reforms, as well as appropriate research methods to use for this task.

Evaluating a Federated Medical Search Engine

PubMed Central

Belden, J.; Williams, J.; Richardson, B.; Schuster, K.

2014-01-01

Summary Background Federated medical search engines are health information systems that provide a single access point to different types of information. Their efficiency as clinical decision support tools has been demonstrated through numerous evaluations. Despite their rigor, very few of these studies report holistic evaluations of medical search engines and even fewer base their evaluations on existing evaluation frameworks. Objectives To evaluate a federated medical search engine, MedSocket, for its potential net benefits in an established clinical setting. Methods This study applied the Human, Organization, and Technology (HOT-fit) evaluation framework in order to evaluate MedSocket. The hierarchical structure of the HOT-factors allowed for identification of a combination of efficiency metrics. Human fit was evaluated through user satisfaction and patterns of system use; technology fit was evaluated through the measurements of time-on-task and the accuracy of the found answers; and organization fit was evaluated from the perspective of system fit to the existing organizational structure. Results Evaluations produced mixed results and suggested several opportunities for system improvement. On average, participants were satisfied with MedSocket searches and confident in the accuracy of retrieved answers. However, MedSocket did not meet participants’ expectations in terms of download speed, access to information, and relevance of the search results. These mixed results made it necessary to conclude that in the case of MedSocket, technology fit had a significant influence on the human and organization fit. Hence, improving technological capabilities of the system is critical before its net benefits can become noticeable. Conclusions The HOT-fit evaluation framework was instrumental in tailoring the methodology for conducting a comprehensive evaluation of the search engine. Such multidimensional evaluation of the search engine resulted in recommendations for system improvement. PMID:25298813

Australia's personally controlled electronic health record and primary healthcare: generating a framework for implementation and evaluation.

PubMed

Almond, H; Cummings, E; Turner, P

2013-01-01

The Australian Government launched a personally controlled electronic health record (PCEHR) system in July 2012 committing $466.7m. Currently Australia lacks a clearly articulated implementation and evaluation framework and there remains limited detail on how this system's success will be determined. These problems are especially visible in primary healthcare. The UK and US, have been advocated as models, however they have started to report points of failure arising from their approaches. Evidence suggests that alternatives need to be considered, if mistakes are not to be replicated. Insights from e-health record implementation and evaluation approaches in Denmark and the Netherlands provide Australia with other approaches. The PCEHR requires different and radical thinking around the delivery of health services. Drawing on a range of English language articles identified between 1996 and 2012, the paper generates a conceptual framework for implementation and evaluation of the PCEHR. The generation of a grounded implementation and evaluation framework in primary healthcare will reduce provider scepticism and facilitate complex changes associated with PCEHR uptake.

Health policy in times of austerity-A conceptual framework for evaluating effects of policy on efficiency and equity illustrated with examples from Europe since 2008.

PubMed

Wenzl, Martin; Naci, Huseyin; Mossialos, Elias

2017-09-01

The objective of this paper is to provide a framework for evaluation of changes in health policy against overarching health system goals. We propose a categorisation of policies into seven distinct health system domains. We then develop existing analytical concepts of insurance coverage and cost-effectiveness further to evaluate the effects of policies in each domain on equity and efficiency. The framework is illustrated with likely effects of policy changes implemented in a sample of European countries since 2008. Our illustrative analysis suggests that cost containment has been the main focus and that countries have implemented a mix of measures that are efficient or efficiency neutral. Similarly, policies are likely to have mixed effects on equity. Additional user charges were a common theme but these were frequently accompanied by additional exemptions, making their likely effects on equity difficult to evaluate. We provide a framework for future, and more detailed, evaluations of changes in health policy. Copyright © 2017 The Authors. Published by Elsevier B.V. All rights reserved.

Child Support Enforcement: A Framework for Evaluating Costs, Benefits, and Effects.

DTIC Science & Technology

1991-03-01

efforts to gain and enforce child support awards might yield additional collections on behalf of these children, they would surely entail additional...framework for evaluating the full costs and ne . effects of child support enforcement.I This framework could assist your office and others in planning...following results of our develop- S . ............. .. mental work: (1) models of the child support enforcement system activi- AvajiabilitY Codes. ties

How to assess driver's interaction with partially automated driving systems - A framework for early concept assessment.

PubMed

van den Beukel, Arie P; van der Voort, Mascha C

2017-03-01

The introduction of partially automated driving systems changes the driving task into supervising the automation with an occasional need to intervene. To develop interface solutions that adequately support drivers in this new role, this study proposes and evaluates an assessment framework that allows designers to evaluate driver-support within relevant real-world scenarios. Aspects identified as requiring assessment in terms of driver-support within the proposed framework are Accident Avoidance, gained Situation Awareness (SA) and Concept Acceptance. Measurement techniques selected to operationalise these aspects and the associated framework are pilot-tested with twenty-four participants in a driving simulator experiment. The objective of the test is to determine the reliability of the applied measurements for the assessment of the framework and whether the proposed framework is effective in predicting the level of support offered by the concepts. Based on the congruency between measurement scores produced in the test and scores with predefined differences in concept-support, this study demonstrates the framework's reliability. A remaining concern is the framework's weak sensitivity to small differences in offered support. The article concludes that applying the framework is especially advantageous for evaluating early design phases and can successfully contribute to the efficient development of driver's in-control and safe means of operating partially automated vehicles. Copyright © 2016 Elsevier Ltd. All rights reserved.

Development of an economic framework to evaluate resilience in recovering from major port disruptions : final report.

DOT National Transportation Integrated Search

2017-02-03

Ports play a critical role in a nations economic system. The impact of a major port disruption can reverberate across the entire economy through regional and national supply-chains. This study develops an operational framework to evaluate the effe...

Performance Indicators (for Kindergartens). (First Edition).

ERIC Educational Resources Information Center

Hong Kong Education Dept.

As part of the 1999 comprehensive review of the education system in Hong Kong, the Education Department introduced a quality assurance framework in local kindergartens. The framework comprises two components: self-evaluation and external monitoring. This guide presents performance indicators for use as evaluation criteria with reference to the…

System modeling with the DISC framework: evidence from safety-critical domains.

PubMed

Reiman, Teemu; Pietikäinen, Elina; Oedewald, Pia; Gotcheva, Nadezhda

2012-01-01

The objective of this paper is to illustrate the development and application of the Design for Integrated Safety Culture (DISC) framework for system modeling by evaluating organizational potential for safety in nuclear and healthcare domains. The DISC framework includes criteria for good safety culture and a description of functions that the organization needs to implement in order to orient the organization toward the criteria. Three case studies will be used to illustrate the utilization of the DISC framework in practice.

Comprehensive Framework for Evaluating e-Learning Systems: Using BSC Framework

ERIC Educational Resources Information Center

Momeni, Mansor; Jamporazmey, Mona; Mehrafrouz, Mohsen; Bahadori, Fatemeh

2013-01-01

The development of information and communication technology (ICT) is changing the way in which people work, communicate and learn. Recently developing and implementing e-learning solutions have increased dramatically. According to heavily investing in this area, it is essential to evaluate its different aspects and understand measures, which…

Rebooting Kirkpatrick: Integrating Information System Theory Into the Evaluation of Web-based Continuing Professional Development Interventions for Interprofessional Education.

PubMed

Shen, Nelson; Yufe, Shira; Saadatfard, Omid; Sockalingam, Sanjeev; Wiljer, David

2017-01-01

Information system research has stressed the importance of theory in understanding how user perceptions can motivate the use and adoption of technology such as web-based continuing professional development programs for interprofessional education (WCPD-IPE). A systematic review was conducted to provide an information system perspective on the current state of WCPD-IPE program evaluation and how current evaluations capture essential theoretical constructs in promoting technology adoption. Six databases were searched to identify studies evaluating WCPD-IPE. Three investigators determined eligibility of the articles. Evaluation items extracted from the studies were assessed using the Kirkpatrick-Barr framework and mapped to the Benefits Evaluation Framework. Thirty-seven eligible studies yielded 362 evaluation items for analysis. Most items (n = 252) were assessed as Kirkpatrick-Barr level 1 (reaction) and were mainly focused on the quality (information, service, and quality) and satisfaction dimensions of the Benefits Evaluation. System quality was the least evaluated quality dimension, accounting for 26 items across 13 studies. WCPD-IPE use was reported in 17 studies and its antecedent factors were evaluated in varying degrees of comprehensiveness. Although user reactions were commonly evaluated, greater focus on user perceptions of system quality (ie, functionality and performance), usefulness, and usability of the web-based platform is required. Surprisingly, WCPD-IPE use was reported in less than half of the studies. This is problematic as use is a prerequisite to realizing any individual, organizational, or societal benefit of WCPD-IPE. This review proposes an integrated framework which accounts for these factors and provides a theoretically grounded guide for future evaluations.

Evaluate, assess, treat: development and evaluation of the EAT framework to increase effective communication regarding sensitive oral-systemic health issues.

PubMed

DeBate, R D; Cragun, D; Gallentine, A A; Severson, H H; Shaw, T; Cantwell, C; Christiansen, S; Koerber, A; Hendricson, W; Tomar, S L; McCormack Brown, K; Tedesco, L A

2012-11-01

Oral healthcare providers are likely to encounter a number of sensitive oral/systemic health issues whilst interacting with patients. The purpose of the current study was to develop and evaluate a framework aimed at oral healthcare providers to engage in active secondary prevention of eating disorders (i.e. early detection of oral manifestations of disordered eating behaviours, patient approach and communication, patient-specific oral treatment, and referral to care) for patients presenting with signs of disordered eating behaviours. The EAT Framework was developed based on the Brief Motivational Interviewing (B-MI) conceptual framework and comprises three continuous steps: Evaluating, Assessing, and Treating. Using a group-randomized control design, 11 dental hygiene (DH) and seven dental (D) classes from eight institutions were randomized to either the intervention or control conditions. Both groups completed pre- and post-intervention assessments. Hierarchical linear models were conducted to measure the effects of the intervention whilst controlling for baseline levels. Statistically significant improvements from pre- to post-intervention were observed in the Intervention group compared with the Control group on knowledge of eating disorders and oral findings, skills-based knowledge, and self-efficacy (all P < 0.01). Effect sizes ranged from 0.57 to 0.95. No statistically significant differences in outcomes were observed by type of student. Although the EAT Framework was developed as part of a larger study on secondary prevention of eating disorders, the procedures and skills presented can be applied to other sensitive oral/systemic health issues. Because the EAT Framework was developed by translating B-MI principles and procedures, the framework can be easily adopted as a non-confrontational method for patient communication. © 2012 John Wiley & Sons A/S.

Evaluate, Assess, Treat: Development and evaluation of the EAT framework to increase effective communication regarding sensitive oral-systemic health issues

PubMed Central

DeBate, Rita D.; Cragun, Deborah; Gallentine, Ashley A.; Severson, Herbert H.; Shaw, Tracey; Cantwell, Carley; Christiansen, Steve; Koerber, Anne; Hendricson, William; Tomar, Scott L.; Brown, Kelli McCormack; Tedesco, Lisa A.

2012-01-01

Oral healthcare providers are likely to encounter a number of sensitive oral/systemic health issues while interacting with patients. The purpose of the current study was to develop and evaluate a framework aimed at oral healthcare providers to engage in active secondary prevention of eating disorders (i.e., early detection of oral manifestations of disordered eating behaviors, patient approach and communication, patient-specific oral treatment, and referral to care) for patients presenting with signs of disordered eating behaviors. The EAT Framework was developed based on the Brief Motivational Interviewing (B-MI) conceptual framework and comprises three continuous steps: Evaluating, Assessing, and Treating. Using a group-randomized control design, 11 dental hygiene (DH) and 7 dental (D) classes from 8 institutions were randomized to either the intervention or control conditions. Both groups completed preand post-intervention assessments. Hierarchical linear models were conducted to measure the effects of the intervention while controlling for baseline levels. Statistically significant improvements from pre-to post-intervention were observed in the Intervention group compared with the Control group on knowledge of eating disorders and oral findings, skills-based knowledge, and self-efficacy (all p < .01). Effect sizes ranged from .57–.95. No statistically significant differences in outcomes were observed by type of student. Although the EAT Framework was developed as part of a larger study on secondary prevention of eating disorders, the procedures and skills presented can be applied to other sensitive oral/systemic health issues. Because the EAT Framework was developed by translating B-MI principles and procedures, the framework can be easily adopted as a non-confrontational method for patient communication. PMID:23050505

Secondary School Administrators' Perceptions of Louisiana's Compass System as a Framework for Teacher Evaluation

ERIC Educational Resources Information Center

Williams, Kathleen; Hebert, Dustin

2017-01-01

Louisiana's relatively new Compass teacher observation and evaluation system is used to evaluate teacher quality or effectiveness in P-12 public schools. Secondary school administrators in one district were interviewed about their perceptions of the system and, especially, an iteration of the Danielson rubric used for teacher evaluation. Findings…

A Health Systems Approach to Integrated Community Case Management of Childhood Illness: Methods and Tools

PubMed Central

McGorman, Laura; Marsh, David R.; Guenther, Tanya; Gilroy, Kate; Barat, Lawrence M.; Hammamy, Diaa; Wansi, Emmanuel; Peterson, Stefan; Hamer, Davidson H.; George, Asha

2012-01-01

Integrated community case management (iCCM) of childhood illness is an increasingly popular strategy to expand life-saving health services to underserved communities. However, community health approaches vary widely across countries and do not always distribute resources evenly across local health systems. We present a harmonized framework, developed through interagency consultation and review, which supports the design of CCM by using a systems approach. To verify that the framework produces results, we also suggest a list of complementary indicators, including nine global metrics, and a menu of 39 country-specific measures. When used by program managers and evaluators, we propose that the framework and indicators can facilitate the design, implementation, and evaluation of community case management. PMID:23136280

Judgment under uncertainty; a probabilistic evaluation framework for decision-making about sanitation systems in low-income countries.

PubMed

Malekpour, Shirin; Langeveld, Jeroen; Letema, Sammy; Clemens, François; van Lier, Jules B

2013-03-30

This paper introduces the probabilistic evaluation framework, to enable transparent and objective decision-making in technology selection for sanitation solutions in low-income countries. The probabilistic framework recognizes the often poor quality of the available data for evaluations. Within this framework, the evaluations will be done based on the probabilities that the expected outcomes occur in practice, considering the uncertainties in evaluation parameters. Consequently, the outcome of evaluations will not be single point estimates; but there exists a range of possible outcomes. A first trial application of this framework for evaluation of sanitation options in the Nyalenda settlement in Kisumu, Kenya, showed how the range of values that an evaluation parameter may obtain in practice would influence the evaluation outcomes. In addition, as the probabilistic evaluation requires various site-specific data, sensitivity analysis was performed to determine the influence of each data set quality on the evaluation outcomes. Based on that, data collection activities could be (re)directed, in a trade-off between the required investments in those activities and the resolution of the decisions that are to be made. Copyright © 2013 Elsevier Ltd. All rights reserved.

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation

PubMed Central

Barasa, Edwine W.; Molyneux, Sassy; English, Mike; Cleary, Susan

2015-01-01

Background: Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices. Methods: We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken. Results: Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values. Conclusion: Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought. PMID:26673332

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation.

PubMed

Barasa, Edwine W; Molyneux, Sassy; English, Mike; Cleary, Susan

2015-09-16

Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices. We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken. Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values. Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought. © 2015 by Kerman University of Medical Sciences.

Dynamic decision-making for reliability and maintenance analysis of manufacturing systems based on failure effects

NASA Astrophysics Data System (ADS)

Zhang, Ding; Zhang, Yingjie

2017-09-01

A framework for reliability and maintenance analysis of job shop manufacturing systems is proposed in this paper. An efficient preventive maintenance (PM) policy in terms of failure effects analysis (FEA) is proposed. Subsequently, reliability evaluation and component importance measure based on FEA are performed under the PM policy. A job shop manufacturing system is applied to validate the reliability evaluation and dynamic maintenance policy. Obtained results are compared with existed methods and the effectiveness is validated. Some vague understandings for issues such as network modelling, vulnerabilities identification, the evaluation criteria of repairable systems, as well as PM policy during manufacturing system reliability analysis are elaborated. This framework can help for reliability optimisation and rational maintenance resources allocation of job shop manufacturing systems.

A frequency averaging framework for the solution of complex dynamic systems

PubMed Central

Lecomte, Christophe

2014-01-01

A frequency averaging framework is proposed for the solution of complex linear dynamic systems. It is remarkable that, while the mid-frequency region is usually very challenging, a smooth transition from low- through mid- and high-frequency ranges is possible and all ranges can now be considered in a single framework. An interpretation of the frequency averaging in the time domain is presented and it is explained that the average may be evaluated very efficiently in terms of system solutions. PMID:24910518

Framework for Evaluating Educational Systemic Initiatives.

ERIC Educational Resources Information Center

Ikegulu, T. Nelson

This paper describes the implementation of the Holistic Systemic Evaluation (HSE), a component of an Education Systemic Initiative's strategic management. The HSE provides general guidance for the implementation and continual improvement of an Education Systemic Initiative Reform (ESIR). The implementation of the education system initiative plan:…

An Architecture Framework for Orchestrating Context-Aware IT Ecosystems: A Case Study for Quantitative Evaluation †.

PubMed

Park, Soojin; Park, Sungyong; Park, Young B

2018-02-12

With the emergence of various forms of smart devices and new paradigms such as the Internet of Things (IoT) concept, the IT (Information Technology) service areas are expanding explosively compared to the provision of services by single systems. A new system operation concept that has emerged in accordance with such technical trends is the IT ecosystem. The IT ecosystem can be considered a special type of system of systems in which multiple systems with various degrees of autonomy achieve common goals while adapting to the given environment. The single systems that participate in the IT ecosystem adapt autonomously to the current situation based on collected data from sensors. Furthermore, to maintain the services supported by the whole IT ecosystem sustainably, the configuration of single systems that participate in the IT ecosystem also changes appropriately in accordance with the changed situation. In order to support the IT ecosystem, this paper proposes an architecture framework that supports dynamic configuration changes to achieve the goal of the whole IT ecosystem, while ensuring the autonomy of single systems through the collection of data from sensors so as to recognize the situational context of individual participating systems. For the feasibility evaluation of the proposed framework, a simulated example of an IT ecosystem for unmanned forest management was constructed, and the quantitative evaluation results are discussed in terms of the extent to which the proposed architecture framework can continuously provide sustainable services in response to diverse environmental context changes.

An Architecture Framework for Orchestrating Context-Aware IT Ecosystems: A Case Study for Quantitative Evaluation †

PubMed Central

Park, Young B.

2018-01-01

With the emergence of various forms of smart devices and new paradigms such as the Internet of Things (IoT) concept, the IT (Information Technology) service areas are expanding explosively compared to the provision of services by single systems. A new system operation concept that has emerged in accordance with such technical trends is the IT ecosystem. The IT ecosystem can be considered a special type of system of systems in which multiple systems with various degrees of autonomy achieve common goals while adapting to the given environment. The single systems that participate in the IT ecosystem adapt autonomously to the current situation based on collected data from sensors. Furthermore, to maintain the services supported by the whole IT ecosystem sustainably, the configuration of single systems that participate in the IT ecosystem also changes appropriately in accordance with the changed situation. In order to support the IT ecosystem, this paper proposes an architecture framework that supports dynamic configuration changes to achieve the goal of the whole IT ecosystem, while ensuring the autonomy of single systems through the collection of data from sensors so as to recognize the situational context of individual participating systems. For the feasibility evaluation of the proposed framework, a simulated example of an IT ecosystem for unmanned forest management was constructed, and the quantitative evaluation results are discussed in terms of the extent to which the proposed architecture framework can continuously provide sustainable services in response to diverse environmental context changes. PMID:29439540

SurF: an innovative framework in biosecurity and animal health surveillance evaluation.

PubMed

Muellner, Petra; Watts, Jonathan; Bingham, Paul; Bullians, Mark; Gould, Brendan; Pande, Anjali; Riding, Tim; Stevens, Paul; Vink, Daan; Stärk, Katharina Dc

2018-05-16

Surveillance for biosecurity hazards is being conducted by the New Zealand Competent Authority, the Ministry for Primary Industries (MPI) to support New Zealand's biosecurity system. Surveillance evaluation should be an integral part of the surveillance life cycle, as it provides a means to identify and correct problems and to sustain and enhance the existing strengths of a surveillance system. The surveillance evaluation Framework (SurF) presented here was developed to provide a generic framework within which the MPI biosecurity surveillance portfolio, and all of its components, can be consistently assessed. SurF is an innovative, cross-sectoral effort that aims to provide a common umbrella for surveillance evaluation in the animal, plant, environment and aquatic sectors. It supports the conduct of the following four distinct components of an evaluation project: (i) motivation for the evaluation, (ii) scope of the evaluation, (iii) evaluation design and implementation and (iv) reporting and communication of evaluation outputs. Case studies, prepared by MPI subject matter experts, are included in the framework to guide users in their assessment. Three case studies were used in the development of SurF in order to assure practical utility and to confirm usability of SurF across all included sectors. It is anticipated that the structured approach and information provided by SurF will not only be of benefit to MPI but also to other New Zealand stakeholders. Although SurF was developed for internal use by MPI, it could be applied to any surveillance system in New Zealand or elsewhere. © 2018 2018 The Authors. Transboundary and Emerging Diseases Published by Blackwell Verlag GmbH.

Principles and Actions: A Framework for Systemic Change.

ERIC Educational Resources Information Center

Barkley, Robert, Jr.; Castle, Shari

This paper outlines a framework designed to help school districts evaluate themselves during the implementation of systemic change. Based on the experiences of districts that participated in the NEA Learning Laboratories Initiative, a process called "rapporteuring" was developed. The process provokes the particular site into serious…

A Systems Framework for Assessing Plumbing Products-Related Water Conservation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Williams, Alison; Dunham Whitehead, Camilla; Lutz, James

2011-12-02

Reducing the water use of plumbing products—toilets, urinals, faucets, and showerheads —has been a popular conservation measure. Improved technologies have created opportunities for additional conservation in this area. However, plumbing products do not operate in a vacuum. This paper reviews the literature related to plumbing products to determine a systems framework for evaluating future conservation measures using these products. The main framework comprises the following categories: water use efficiency, product components, product performance, source water, energy, and plumbing/sewer infrastructure. This framework for analysis provides a starting point for professionals considering future water conservation measures to evaluate the need for additionalmore » research, collaboration with other standards or codes committees, and attachment of additional metrics to water use efficiency (such as performance).« less

Screening Systems and Decision Making at the Preschool Level: Application of a Comprehensive Validity Framework

ERIC Educational Resources Information Center

Kettler, Ryan J.; Feeney-Kettler, Kelly A.

2011-01-01

Universal screening is designed to be an efficient method for identifying preschool students with mental health problems, but prior to use, screening systems must be evaluated to determine their appropriateness within a specific setting. In this article, an evidence-based validity framework is applied to four screening systems for identifying…

The Power of the Frame: Systems Transformation Framework for Health Care Leaders.

PubMed

Scott, Kathy A; Pringle, Janice

Health care leaders are responsible for oversight of multiple and competing change interventions. These interventions regularly fail to achieve the desired outcomes and/or sustainable results. This often occurs because of the mental models and approaches that are used to plan, design, implement, and evaluate the system. These do not account for inherent characteristics that determine the system's likely ability to innovate while maintaining operational effectiveness. Theories exist on how to assess a system's readiness to change, but the definitions, constructs, and assessments are diverse and often look at facets of systems in isolation. The Systems Transformation Framework prescriptively defines and characterizes system domains on the basis of complex adaptive systems theory so that domains can be assessed in tandem. As a result, strengths and challenges to implementation are recognized before implementation begins. The Systems Transformation Framework defines 8 major domains: vision, leadership, organizational culture, organizational behavior, organizational structure, performance measurements, internal learning, and external learning. Each domain has principles that are critical for creating the conditions that lead to successful organizational adaptation and change. The Systems Transformation Framework can serve as a guide for health care leaders at all levels of the organization to (1) create environments that are change ready and (2) plan, design, implement, and evaluate change within complex adaptive systems.

Investigating Choice Experiments for Preferences of Older People (ICEPOP): evaluative spaces in health economics.

PubMed

Coast, Joanna; Flynn, Terry; Sutton, Eileen; Al-Janabi, Hareth; Vosper, Jane; Lavender, Sarita; Louviere, Jordan; Peters, Tim

2008-10-01

This paper deals with three concerns about the evaluative framework that is currently dominant within health economics. These concerns are: that the evaluative framework is concerned entirely with health; that the evaluative framework has an individualistic focus on patients alone; and that the methods used to estimate 'health' within the current evaluative framework could be improved both in terms of the generation of descriptive systems and in using valuation methods that rely less on people's ability to express their preferences on a cardinal scale. In exploring these issues the Investigating Choice Experiments for Preferences of Older People (ICEPOP) programme has explicitly focused on both the topic of older people and the methods of discrete choice experiments. A capability index has been developed and attributes for an economic measure of end-of-life care are currently being generated, providing the possibility of extending the evaluative framework beyond health alone. A measure of carer's experience and a framework for extending measurement in end-of-life care to loved ones are both also in development, thus extending the evaluative framework beyond the patient alone. Rigorous qualitative methods employing an iterative approach have been developed for use in constructing attributes, and best-worst scaling has been utilized to reduce task complexity and provide insights into heterogeneity. There are a number of avenues for further research in all these areas, but in particular there is need for greater attention to be paid to the theory underlying the evaluative framework within health economics.

Development of intuitive rules: evaluating the application of the dual-system framework to understanding children's intuitive reasoning.

PubMed

Osman, Magda; Stavy, Ruth

2006-12-01

Theories of adult reasoning propose that reasoning consists of two functionally distinct systems that operate under entirely different mechanisms. This theoretical framework has been used to account for a wide range of phenomena, which now encompasses developmental research on reasoning and problem solving. We begin this review by contrasting three main dual-system theories of adult reasoning (Evans & Over, 1996; Sloman, 1996; Stanovich & West, 2000) with a well-established developmental account that also incorporates a dual-system framework (Brainerd & Reyna, 2001). We use developmental studies of the formation and application of intuitive rules in science and mathematics to evaluate the claims that these theories make. Overall, the evidence reviewed suggests that what is crucial to understanding how children reason is the saliency of the features that are presented within a task. By highlighting the importance of saliency as a way of understanding reasoning, we aim to provide clarity concerning the benefits and limitations of adopting a dual-system framework to account for evidence from developmental studies of intuitive reasoning.

The Contribution of Quality Assurance Reviews to Development in School Systems. Draft.

ERIC Educational Resources Information Center

Cuttance, Peter

This paper discusses key aspects of the British experience with school review and evaluation, which influenced the development of the South Australian framework for school evaluation. The discussion combines the requirements of accountability and development within a framework of quality assurance. The paper describes shortcomings of the internal…

Advanced Public Transportation Systems. Technical Assistance Brief 2

DOT National Transportation Integrated Search

1993-01-01

The Advanced Public Transportation Systems (APTS) Program requires evaluation : of pilot projects. This technical assistance brief discusses the guidelines set : for developing evaluation framework and methodology for local projects. 4p.

"Second Chance": Some Theoretical and Empirical Remarks.

ERIC Educational Resources Information Center

Inbar, Dan E.; Sever, Rita

1986-01-01

Presents a conceptual framework of second-chance systems analyzable in terms of several basic parameters (targeted population, declared goals, processes, options for students, evaluation criteria, and implications for the regular system). Uses this framework to analyze an Israeli external high school, the subject of a large-scale study. Includes 3…

Graphical Means for Inspecting Qualitative Models of System Behaviour

ERIC Educational Resources Information Center

Bouwer, Anders; Bredeweg, Bert

2010-01-01

This article presents the design and evaluation of a tool for inspecting conceptual models of system behaviour. The basis for this research is the Garp framework for qualitative simulation. This framework includes modelling primitives, such as entities, quantities and causal dependencies, which are combined into model fragments and scenarios.…

HCI∧2 framework: a software framework for multimodal human-computer interaction systems.

PubMed

Shen, Jie; Pantic, Maja

2013-12-01

This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI∧2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-based data transport protocol for message delivery and a TCP-based system management protocol. The latter ensures that the integrity of system structure is maintained at runtime. With the inclusion of bridging modules, the HCI∧2 Framework is interoperable with other software frameworks including Psyclone and ActiveMQ. In addition to the core communication middleware, we also present the integrated development environment (IDE) of the HCI∧2 Framework. It provides a complete graphical environment to support every step in a typical MHCI system development process, including module development, debugging, packaging, and management, as well as the whole system management and testing. The quantitative evaluation indicates that our framework outperforms other similar tools in terms of average message latency and maximum data throughput under a typical single PC scenario. To demonstrate HCI∧2 Framework's capabilities in integrating heterogeneous modules, we present several example modules working with a variety of hardware and software. We also present an example of a full system developed using the proposed HCI∧2 Framework, which is called the CamGame system and represents a computer game based on hand-held marker(s) and low-cost camera(s).

Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation

PubMed Central

2013-01-01

Background Performance-based financing is increasingly being applied in a variety of contexts, with the expectation that it can improve the performance of health systems. However, while there is a growing literature on implementation issues and effects on outputs, there has been relatively little focus on interactions between PBF and health systems and how these should be studied. This paper aims to contribute to filling that gap by developing a framework for assessing the interactions between PBF and health systems, focusing on low and middle income countries. In doing so, it elaborates a general framework for monitoring and evaluating health system reforms in general. Methods This paper is based on an exploratory literature review and on the work of a group of academics and PBF practitioners. The group developed ideas for the monitoring and evaluation framework through exchange of emails and working documents. Ideas were further refined through discussion at the Health Systems Research symposium in Beijing in October 2012, through comments from members of the online PBF Community of Practice and Beijing participants, and through discussion with PBF experts in Bergen in June 2013. Results The paper starts with a discussion of definitions, to clarify the core concept of PBF and how the different terms are used. It then develops a framework for monitoring its interactions with the health system, structured around five domains of context, the development process, design, implementation and effects. Some of the key questions for monitoring and evaluation are highlighted, and a systematic approach to monitoring effects proposed, structured according to the health system pillars, but also according to inputs, processes and outputs. Conclusions The paper lays out a broad framework within which indicators can be prioritised for monitoring and evaluation of PBF or other health system reforms. It highlights the dynamic linkages between the domains and the different pillars. All of these are also framed within inter-sectoral and wider societal contexts. It highlights the importance of differentiating short term and long term effects, and also effects (intended and unintended) at different levels of the health system, and for different sectors and areas of the country. Outstanding work will include using and refining the framework and agreeing on the most important hypotheses to test using it, in relation to PBF but also other purchasing and provider payment reforms, as well as appropriate research methods to use for this task. PMID:24073625

Framework for evaluating disease severity measures in older adults with comorbidity.

PubMed

Boyd, Cynthia M; Weiss, Carlos O; Halter, Jeff; Han, K Carol; Ershler, William B; Fried, Linda P

2007-03-01

Accounting for the influence of concurrent conditions on health and functional status for both research and clinical decision-making purposes is especially important in older adults. Although approaches to classifying severity of individual diseases and conditions have been developed, the utility of these classification systems has not been evaluated in the presence of multiple conditions. We present a framework for evaluating severity classification systems for common chronic diseases. The framework evaluates the: (a) goal or purpose of the classification system; (b) physiological and/or functional criteria for severity graduation; and (c) potential reliability and validity of the system balanced against burden and costs associated with classification. Approaches to severity classification of individual diseases were not originally conceived for the study of comorbidity. Therefore, they vary greatly in terms of objectives, physiological systems covered, level of severity characterization, reliability and validity, and costs and burdens. Using different severity classification systems to account for differing levels of disease severity in a patient with multiple diseases, or, assessing global disease burden may be challenging. Most approaches to severity classification are not adequate to address comorbidity. Nevertheless, thoughtful use of some existing approaches and refinement of others may advance the study of comorbidity and diagnostic and therapeutic approaches to patients with multimorbidity.

Challenges in evaluating clinical governance systems in iran: a qualitative study.

PubMed

Hooshmand, Elaheh; Tourani, Sogand; Ravaghi, Hamid; Ebrahimipour, Hossein

2014-04-01

In spite of the pivotal role of clinical governance in enhancing quality of services provided by hospitals across the country, a scientific framework with specific criteria for evaluating hospitals has not been developed so far. This study was conducted with the aim to identify the challenges involved in evaluating systems of clinical governance in Iran. For the purposes of this qualitative study, 15 semi-structured interviews with experts in the field were conducted in 2011 and the data were analyzed using framework analysis method. Five major challenges in evaluating clinical governance include managing human resources, improving clinical quality, managing development, organizing clinical governance, and providing patient-oriented healthcare system. Healthcare system in Iran requires a clinical governance program which has a patient-oriented approach in philosophy, operation, and effectiveness in order to meet the challenges ahead.

A fuzzy logic expert system for evaluating policy progress towards sustainability goals.

PubMed

Cisneros-Montemayor, Andrés M; Singh, Gerald G; Cheung, William W L

2017-12-16

Evaluating progress towards environmental sustainability goals can be difficult due to a lack of measurable benchmarks and insufficient or uncertain data. Marine settings are particularly challenging, as stakeholders and objectives tend to be less well defined and ecosystem components have high natural variability and are difficult to observe directly. Fuzzy logic expert systems are useful analytical frameworks to evaluate such systems, and we develop such a model here to formally evaluate progress towards sustainability targets based on diverse sets of indicators. Evaluation criteria include recent (since policy enactment) and historical (from earliest known state) change, type of indicators (state, benefit, pressure, response), time span and spatial scope, and the suitability of an indicator in reflecting progress toward a specific objective. A key aspect of the framework is that all assumptions are transparent and modifiable to fit different social and ecological contexts. We test the method by evaluating progress towards four Aichi Biodiversity Targets in Canadian oceans, including quantitative progress scores, information gaps, and the sensitivity of results to model and data assumptions. For Canadian marine systems, national protection plans and biodiversity awareness show good progress, but species and ecosystem states overall do not show strong improvement. Well-defined goals are vital for successful policy implementation, as ambiguity allows for conflicting potential indicators, which in natural systems increases uncertainty in progress evaluations. Importantly, our framework can be easily adapted to assess progress towards policy goals with different themes, globally or in specific regions.

Re-Evaluating Course Evaluations: Clarity, Visibility, and Functionality

ERIC Educational Resources Information Center

Richardson, Stephanie Jean; Coleman, Darrell; Stephenson, Jill

2014-01-01

This article presents an innovative framework that provides a means to understand and reevaluate student course evaluation systems. We present three major concepts vital to course evaluation systems and explain how they inform five evolutionary stages. Additionally, we show how the major stakeholders--students, faculty and administrators--are…

A Conceptual Framework for the Evaluation of Emergency Risk Communications

PubMed Central

Lin, Leesa; Gamhewage, Gaya M.

2017-01-01

Objectives. To articulate a conceptual framework in support of evaluation activities in emergency risk communications (ERC). Methods. The framework proposed is based on a systematic review of the scientific literature (2001–2016) combined with data derived from a series of semistructured interviews with experts and practitioners in ERC, and it is designed to support local, national, and international public health organizations in implementing evaluation studies in ERC. Results. We identified a list of ERC outcomes from the full-text review of 152 articles and categorized these into 3 groups, depending upon the level at which the outcome was measured: (1) information environment, (2) population, and (3) public health system. We analyzed interviewees’ data from 18 interviews to identify practices and processes related to the effectiveness of ERC and included these as key structural components and processes in the developed evaluation framework. Conclusions. Researchers and public health practitioners interested in the evaluation of ERC can use the conceptual framework described in this article to guide the development of evaluation studies and methods for assessing communication outcomes related to public health emergencies. PMID:28892436

A Conceptual Framework for the Evaluation of Emergency Risk Communications.

PubMed

Savoia, Elena; Lin, Leesa; Gamhewage, Gaya M

2017-09-01

To articulate a conceptual framework in support of evaluation activities in emergency risk communications (ERC). The framework proposed is based on a systematic review of the scientific literature (2001-2016) combined with data derived from a series of semistructured interviews with experts and practitioners in ERC, and it is designed to support local, national, and international public health organizations in implementing evaluation studies in ERC. We identified a list of ERC outcomes from the full-text review of 152 articles and categorized these into 3 groups, depending upon the level at which the outcome was measured: (1) information environment, (2) population, and (3) public health system. We analyzed interviewees' data from 18 interviews to identify practices and processes related to the effectiveness of ERC and included these as key structural components and processes in the developed evaluation framework. Researchers and public health practitioners interested in the evaluation of ERC can use the conceptual framework described in this article to guide the development of evaluation studies and methods for assessing communication outcomes related to public health emergencies.

Feasibility Study of a Generalized Framework for Developing Computer-Aided Detection Systems-a New Paradigm.

PubMed

Nemoto, Mitsutaka; Hayashi, Naoto; Hanaoka, Shouhei; Nomura, Yukihiro; Miki, Soichiro; Yoshikawa, Takeharu

2017-10-01

We propose a generalized framework for developing computer-aided detection (CADe) systems whose characteristics depend only on those of the training dataset. The purpose of this study is to show the feasibility of the framework. Two different CADe systems were experimentally developed by a prototype of the framework, but with different training datasets. The CADe systems include four components; preprocessing, candidate area extraction, candidate detection, and candidate classification. Four pretrained algorithms with dedicated optimization/setting methods corresponding to the respective components were prepared in advance. The pretrained algorithms were sequentially trained in the order of processing of the components. In this study, two different datasets, brain MRA with cerebral aneurysms and chest CT with lung nodules, were collected to develop two different types of CADe systems in the framework. The performances of the developed CADe systems were evaluated by threefold cross-validation. The CADe systems for detecting cerebral aneurysms in brain MRAs and for detecting lung nodules in chest CTs were successfully developed using the respective datasets. The framework was shown to be feasible by the successful development of the two different types of CADe systems. The feasibility of this framework shows promise for a new paradigm in the development of CADe systems: development of CADe systems without any lesion specific algorithm designing.

A development framework for semantically interoperable health information systems.

PubMed

Lopez, Diego M; Blobel, Bernd G M E

2009-02-01

Semantic interoperability is a basic challenge to be met for new generations of distributed, communicating and co-operating health information systems (HIS) enabling shared care and e-Health. Analysis, design, implementation and maintenance of such systems and intrinsic architectures have to follow a unified development methodology. The Generic Component Model (GCM) is used as a framework for modeling any system to evaluate and harmonize state of the art architecture development approaches and standards for health information systems as well as to derive a coherent architecture development framework for sustainable, semantically interoperable HIS and their components. The proposed methodology is based on the Rational Unified Process (RUP), taking advantage of its flexibility to be configured for integrating other architectural approaches such as Service-Oriented Architecture (SOA), Model-Driven Architecture (MDA), ISO 10746, and HL7 Development Framework (HDF). Existing architectural approaches have been analyzed, compared and finally harmonized towards an architecture development framework for advanced health information systems. Starting with the requirements for semantic interoperability derived from paradigm changes for health information systems, and supported in formal software process engineering methods, an appropriate development framework for semantically interoperable HIS has been provided. The usability of the framework has been exemplified in a public health scenario.

On the Validity of Educational Evaluation and Its Construction

ERIC Educational Resources Information Center

Huang, Xiaoping; Hu, Zhongfeng

2015-01-01

The main problem of the educational evaluation validity is that it just copies the conceptual framework system of validity from educational measurement to its own conceptual system. The validity conceptual system that fits the need of theory and practice of educational evaluation has not been established yet. According to the inherent attributive…

A general framework for multicharacter segmentation and its application in recognizing multilingual Asian documents

NASA Astrophysics Data System (ADS)

Wen, Di; Ding, Xiaoqing

2003-12-01

In this paper we propose a general framework for character segmentation in complex multilingual documents, which is an endeavor to combine the traditionally separated segmentation and recognition processes into a cooperative system. The framework contains three basic steps: Dissection, Local Optimization and Global Optimization, which are designed to fuse various properties of the segmentation hypotheses hierarchically into a composite evaluation to decide the final recognition results. Experimental results show that this framework is general enough to be applied in variety of documents. A sample system based on this framework to recognize Chinese, Japanese and Korean documents and experimental performance is reported finally.

A conceptual framework for evaluation of public health and primary care system performance in iran.

PubMed

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Akbari Sari, Ali; Mesdaghinia, Alireza

2015-01-26

The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report.

Evaluation framework for 16 earmarked projects in Washington State

DOT National Transportation Integrated Search

2007-05-01

This report documents the results of applying a previously developed, standardized approach for evaluating advanced traveler information systems (ATIS) projects to a much more diverse group of 16 intelligent transportation systems (ITS) projects. The...

A usability evaluation toolkit for In-Vehicle Information Systems (IVISs).

PubMed

Harvey, Catherine; Stanton, Neville A; Pickering, Carl A; McDonald, Mike; Zheng, Pengjun

2011-05-01

Usability must be defined specifically for the context of use of the particular system under investigation. This specific context of use should also be used to guide the definition of specific usability criteria and the selection of appropriate evaluation methods. There are four principles which can guide the selection of evaluation methods, relating to the information required in the evaluation, the stage at which to apply methods, the resources required and the people involved in the evaluation. This paper presents a framework for the evaluation of usability in the context of In-Vehicle Information Systems (IVISs). This framework guides designers through defining usability criteria for an evaluation, selecting appropriate evaluation methods and applying those methods. These stages form an iterative process of design-evaluation-redesign with the overall aim of improving the usability of IVISs and enhancing the driving experience, without compromising the safety of the driver. Copyright © 2010 Elsevier Ltd and The Ergonomics Society. All rights reserved.

The impact of contracting-out on health system performance: a conceptual framework.

PubMed

Liu, Xingzhu; Hotchkiss, David R; Bose, Sujata

2007-07-01

Despite the increased popularity of contracting-out of health services in developing countries, its effectiveness on overall health system performance is not yet conclusive. Except for substantial evidence of contracting-out's positive effect on access to health services and some evidence on improved equity in access, there is little evidence of contracting-out's impact on quality and efficiency. Most studies on the subject evaluate specific contracting-out projects against narrowly specified project objectives, not against more broadly defined health system goals. For this reason, conclusions of positive effects pertaining to project level may not hold at system level. This paper presents a conceptual framework that is expected to facilitate comprehensive, rigorous, and standardized evaluation of contracting-out at health system level. Specifically, this framework supports: full and standardized description of contracting-out interventions, study of the determinants of effectiveness, examination of provider and purchaser responses, assessment of the impact of contracting-out on all dimensions of health system performance, and cross-project analyses.

A Framework for Evidence-Based Licensure of Adaptive Autonomous Systems

DTIC Science & Technology

2016-03-01

insights gleaned to DoD. The autonomy community has identified significant challenges associated with test, evaluation verification and validation of...licensure as a test, evaluation, verification , and validation (TEVV) framework that can address these challenges. IDA found that traditional...language requirements to testable (preferably machine testable) specifications • Design of architectures that treat development and verification of

Development of a Design Supporting System for Nano-Materials based on a Framework for Integrated Knowledge of Functioning-Manufacturing Process

NASA Astrophysics Data System (ADS)

Tarumi, Shinya; Kozaki, Kouji; Kitamura, Yoshinobu; Mizoguchi, Riichiro

In the recent materials research, much work aims at realization of ``functional materials'' by changing structure and/or manufacturing process with nanotechnology. However, knowledge about the relationship among function, structure and manufacturing process is not well organized. So, material designers have to consider a lot of things at the same time. It would be very helpful for them to support their design process by a computer system. In this article, we discuss a conceptual design supporting system for nano-materials. Firstly, we consider a framework for representing functional structures and manufacturing processes of nano-materials with relationships among them. We expand our former framework for representing functional knowledge based on our investigation through discussion with experts of nano-materials. The extended framework has two features: 1) it represents functional structures and manufacturing processes comprehensively, 2) it expresses parameters of function and ways with their dependencies because they are important for material design. Next, we describe a conceptual design support system we developed based on the framework with its functionalities. Lastly, we evaluate the utility of our system in terms of functionality for design supports. For this purpose, we tried to represent two real examples of material design. And then we did an evaluation experiment on conceptual design of material using our system with the collaboration of domain experts.

Framework for Architecture Trade Study Using MBSE and Performance Simulation

NASA Technical Reports Server (NTRS)

Ryan, Jessica; Sarkani, Shahram; Mazzuchim, Thomas

2012-01-01

Increasing complexity in modern systems as well as cost and schedule constraints require a new paradigm of system engineering to fulfill stakeholder needs. Challenges facing efficient trade studies include poor tool interoperability, lack of simulation coordination (design parameters) and requirements flowdown. A recent trend toward Model Based System Engineering (MBSE) includes flexible architecture definition, program documentation, requirements traceability and system engineering reuse. As a new domain MBSE still lacks governing standards and commonly accepted frameworks. This paper proposes a framework for efficient architecture definition using MBSE in conjunction with Domain Specific simulation to evaluate trade studies. A general framework is provided followed with a specific example including a method for designing a trade study, defining candidate architectures, planning simulations to fulfill requirements and finally a weighted decision analysis to optimize system objectives.

Education for All in South Africa: Developing a National System for Quality Assurance.

ERIC Educational Resources Information Center

Smith, William J.; Ngoma-Maema, Wendy Yolisa

2003-01-01

Draws on international research, policy, and practice relevant to quality assurance systems to analyze the development of a national framework for educational quality assurance in South Africa. Describes an emerging framework for quality assurance that encompasses evaluation of student achievement, quality audits and reviews, program and service…

Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation. Volume 1: Study results

NASA Technical Reports Server (NTRS)

Lowrie, J. W.; Fermelia, A. J.; Haley, D. C.; Gremban, K. D.; Vanbaalen, J.; Walsh, R. W.

1982-01-01

A variety of artificial intelligence techniques which could be used with regard to NASA space applications and robotics were evaluated. The techniques studied were decision tree manipulators, problem solvers, rule based systems, logic programming languages, representation language languages, and expert systems. The overall structure of a robotic simulation tool was defined and a framework for that tool developed. Nonlinear and linearized dynamics equations were formulated for n link manipulator configurations. A framework for the robotic simulation was established which uses validated manipulator component models connected according to a user defined configuration.

Strategic Frameworks for Human Performance and Human Systems Integration: Spotlight on General Systems Performance Theory and Nonlinear Causal Resource Analysis

DTIC Science & Technology

2010-12-14

Robotics Research Institute Auditorium Riverbend Campus, Fort Worth, TX Report Documentation Page Form ApprovedOMB No. 0704-0188 Public reporting burden...for HP and HSI. Format Participants were presented with information on frameworks and the benefits they can have in support of the various roles... benefit from understanding human performance. She concluded with the objectives of the workshop: 1. Evaluate strategic frameworks for representing

Sensor fusion display evaluation using information integration models in enhanced/synthetic vision applications

NASA Technical Reports Server (NTRS)

Foyle, David C.

1993-01-01

Based on existing integration models in the psychological literature, an evaluation framework is developed to assess sensor fusion displays as might be implemented in an enhanced/synthetic vision system. The proposed evaluation framework for evaluating the operator's ability to use such systems is a normative approach: The pilot's performance with the sensor fusion image is compared to models' predictions based on the pilot's performance when viewing the original component sensor images prior to fusion. This allows for the determination as to when a sensor fusion system leads to: poorer performance than one of the original sensor displays, clearly an undesirable system in which the fused sensor system causes some distortion or interference; better performance than with either single sensor system alone, but at a sub-optimal level compared to model predictions; optimal performance compared to model predictions; or, super-optimal performance, which may occur if the operator were able to use some highly diagnostic 'emergent features' in the sensor fusion display, which were unavailable in the original sensor displays.

Challenges in Evaluating Clinical Governance Systems in Iran: A Qualitative Study

PubMed Central

Hooshmand, Elaheh; Tourani, Sogand; Ravaghi, Hamid; Ebrahimipour, Hossein

2014-01-01

Background: In spite of the pivotal role of clinical governance in enhancing quality of services provided by hospitals across the country, a scientific framework with specific criteria for evaluating hospitals has not been developed so far. Objectives: This study was conducted with the aim to identify the challenges involved in evaluating systems of clinical governance in Iran. Materials and Methods: For the purposes of this qualitative study, 15 semi-structured interviews with experts in the field were conducted in 2011 and the data were analyzed using framework analysis method. Results: Five major challenges in evaluating clinical governance include managing human resources, improving clinical quality, managing development, organizing clinical governance, and providing patient-oriented healthcare system. Conclusions: Healthcare system in Iran requires a clinical governance program which has a patient-oriented approach in philosophy, operation, and effectiveness in order to meet the challenges ahead. PMID:24910799

A portfolio evaluation framework for air transportation improvement projects

NASA Astrophysics Data System (ADS)

Baik, Hyeoncheol

This thesis explores the application of portfolio theory to the Air Transportation System (ATS) improvement. The ATS relies on complexly related resources and different stakeholder groups. Moreover, demand for air travel is significantly increasing relative to capacity of air transportation. In this environment, improving the ATS is challenging. Many projects, which are defined as technologies or initiatives, for improvement have been proposed and some have been demonstrated in practice. However, there is no clear understanding of how well these projects work in different conditions nor of how they interact with each other or with existing systems. These limitations make it difficult to develop good project combinations, or portfolios that maximize improvement. To help address this gap, a framework for identifying good portfolios is proposed. The framework can be applied to individual projects or portfolios of projects. Projects or portfolios are evaluated using four different groups of factors (effectiveness, time-to-implement, scope of applicability, and stakeholder impacts). Portfolios are also evaluated in terms of interaction-determining factors (prerequisites, co-requisites, limiting factors, and amplifying factors) because, while a given project might work well in isolation, interdependencies between projects or with existing systems could result in lower overall performance in combination. Ways to communicate a portfolio to decision makers are also introduced. The framework is unique because (1) it allows using a variety of available data, and (2) it covers diverse benefit metrics. For demonstrating the framework, an application to ground delay management projects serves as a case study. The portfolio evaluation approach introduced in this thesis can aid decision makers and researchers at universities and aviation agencies such as Federal Aviation Administration (FAA), National Aeronautics and Space Administration (NASA), and Department of Defense (DoD), in more fully evaluating the air transportation improvement projects.

A New Framework for Systematic Reviews: Application to Social Skills Interventions for Preschoolers with Autism

ERIC Educational Resources Information Center

Goldstein, Howard; Lackey, Kimberly C.; Schneider, Naomi J. B.

2014-01-01

This review presents a novel framework for evaluating evidence based on a set of parallel criteria that can be applied to both group and single-subject experimental design (SSED) studies. The authors illustrate use of this evaluation system in a systematic review of 67 articles investigating social skills interventions for preschoolers with autism…

PESTE Factors in Developing a Framework for E-Learning

ERIC Educational Resources Information Center

Graham, Deryn

2007-01-01

This article describes the development of a framework for e-learning and presents the findings of a study investigating whether the use of blended learning can fulfil or at least accommodate some of the human requirements presently neglected by current e-learning systems. The study evaluated the in-house system, Teachmat, and discussed how the use…

Organizational Evaluation Systems and Student Disengagement in Secondary Schools.

ERIC Educational Resources Information Center

Natriello, Gary

This paper reports on a study intended to examine problems in student evaluation systems that may lead to manifestations of student disengagement from secondary school such as absenteeism, low effort, violence, or vandalism. A theoretical framework is provided by Dornbusch and Scott's theory of evaluation and authority in organizations, which…

Implementation and Evaluation of Weather Responsive Traffic Estimation and Prediction System

DOT National Transportation Integrated Search

2012-06-01

The objective of the project is to develop a framework and procedures for implementing and evaluating weather-responsive traffic management (WRTM) strategies using Traffic Estimation and Prediction System (TrEPS) methodologies. In a previous FHWA-fun...

Advancing a Framework to Enable Characterization and Evaluation of Data Streams Useful for Biosurveillance

PubMed Central

Margevicius, Kristen J.; Generous, Nicholas; Taylor-McCabe, Kirsten J.; Brown, Mac; Daniel, W. Brent; Castro, Lauren; Hengartner, Andrea; Deshpande, Alina

2014-01-01

In recent years, biosurveillance has become the buzzword under which a diverse set of ideas and activities regarding detecting and mitigating biological threats are incorporated depending on context and perspective. Increasingly, biosurveillance practice has become global and interdisciplinary, requiring information and resources across public health, One Health, and biothreat domains. Even within the scope of infectious disease surveillance, multiple systems, data sources, and tools are used with varying and often unknown effectiveness. Evaluating the impact and utility of state-of-the-art biosurveillance is, in part, confounded by the complexity of the systems and the information derived from them. We present a novel approach conceptualizing biosurveillance from the perspective of the fundamental data streams that have been or could be used for biosurveillance and to systematically structure a framework that can be universally applicable for use in evaluating and understanding a wide range of biosurveillance activities. Moreover, the Biosurveillance Data Stream Framework and associated definitions are proposed as a starting point to facilitate the development of a standardized lexicon for biosurveillance and characterization of currently used and newly emerging data streams. Criteria for building the data stream framework were developed from an examination of the literature, analysis of information on operational infectious disease biosurveillance systems, and consultation with experts in the area of biosurveillance. To demonstrate utility, the framework and definitions were used as the basis for a schema of a relational database for biosurveillance resources and in the development and use of a decision support tool for data stream evaluation. PMID:24392093

A framework for telehealth program evaluation.

PubMed

Nepal, Surya; Li, Jane; Jang-Jaccard, Julian; Alem, Leila

2014-04-01

Evaluating telehealth programs is a challenging task, yet it is the most sensible first step when embarking on a telehealth study. How can we frame and report on telehealth studies? What are the health services elements to select based on the application needs? What are the appropriate terms to use to refer to such elements? Various frameworks have been proposed in the literature to answer these questions, and each framework is defined by a set of properties covering different aspects of telehealth systems. The most common properties include application, technology, and functionality. With the proliferation of telehealth, it is important not only to understand these properties, but also to define new properties to account for a wider range of context of use and evaluation outcomes. This article presents a comprehensive framework for delivery design, implementation, and evaluation of telehealth services. We first survey existing frameworks proposed in the literature and then present our proposed comprehensive multidimensional framework for telehealth. Six key dimensions of the proposed framework include health domains, health services, delivery technologies, communication infrastructure, environment setting, and socioeconomic analysis. We define a set of example properties for each dimension. We then demonstrate how we have used our framework to evaluate telehealth programs in rural and remote Australia. A few major international studies have been also mapped to demonstrate the feasibility of the framework. The key characteristics of the framework are as follows: (a) loosely coupled and hence easy to use, (b) provides a basis for describing a wide range of telehealth programs, and (c) extensible to future developments and needs.

Matching methods evaluation framework for stereoscopic breast x-ray images.

PubMed

Rousson, Johanna; Naudin, Mathieu; Marchessoux, Cédric

2016-01-01

Three-dimensional (3-D) imaging has been intensively studied in the past few decades. Depth information is an important added value of 3-D systems over two-dimensional systems. Special focuses were devoted to the development of stereo matching methods for the generation of disparity maps (i.e., depth information within a 3-D scene). Dedicated frameworks were designed to evaluate and rank the performance of different stereo matching methods but never considering x-ray medical images. Yet, 3-D x-ray acquisition systems and 3-D medical displays have already been introduced into the diagnostic market. To access the depth information within x-ray stereoscopic images, computing accurate disparity maps is essential. We aimed at developing a framework dedicated to x-ray stereoscopic breast images used to evaluate and rank several stereo matching methods. A multiresolution pyramid optimization approach was integrated to the framework to increase the accuracy and the efficiency of the stereo matching techniques. Finally, a metric was designed to score the results of the stereo matching compared with the ground truth. Eight methods were evaluated and four of them [locally scaled sum of absolute differences (LSAD), zero mean sum of absolute differences, zero mean sum of squared differences, and locally scaled mean sum of squared differences] appeared to perform equally good with an average error score of 0.04 (0 is the perfect matching). LSAD was selected for generating the disparity maps.

Uncertainty evaluation of a regional real-time system for rain-induced landslides

NASA Astrophysics Data System (ADS)

Kirschbaum, Dalia; Stanley, Thomas; Yatheendradas, Soni

2015-04-01

A new prototype regional model and evaluation framework has been developed over Central America and the Caribbean region using satellite-based information including precipitation estimates, modeled soil moisture, topography, soils, as well as regionally available datasets such as road networks and distance to fault zones. The algorithm framework incorporates three static variables: a susceptibility map; a 24-hr rainfall triggering threshold; and an antecedent soil moisture variable threshold, which have been calibrated using historic landslide events. The thresholds are regionally heterogeneous and are based on the percentile distribution of the rainfall or antecedent moisture time series. A simple decision tree algorithm framework integrates all three variables with the rainfall and soil moisture time series and generates a landslide nowcast in real-time based on the previous 24 hours over this region. This system has been evaluated using several available landslide inventories over the Central America and Caribbean region. Spatiotemporal uncertainty and evaluation metrics of the model are presented here based on available landslides reports. This work also presents a probabilistic representation of potential landslide activity over the region which can be used to further refine and improve the real-time landslide hazard assessment system as well as better identify and characterize the uncertainties inherent in this type of regional approach. The landslide algorithm provides a flexible framework to improve hazard estimation and reduce uncertainty at any spatial and temporal scale.

Framework for culturally competent decisionmaking in child welfare.

PubMed

Cohen, Elena P

2003-01-01

This article provides a framework to understand the cultural, social, political, and economic factors that affect decisionmaking when working with ethnically and racially diverse families in the child welfare system. The article describes external factors affecting the decisionmaking process, including community environment, agency structure, and family characteristics. It then reviews the core stages of the casework process, describing key decisions during intake, assessment, service planning, implementation, evaluation, and closure. Although the framework is based on casework process in the child welfare system, it can be adapted to other child-serving systems, including education, mental health, and juvenile justice.

A Conceptual Framework for Evaluation of Public Health and Primary Care System Performance in Iran

PubMed Central

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Sari, Ali Akbari; Mesdaghinia, Alireza

2015-01-01

Introduction: The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. Methods: We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. Results: We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. Conclusion: The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report. PMID:25946937

Teachers' Perceptions of Administrative Autonomy-Supportive versus Controlling Behaviors: Factors That Influence Teacher Intrinsic Motivation and Relationships with Administrators

ERIC Educational Resources Information Center

Scudella, Christine M.

2015-01-01

Federal policy around supervision and evaluation has prompted state action to develop new evaluation systems to increase student achievement. Systems, such as the Framework for Educator Effectiveness implemented in Wisconsin, are tied to teacher performance as well as student achievement. This type of performance-based evaluation system relies on…

Summary of the SWOT panel's evaluation of the organisation and financing of the Danish health care system.

PubMed

Christiansen, Terkel

2002-02-01

The organisation and financing of the Danish health care system was evaluated within a framework of a SWOT analysis (analysis of strengths, weaknesses, opportunities and threats) by a panel of five members with a background in health economics. This paper systematically summarises the panel's assessments, within the framework of the triangular model of health care. The members of the panel are in agreement on a number of aspects, while their views on other aspects differ. In general they find many strength in the way the system is organised and financed more so in the primary sector than in the hospital sector.

A Framework for Evaluating Regional-Scale Numerical Photochemical Modeling Systems

EPA Science Inventory

This paper discusses the need for critically evaluating regional-scale (~ 200-2000 km) three dimensional numerical photochemical air quality modeling systems to establish a model's credibility in simulating the spatio-temporal features embedded in the observations. Because of li...

FSM-F: Finite State Machine Based Framework for Denial of Service and Intrusion Detection in MANET.

PubMed

N Ahmed, Malik; Abdullah, Abdul Hanan; Kaiwartya, Omprakash

2016-01-01

Due to the continuous advancements in wireless communication in terms of quality of communication and affordability of the technology, the application area of Mobile Adhoc Networks (MANETs) significantly growing particularly in military and disaster management. Considering the sensitivity of the application areas, security in terms of detection of Denial of Service (DoS) and intrusion has become prime concern in research and development in the area. The security systems suggested in the past has state recognition problem where the system is not able to accurately identify the actual state of the network nodes due to the absence of clear definition of states of the nodes. In this context, this paper proposes a framework based on Finite State Machine (FSM) for denial of service and intrusion detection in MANETs. In particular, an Interruption Detection system for Adhoc On-demand Distance Vector (ID-AODV) protocol is presented based on finite state machine. The packet dropping and sequence number attacks are closely investigated and detection systems for both types of attacks are designed. The major functional modules of ID-AODV includes network monitoring system, finite state machine and attack detection model. Simulations are carried out in network simulator NS-2 to evaluate the performance of the proposed framework. A comparative evaluation of the performance is also performed with the state-of-the-art techniques: RIDAN and AODV. The performance evaluations attest the benefits of proposed framework in terms of providing better security for denial of service and intrusion detection attacks.

Effective integrated frameworks for assessing mining sustainability.

PubMed

Virgone, K M; Ramirez-Andreotta, M; Mainhagu, J; Brusseau, M L

2018-05-28

The objectives of this research are to review existing methods used for assessing mining sustainability, analyze the limited prior research that has evaluated the methods, and identify key characteristics that would constitute an enhanced sustainability framework that would serve to improve sustainability reporting in the mining industry. Five of the most relevant frameworks were selected for comparison in this analysis, and the results show that there are many commonalities among the five, as well as some disparities. In addition, relevant components are missing from all five. An enhanced evaluation system and framework were created to provide a more holistic, comprehensive method for sustainability assessment and reporting. The proposed framework has five components that build from and encompass the twelve evaluation characteristics used in the analysis. The components include Foundation, Focus, Breadth, Quality Assurance, and Relevance. The enhanced framework promotes a comprehensive, location-specific reporting approach with a concise set of well-defined indicators. Built into the framework is quality assurance, as well as a defined method to use information from sustainability reports to inform decisions. The framework incorporates human health and socioeconomic aspects via initiatives such as community-engaged research, economic valuations, and community-initiated environmental monitoring.

Integrated health monitoring and controls for rocket engines

NASA Technical Reports Server (NTRS)

Merrill, W. C.; Musgrave, J. L.; Guo, T. H.

1992-01-01

Current research in intelligent control systems at the Lewis Research Center is described in the context of a functional framework. The framework is applicable to a variety of reusable space propulsion systems for existing and future launch vehicles. It provides a 'road map' technology development to enable enhanced engine performance with increased reliability, durability, and maintainability. The framework hierarchy consists of a mission coordination level, a propulsion system coordination level, and an engine control level. Each level is described in the context of the Space Shuttle Main Engine. The concept of integrating diagnostics with control is discussed within the context of the functional framework. A distributed real time simulation testbed is used to realize and evaluate the functionalities in closed loop.

A UML-based meta-framework for system design in public health informatics.

PubMed

Orlova, Anna O; Lehmann, Harold

2002-01-01

The National Agenda for Public Health Informatics calls for standards in data and knowledge representation within public health, which requires a multi-level framework that links all aspects of public health. The literature of public health informatics and public health informatics application were reviewed. A UML-based systems analysis was performed. Face validity of results was evaluated in analyzing the public health domain of lead poisoning. The core class of the UML-based system of public health is the Public Health Domain, which is associated with multiple Problems, for which Actors provide Perspectives. Actors take Actions that define, generate, utilize and/or evaluate Data Sources. The life cycle of the domain is a sequence of activities attributed to its problems that spirals through multiple iterations and realizations within a domain. The proposed Public Health Informatics Meta-Framework broadens efforts in applying informatics principles to the field of public health

Design and Evaluation of a Scalable and Reconfigurable Multi-Platform System for Acoustic Imaging

PubMed Central

Izquierdo, Alberto; Villacorta, Juan José; del Val Puente, Lara; Suárez, Luis

2016-01-01

This paper proposes a scalable and multi-platform framework for signal acquisition and processing, which allows for the generation of acoustic images using planar arrays of MEMS (Micro-Electro-Mechanical Systems) microphones with low development and deployment costs. Acoustic characterization of MEMS sensors was performed, and the beam pattern of a module, based on an 8 × 8 planar array and of several clusters of modules, was obtained. A flexible framework, formed by an FPGA, an embedded processor, a computer desktop, and a graphic processing unit, was defined. The processing times of the algorithms used to obtain the acoustic images, including signal processing and wideband beamforming via FFT, were evaluated in each subsystem of the framework. Based on this analysis, three frameworks are proposed, defined by the specific subsystems used and the algorithms shared. Finally, a set of acoustic images obtained from sound reflected from a person are presented as a case study in the field of biometric identification. These results reveal the feasibility of the proposed system. PMID:27727174

Evaluation Framework Based on Fuzzy Measured Method in Adaptive Learning Systems

ERIC Educational Resources Information Center

Ounaies, Houda Zouari; Jamoussi, Yassine; Ben Ghezala, Henda Hajjami

2008-01-01

Currently, e-learning systems are mainly web-based applications and tackle a wide range of users all over the world. Fitting learners' needs is considered as a key issue to guaranty the success of these systems. Many researches work on providing adaptive systems. Nevertheless, evaluation of the adaptivity is still in an exploratory phase.…

[Development of a Conceptual Framework for the Assessment of Chronic Care in the Spanish National Health System].

PubMed

Espallargues, Mireia; Serra-Sutton, Vicky; Solans-Domènech, Maite; Torrente, Elena; Moharra, Montse; Benítez, Dolors; Robles, Noemí; Domíngo, Laia; Escarrabill Sanglas, Joan

2016-07-07

The aim was to develop a conceptual framework for the assessment of new healthcare initiatives on chronic diseases within the Spanish National Health System. A comprehensive literature review between 2002 and 2013, including systematic reviews, meta-analysis, and reports with evaluation frameworks and/or assessment of initiatives was carried out; integrated care initiatives established in Catalonia were studied and described; and semistructured interviews with key stakeholders were performed. The scope and conceptual framework were defined by using the brainstorming approach.Of 910 abstracts identified, a total of 116 studies were included. They referred to several conceptual frameworks and/or assessment indicators at a national and international level. An overall of 24 established chronic care initiatives were identified (9 integrated care initiatives); 10 in-depth interviews were carried out. The proposed conceptual framework envisages: 1)the target population according to complexity levels; 2)an evaluation approach of the structure, processes, and outcomes considering the health status achieved, the recovery process and the maintenance of health; and 3)the dimensions or attributes to be assessed. The proposed conceptual framework will be helpful has been useful to develop indicators and implement them with a community-based and result-oriented approach and a territorial or population-based perspective within the Spanish Health System. This will be essential to know which are the most effective strategies, what are the key elements that determine greater success and what are the groups of patients who can most benefit.

L.E.A.D.: A Framework for Evidence Gathering and Use for the Prevention of Obesity and Other Complex Public Health Problems

ERIC Educational Resources Information Center

Chatterji, Madhabi; Green, Lawrence W.; Kumanyika, Shiriki

2014-01-01

This article summarizes a comprehensive, systems-oriented framework designed to improve the use of a wide variety of evidence sources to address population-wide obesity problems. The L.E.A.D. framework (for "Locate" the evidence, "Evaluate" the evidence, "Assemble" the evidence, and inform "Decisions"),…

Towards a Framework for Evaluating and Comparing Diagnosis Algorithms

NASA Technical Reports Server (NTRS)

Kurtoglu, Tolga; Narasimhan, Sriram; Poll, Scott; Garcia,David; Kuhn, Lukas; deKleer, Johan; vanGemund, Arjan; Feldman, Alexander

2009-01-01

Diagnostic inference involves the detection of anomalous system behavior and the identification of its cause, possibly down to a failed unit or to a parameter of a failed unit. Traditional approaches to solving this problem include expert/rule-based, model-based, and data-driven methods. Each approach (and various techniques within each approach) use different representations of the knowledge required to perform the diagnosis. The sensor data is expected to be combined with these internal representations to produce the diagnosis result. In spite of the availability of various diagnosis technologies, there have been only minimal efforts to develop a standardized software framework to run, evaluate, and compare different diagnosis technologies on the same system. This paper presents a framework that defines a standardized representation of the system knowledge, the sensor data, and the form of the diagnosis results and provides a run-time architecture that can execute diagnosis algorithms, send sensor data to the algorithms at appropriate time steps from a variety of sources (including the actual physical system), and collect resulting diagnoses. We also define a set of metrics that can be used to evaluate and compare the performance of the algorithms, and provide software to calculate the metrics.

[The application and evaluation of the system theory for community health assessment in a rural village].

PubMed

Wu, Guey-Hau; Liou, Yiing-Mei; Huang, Lian-Hua

2004-10-01

In assessing the health of a community is important to select tools appropriate to the community's characteristics. The framework for this paper is the system framework for community assessment developed by Trotter, Smith and Maurer (2000); the data were collected by windshield survey, literature review, interview, and observation. Through data analysis and the identification of the community's problem, the authors prioritize those problems in accordance with Goeppinger and Schuste's (1992) criteria. They illustrate the practicality and local applicability of this method by means of a local case. Finally, the authors evaluate the framework in terms of concept clearance, variable classification, and indicator measurement. In addition, they propose concrete suggestions for community workers to consider in the selection of assessment tools, and to enrich nursing knowledge.

A 3D Human-Machine Integrated Design and Analysis Framework for Squat Exercises with a Smith Machine.

PubMed

Lee, Haerin; Jung, Moonki; Lee, Ki-Kwang; Lee, Sang Hun

2017-02-06

In this paper, we propose a three-dimensional design and evaluation framework and process based on a probabilistic-based motion synthesis algorithm and biomechanical analysis system for the design of the Smith machine and squat training programs. Moreover, we implemented a prototype system to validate the proposed framework. The framework consists of an integrated human-machine-environment model as well as a squat motion synthesis system and biomechanical analysis system. In the design and evaluation process, we created an integrated model in which interactions between a human body and machine or the ground are modeled as joints with constraints at contact points. Next, we generated Smith squat motion using the motion synthesis program based on a Gaussian process regression algorithm with a set of given values for independent variables. Then, using the biomechanical analysis system, we simulated joint moments and muscle activities from the input of the integrated model and squat motion. We validated the model and algorithm through physical experiments measuring the electromyography (EMG) signals, ground forces, and squat motions as well as through a biomechanical simulation of muscle forces. The proposed approach enables the incorporation of biomechanics in the design process and reduces the need for physical experiments and prototypes in the development of training programs and new Smith machines.

A 3D Human-Machine Integrated Design and Analysis Framework for Squat Exercises with a Smith Machine

PubMed Central

Lee, Haerin; Jung, Moonki; Lee, Ki-Kwang; Lee, Sang Hun

2017-01-01

In this paper, we propose a three-dimensional design and evaluation framework and process based on a probabilistic-based motion synthesis algorithm and biomechanical analysis system for the design of the Smith machine and squat training programs. Moreover, we implemented a prototype system to validate the proposed framework. The framework consists of an integrated human–machine–environment model as well as a squat motion synthesis system and biomechanical analysis system. In the design and evaluation process, we created an integrated model in which interactions between a human body and machine or the ground are modeled as joints with constraints at contact points. Next, we generated Smith squat motion using the motion synthesis program based on a Gaussian process regression algorithm with a set of given values for independent variables. Then, using the biomechanical analysis system, we simulated joint moments and muscle activities from the input of the integrated model and squat motion. We validated the model and algorithm through physical experiments measuring the electromyography (EMG) signals, ground forces, and squat motions as well as through a biomechanical simulation of muscle forces. The proposed approach enables the incorporation of biomechanics in the design process and reduces the need for physical experiments and prototypes in the development of training programs and new Smith machines. PMID:28178184

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ehlen, Mark Andrew; Vugrin, Eric D.; Warren, Drake E.

In recent years, the nation has recognized that critical infrastructure protection should consider not only the prevention of disruptive events, but also the processes that infrastructure systems undergo to maintain functionality following disruptions. This more comprehensive approach has been termed critical infrastructure resilience (CIR). Given the occurrence of a particular disruptive event, the resilience of a system to that event is the system's ability to efficiently reduce both the magnitude and duration of the deviation from targeted system performance levels. Sandia National Laboratories (Sandia) has developed a comprehensive resilience assessment framework for evaluating the resilience of infrastructure and economic systems.more » The framework includes a quantitative methodology that measures resilience costs that result from a disruption to infrastructure function. The framework also includes a qualitative analysis methodology that assesses system characteristics that affect resilience in order to provide insight and direction for potential improvements to resilience. This paper describes the resilience assessment framework. This paper further demonstrates the utility of the assessment framework through application to a hypothetical scenario involving the disruption of a petrochemical supply chain by a hurricane.« less

A person-centered integrated care quality framework, based on a qualitative study of patients' evaluation of care in light of chronic care ideals.

PubMed

Berntsen, Gro; Høyem, Audhild; Lettrem, Idar; Ruland, Cornelia; Rumpsfeld, Markus; Gammon, Deede

2018-06-20

Person-Centered Integrated Care (PC-IC) is believed to improve outcomes and experience for persons with multiple long-term and complex conditions. No broad consensus exists regarding how to capture the patient-experienced quality of PC-IC. Most PC-IC evaluation tools focus on care events or care in general. Building on others' and our previous work, we outlined a 4-stage goal-oriented PC-IC process ideal: 1) Personalized goal setting 2) Care planning aligned with goals 3) Care delivery according to plan, and 4) Evaluation of goal attainment. We aimed to explore, apply, refine and operationalize this quality of care framework. This paper is a qualitative evaluative review of the individual Patient Pathways (iPP) experiences of 19 strategically chosen persons with multimorbidity in light of ideals for chronic care. The iPP includes all care events, addressing the persons collected health issues, organized by time. We constructed iPPs based on the electronic health record (from general practice, nursing services, and hospital) with patient follow-up interviews. The application of the framework and its refinement were parallel processes. Both were based on analysis of salient themes in the empirical material in light of the PC-IC process ideal and progressively more informed applications of themes and questions. The informants consistently reviewed care quality by how care supported/ threatened their long-term goals. Personal goals were either implicit or identified by "What matters to you?" Informants expected care to address their long-term goals and placed responsibility for care quality and delivery at the system level. The PC-IC process framework exposed system failure in identifying long-term goals, provision of shared long-term multimorbidity care plans, monitoring of care delivery and goal evaluation. The PC-IC framework includes descriptions of ideal care, key questions and literature references for each stage of the PC-IC process. This first version of a PC-IC process framework needs further validation in other settings. Gaps in care that are invisible with event-based quality of care frameworks become apparent when evaluated by a long-term goal-driven PC-IC process framework. The framework appears meaningful to persons with multimorbidity.

New Evaluation System Wins Approval

ERIC Educational Resources Information Center

Barrett, Laura

2011-01-01

All Massachusetts school districts will have to adopt new evaluation systems based on a state framework that was approved by the Board of Elementary and Secondary Education on June 28. The new system will be phased in over three years, beginning with Level 4 schools--those designated "underperforming" by the state--in the 2011-12 school…

Advanced error diagnostics of the CMAQ and Chimere modelling systems within the AQMEII3 model evaluation framework

EPA Science Inventory

The work here complements the overview analysis of the modelling systems participating in the third phase of the Air Quality Model Evaluation International Initiative (AQMEII3) by focusing on the performance for hourly surface ozone by two modelling systems, Chimere for Europe an...

Autonomic Management of Application Workflows on Hybrid Computing Infrastructure

DOE PAGES

Kim, Hyunjoo; el-Khamra, Yaakoub; Rodero, Ivan; ...

2011-01-01

In this paper, we present a programming and runtime framework that enables the autonomic management of complex application workflows on hybrid computing infrastructures. The framework is designed to address system and application heterogeneity and dynamics to ensure that application objectives and constraints are satisfied. The need for such autonomic system and application management is becoming critical as computing infrastructures become increasingly heterogeneous, integrating different classes of resources from high-end HPC systems to commodity clusters and clouds. For example, the framework presented in this paper can be used to provision the appropriate mix of resources based on application requirements and constraints.more » The framework also monitors the system/application state and adapts the application and/or resources to respond to changing requirements or environment. To demonstrate the operation of the framework and to evaluate its ability, we employ a workflow used to characterize an oil reservoir executing on a hybrid infrastructure composed of TeraGrid nodes and Amazon EC2 instances of various types. Specifically, we show how different applications objectives such as acceleration, conservation and resilience can be effectively achieved while satisfying deadline and budget constraints, using an appropriate mix of dynamically provisioned resources. Our evaluations also demonstrate that public clouds can be used to complement and reinforce the scheduling and usage of traditional high performance computing infrastructure.« less

Human factors evaluation of level 2 and level 3 automated driving concepts : concepts of operation.

DOT National Transportation Integrated Search

2014-07-01

The Concepts of Operation document evaluates the functional framework of operations for Level 2 and Level 3 automated vehicle systems. This is done by defining the varying levels of automation, the operator vehicle interactions, and system components...

A methodological framework for the evaluation of syndromic surveillance systems: a case study of England.

PubMed

Colón-González, Felipe J; Lake, Iain R; Morbey, Roger A; Elliot, Alex J; Pebody, Richard; Smith, Gillian E

2018-04-24

Syndromic surveillance complements traditional public health surveillance by collecting and analysing health indicators in near real time. The rationale of syndromic surveillance is that it may detect health threats faster than traditional surveillance systems permitting more timely, and hence potentially more effective public health action. The effectiveness of syndromic surveillance largely relies on the methods used to detect aberrations. Very few studies have evaluated the performance of syndromic surveillance systems and consequently little is known about the types of events that such systems can and cannot detect. We introduce a framework for the evaluation of syndromic surveillance systems that can be used in any setting based upon the use of simulated scenarios. For a range of scenarios this allows the time and probability of detection to be determined and uncertainty is fully incorporated. In addition, we demonstrate how such a framework can model the benefits of increases in the number of centres reporting syndromic data and also determine the minimum size of outbreaks that can or cannot be detected. Here, we demonstrate its utility using simulations of national influenza outbreaks and localised outbreaks of cryptosporidiosis. Influenza outbreaks are consistently detected with larger outbreaks being detected in a more timely manner. Small cryptosporidiosis outbreaks (<1000 symptomatic individuals) are unlikely to be detected. We also demonstrate the advantages of having multiple syndromic data streams (e.g. emergency attendance data, telephone helpline data, general practice consultation data) as different streams are able to detect different outbreak types with different efficacy (e.g. emergency attendance data are useful for the detection of pandemic influenza but not for outbreaks of cryptosporidiosis). We also highlight that for any one disease, the utility of data streams may vary geographically, and that the detection ability of syndromic surveillance varies seasonally (e.g. an influenza outbreak starting in July is detected sooner than one starting later in the year). We argue that our framework constitutes a useful tool for public health emergency preparedness in multiple settings. The proposed framework allows the exhaustive evaluation of any syndromic surveillance system and constitutes a useful tool for emergency preparedness and response.

Cyber Security Research Frameworks For Coevolutionary Network Defense

DOE Office of Scientific and Technical Information (OSTI.GOV)

Rush, George D.; Tauritz, Daniel Remy

Several architectures have been created for developing and testing systems used in network security, but most are meant to provide a platform for running cyber security experiments as opposed to automating experiment processes. In the first paper, we propose a framework termed Distributed Cyber Security Automation Framework for Experiments (DCAFE) that enables experiment automation and control in a distributed environment. Predictive analysis of adversaries is another thorny issue in cyber security. Game theory can be used to mathematically analyze adversary models, but its scalability limitations restrict its use. Computational game theory allows us to scale classical game theory to larger,more » more complex systems. In the second paper, we propose a framework termed Coevolutionary Agent-based Network Defense Lightweight Event System (CANDLES) that can coevolve attacker and defender agent strategies and capabilities and evaluate potential solutions with a custom network defense simulation. The third paper is a continuation of the CANDLES project in which we rewrote key parts of the framework. Attackers and defenders have been redesigned to evolve pure strategy, and a new network security simulation is devised which specifies network architecture and adds a temporal aspect. We also add a hill climber algorithm to evaluate the search space and justify the use of a coevolutionary algorithm.« less

Visual Representations of Microcosm in Textbooks of Chemistry: Constructing a Systemic Network for Their Main Conceptual Framework

ERIC Educational Resources Information Center

Papageorgiou, George; Amariotakis, Vasilios; Spiliotopoulou, Vasiliki

2017-01-01

The main objective of this work is to analyse the visual representations (VRs) of the microcosm depicted in nine Greek secondary chemistry school textbooks of the last three decades in order to construct a systemic network for their main conceptual framework and to evaluate the contribution of each one of the resulting categories to the network.…

Feedback (F) Fueling Adaptation (A) Network Growth (N) and Self-Organization (S): A Complex Systems Design and Evaluation Approach to Professional Development

ERIC Educational Resources Information Center

Yoon, Susan A.; Klopfer, Eric

2006-01-01

This paper reports on the efficacy of a professional development framework premised on four complex systems design principles: Feedback, Adaptation, Network Growth and Self-organization (FANS). The framework is applied to the design and delivery of the first 2 years of a 3-year study aimed at improving teacher and student understanding of…

A Comprehensive Guide to C3 System Development

DTIC Science & Technology

1990-03-01

This thesis provides guidelines to develop a C3 system, including both organizational and physical systems. It contains the concept, architecture ... design and engineering approaches, the integrated C3 framework, test and evaluation methodologies, system acquisition procedures, system development

Design of the HELICS High-Performance Transmission-Distribution-Communication-Market Co-Simulation Framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Palmintier, Bryan S; Krishnamurthy, Dheepak; Top, Philip

This paper describes the design rationale for a new cyber-physical-energy co-simulation framework for electric power systems. This new framework will support very large-scale (100,000+ federates) co-simulations with off-the-shelf power-systems, communication, and end-use models. Other key features include cross-platform operating system support, integration of both event-driven (e.g. packetized communication) and time-series (e.g. power flow) simulation, and the ability to co-iterate among federates to ensure model convergence at each time step. After describing requirements, we begin by evaluating existing co-simulation frameworks, including HLA and FMI, and conclude that none provide the required features. Then we describe the design for the new layeredmore » co-simulation architecture.« less

Design of the HELICS High-Performance Transmission-Distribution-Communication-Market Co-Simulation Framework: Preprint

DOE Office of Scientific and Technical Information (OSTI.GOV)

Palmintier, Bryan S; Krishnamurthy, Dheepak; Top, Philip

This paper describes the design rationale for a new cyber-physical-energy co-simulation framework for electric power systems. This new framework will support very large-scale (100,000+ federates) co-simulations with off-the-shelf power-systems, communication, and end-use models. Other key features include cross-platform operating system support, integration of both event-driven (e.g. packetized communication) and time-series (e.g. power flow) simulation, and the ability to co-iterate among federates to ensure model convergence at each time step. After describing requirements, we begin by evaluating existing co-simulation frameworks, including HLA and FMI, and conclude that none provide the required features. Then we describe the design for the new layeredmore » co-simulation architecture.« less

Comparing the Performance of Indoor Localization Systems through the EvAAL Framework.

PubMed

Potortì, Francesco; Park, Sangjoon; Jiménez Ruiz, Antonio Ramón; Barsocchi, Paolo; Girolami, Michele; Crivello, Antonino; Lee, So Yeon; Lim, Jae Hyun; Torres-Sospedra, Joaquín; Seco, Fernando; Montoliu, Raul; Mendoza-Silva, Germán Martin; Pérez Rubio, Maria Del Carmen; Losada-Gutiérrez, Cristina; Espinosa, Felipe; Macias-Guarasa, Javier

2017-10-13

In recent years, indoor localization systems have been the object of significant research activity and of growing interest for their great expected social impact and their impressive business potential. Application areas include tracking and navigation, activity monitoring, personalized advertising, Active and Assisted Living (AAL), traceability, Internet of Things (IoT) networks, and Home-land Security. In spite of the numerous research advances and the great industrial interest, no canned solutions have yet been defined. The diversity and heterogeneity of applications, scenarios, sensor and user requirements, make it difficult to create uniform solutions. From that diverse reality, a main problem is derived that consists in the lack of a consensus both in terms of the metrics and the procedures used to measure the performance of the different indoor localization and navigation proposals. This paper introduces the general lines of the EvAAL benchmarking framework, which is aimed at a fair comparison of indoor positioning systems through a challenging competition under complex, realistic conditions. To evaluate the framework capabilities, we show how it was used in the 2016 Indoor Positioning and Indoor Navigation (IPIN) Competition. The 2016 IPIN competition considered three different scenario dimensions, with a variety of use cases: (1) pedestrian versus robotic navigation, (2) smartphones versus custom hardware usage and (3) real-time positioning versus off-line post-processing. A total of four competition tracks were evaluated under the same EvAAL benchmark framework in order to validate its potential to become a standard for evaluating indoor localization solutions. The experience gained during the competition and feedback from track organizers and competitors showed that the EvAAL framework is flexible enough to successfully fit the very different tracks and appears adequate to compare indoor positioning systems.

Comparing the Performance of Indoor Localization Systems through the EvAAL Framework

PubMed Central

2017-01-01

In recent years, indoor localization systems have been the object of significant research activity and of growing interest for their great expected social impact and their impressive business potential. Application areas include tracking and navigation, activity monitoring, personalized advertising, Active and Assisted Living (AAL), traceability, Internet of Things (IoT) networks, and Home-land Security. In spite of the numerous research advances and the great industrial interest, no canned solutions have yet been defined. The diversity and heterogeneity of applications, scenarios, sensor and user requirements, make it difficult to create uniform solutions. From that diverse reality, a main problem is derived that consists in the lack of a consensus both in terms of the metrics and the procedures used to measure the performance of the different indoor localization and navigation proposals. This paper introduces the general lines of the EvAAL benchmarking framework, which is aimed at a fair comparison of indoor positioning systems through a challenging competition under complex, realistic conditions. To evaluate the framework capabilities, we show how it was used in the 2016 Indoor Positioning and Indoor Navigation (IPIN) Competition. The 2016 IPIN competition considered three different scenario dimensions, with a variety of use cases: (1) pedestrian versus robotic navigation, (2) smartphones versus custom hardware usage and (3) real-time positioning versus off-line post-processing. A total of four competition tracks were evaluated under the same EvAAL benchmark framework in order to validate its potential to become a standard for evaluating indoor localization solutions. The experience gained during the competition and feedback from track organizers and competitors showed that the EvAAL framework is flexible enough to successfully fit the very different tracks and appears adequate to compare indoor positioning systems. PMID:29027948

Framework for the Intelligent Transportation System (ITS) Evaluation : ITS Integration Activities

DOT National Transportation Integrated Search

2006-08-01

Intelligent Transportation Systems (ITS) represent a significant opportunity to improve the efficiency and safety of the surface transportation system. ITS includes technologies to support information processing, communications, surveillance and cont...

Policy reconciliation for access control in dynamic cross-enterprise collaborations

NASA Astrophysics Data System (ADS)

Preuveneers, D.; Joosen, W.; Ilie-Zudor, E.

2018-03-01

In dynamic cross-enterprise collaborations, different enterprises form a - possibly temporary - business relationship. To integrate their business processes, enterprises may need to grant each other limited access to their information systems. Authentication and authorization are key to secure information handling. However, access control policies often rely on non-standardized attributes to describe the roles and permissions of their employees which convolutes cross-organizational authorization when business relationships evolve quickly. Our framework addresses the managerial overhead of continuous updates to access control policies for enterprise information systems to accommodate disparate attribute usage. By inferring attribute relationships, our framework facilitates attribute and policy reconciliation, and automatically aligns dynamic entitlements during the evaluation of authorization decisions. We validate our framework with a Industry 4.0 motivating scenario on networked production where such dynamic cross-enterprise collaborations are quintessential. The evaluation reveals the capabilities and performance of our framework, and illustrates the feasibility of liberating the security administrator from manually provisioning and aligning attributes, and verifying the consistency of access control policies for cross-enterprise collaborations.

Prioritizing Training To Maximize Results: The 3 Box System.

ERIC Educational Resources Information Center

Kearns, Paul

2003-01-01

Considers fundamentals of effective training and focuses on the evaluation of training. Describes the 3 Box System, which provides a framework for discussing: (1) basic training needs and priorities; (2) added value training, including ROI (return on investment); evaluation; and (3) prioritizing training budgets. (LRW)

FSM-F: Finite State Machine Based Framework for Denial of Service and Intrusion Detection in MANET

PubMed Central

N. Ahmed, Malik; Abdullah, Abdul Hanan; Kaiwartya, Omprakash

2016-01-01

Due to the continuous advancements in wireless communication in terms of quality of communication and affordability of the technology, the application area of Mobile Adhoc Networks (MANETs) significantly growing particularly in military and disaster management. Considering the sensitivity of the application areas, security in terms of detection of Denial of Service (DoS) and intrusion has become prime concern in research and development in the area. The security systems suggested in the past has state recognition problem where the system is not able to accurately identify the actual state of the network nodes due to the absence of clear definition of states of the nodes. In this context, this paper proposes a framework based on Finite State Machine (FSM) for denial of service and intrusion detection in MANETs. In particular, an Interruption Detection system for Adhoc On-demand Distance Vector (ID-AODV) protocol is presented based on finite state machine. The packet dropping and sequence number attacks are closely investigated and detection systems for both types of attacks are designed. The major functional modules of ID-AODV includes network monitoring system, finite state machine and attack detection model. Simulations are carried out in network simulator NS-2 to evaluate the performance of the proposed framework. A comparative evaluation of the performance is also performed with the state-of-the-art techniques: RIDAN and AODV. The performance evaluations attest the benefits of proposed framework in terms of providing better security for denial of service and intrusion detection attacks. PMID:27285146

A framework for m-health service development and success evaluation.

PubMed

Sadegh, S Saeedeh; Khakshour Saadat, Parisa; Sepehri, Mohammad Mehdi; Assadi, Vahid

2018-04-01

The emergence of mobile technology has influenced many service industries including health care. Mobile health (m-Health) applications have been used widely, and many services have been developed that have changed delivery systems and have improved effectiveness of health care services. Stakeholders of m-Health services have various resources and rights that lends to a complexity in service delivery. In addition, abundance of different m-Health services makes it difficult to choose an appropriate service for these stakeholders that include customers, patients, users or even providers. Moreover, a comprehensive framework is not yet provided in the literature that would help manage and evaluate m-health services, considering various stakeholder's benefits. In this paper, a comprehensive literature review has been done on famous frameworks and models in the field of Information Technology and electronic health with the aim of finding different aspects of developing and managing m-health services. Using the results of literature review and conducting a stakeholder analysis, we have proposed an m-health evaluation framework which evaluates the success of a given m-health service through a three-stage life cycle: (1) Service Requirement Analysis, (2) Service Development, and (3) Service Delivery. Key factors of m-health evaluation in each step are introduced in the proposed framework considering m-health key stakeholder's benefits. The proposed framework is validated via expert interviews, and key factors in each evaluation step is validated using PLS model. Results show that path coefficients are higher than their threshold which supports the validity of proposed framework. Copyright © 2018 Elsevier B.V. All rights reserved.

ATR evaluation through the synthesis of multiple performance measures

NASA Astrophysics Data System (ADS)

Bassham, Christopher B.; Klimack, William K.; Bauer, Kenneth W., Jr.

2002-07-01

This research demonstrates the application of decision analysis (DA) techniques to decisions made within Automatic Target Recognition (ATR) technology development. This work is accomplished to improve the means by which ATR technologies are evaluated. The first step in this research was to create a flexible decision analysis framework that could be applied to several decisions across different ATR programs evaluated by the Comprehensive ATR Scientific Evaluation (COMPASE) Center of the Air Force Research Laboratory (AFRL). For the purposes of this research, a single COMPASE Center representative provided the value, utility, and preference functions for the DA framework. The DA framework employs performance measures collected during ATR classification system (CS) testing to calculate value and utility scores. The authors gathered data from the Moving and Stationary Target Acquisition and Recognition (MSTAR) program to demonstrate how the decision framework could be used to evaluate three different ATR CSs. A decision-maker may use the resultant scores to gain insight into any of the decisions that occur throughout the lifecycle of ATR technologies. Additionally, a means of evaluating ATR CS self-assessment ability is presented. This represents a new criterion that emerged from this study, and no present evaluation metric is known.

A unified framework for evaluating the risk of re-identification of text de-identification tools.

PubMed

Scaiano, Martin; Middleton, Grant; Arbuckle, Luk; Kolhatkar, Varada; Peyton, Liam; Dowling, Moira; Gipson, Debbie S; El Emam, Khaled

2016-10-01

It has become regular practice to de-identify unstructured medical text for use in research using automatic methods, the goal of which is to remove patient identifying information to minimize re-identification risk. The metrics commonly used to determine if these systems are performing well do not accurately reflect the risk of a patient being re-identified. We therefore developed a framework for measuring the risk of re-identification associated with textual data releases. We apply the proposed evaluation framework to a data set from the University of Michigan Medical School. Our risk assessment results are then compared with those that would be obtained using a typical contemporary micro-average evaluation of recall in order to illustrate the difference between the proposed evaluation framework and the current baseline method. We demonstrate how this framework compares against common measures of the re-identification risk associated with an automated text de-identification process. For the probability of re-identification using our evaluation framework we obtained a mean value for direct identifiers of 0.0074 and a mean value for quasi-identifiers of 0.0022. The 95% confidence interval for these estimates were below the relevant thresholds. The threshold for direct identifier risk was based on previously used approaches in the literature. The threshold for quasi-identifiers was determined based on the context of the data release following commonly used de-identification criteria for structured data. Our framework attempts to correct for poorly distributed evaluation corpora, accounts for the data release context, and avoids the often optimistic assumptions that are made using the more traditional evaluation approach. It therefore provides a more realistic estimate of the true probability of re-identification. This framework should be used as a basis for computing re-identification risk in order to more realistically evaluate future text de-identification tools. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

The Development and Initial Evaluation of the Human Readiness Level Framework

DTIC Science & Technology

2010-06-01

View ICD Initial Capabilities Document ICW Interactive Course Ware ILE Interactive Learning Environment ILT Instructor Led Training IOC...Programmatic Environmental Safety and Health Evaluation PHA Preliminary Hazard Analysis PHL Preliminary Hazard List xiv PM Program Manager PQS...Occupational Health SOW Statement of Work SRD System Requirements Document SPS System Performance Specification SRR System Requirements Review SVR

Evaluating Monitoring Systems in the European Social Fund Context: A Sociotechnical Approach

ERIC Educational Resources Information Center

Iannacci, Federico; Cornford, Tony; Cordella, Antonio; Grillo, Francesco

2009-01-01

In contrast to the prevailing image of monitoring systems as technical systems, it is proposed that they should rather be conceived of as social endeavors at exchanging information. Drawing on the monitoring and evaluation framework of Cornford, Doukidis, and Forster, the concept of information agreement is suggested as a way of assessing the…

DRUG EVALUATION AND DECISION MAKING IN CATALONIA: DEVELOPMENT AND VALIDATION OF A METHODOLOGICAL FRAMEWORK BASED ON MULTI-CRITERIA DECISION ANALYSIS (MCDA) FOR ORPHAN DRUGS.

PubMed

Gilabert-Perramon, Antoni; Torrent-Farnell, Josep; Catalan, Arancha; Prat, Alba; Fontanet, Manel; Puig-Peiró, Ruth; Merino-Montero, Sandra; Khoury, Hanane; Goetghebeur, Mireille M; Badia, Xavier

2017-01-01

The aim of this study was to adapt and assess the value of a Multi-Criteria Decision Analysis (MCDA) framework (EVIDEM) for the evaluation of Orphan drugs in Catalonia (Catalan Health Service). The standard evaluation and decision-making procedures of CatSalut were compared with the EVIDEM methodology and contents. The EVIDEM framework was adapted to the Catalan context, focusing on the evaluation of Orphan drugs (PASFTAC program), during a Workshop with sixteen PASFTAC members. The criteria weighting was done using two different techniques (nonhierarchical and hierarchical). Reliability was assessed by re-test. The EVIDEM framework and methodology was found useful and feasible for Orphan drugs evaluation and decision making in Catalonia. All the criteria considered for the development of the CatSalut Technical Reports and decision making were considered in the framework. Nevertheless, the framework could improve the reporting of some of these criteria (i.e., "unmet needs" or "nonmedical costs"). Some Contextual criteria were removed (i.e., "Mandate and scope of healthcare system", "Environmental impact") or adapted ("population priorities and access") for CatSalut purposes. Independently of the weighting technique considered, the most important evaluation criteria identified for orphan drugs were: "disease severity", "unmet needs" and "comparative effectiveness", while the "size of the population" had the lowest relevance for decision making. Test-retest analysis showed weight consistency among techniques, supporting reliability overtime. MCDA (EVIDEM framework) could be a useful tool to complement the current evaluation methods of CatSalut, contributing to standardization and pragmatism, providing a method to tackle ethical dilemmas and facilitating discussions related to decision making.

ATIS evaluation framework

DOT National Transportation Integrated Search

2005-04-01

This report documents the results of a series of ITS project evaluations performed for the Washington State Department of Transportation (WSDOT). Five Washington state advanced traveler information system (ATIS) projects selected for federal ITS fund...

Robustness Analysis of Integrated LPV-FDI Filters and LTI-FTC System for a Transport Aircraft

NASA Technical Reports Server (NTRS)

Khong, Thuan H.; Shin, Jong-Yeob

2007-01-01

This paper proposes an analysis framework for robustness analysis of a nonlinear dynamics system that can be represented by a polynomial linear parameter varying (PLPV) system with constant bounded uncertainty. The proposed analysis framework contains three key tools: 1) a function substitution method which can convert a nonlinear system in polynomial form into a PLPV system, 2) a matrix-based linear fractional transformation (LFT) modeling approach, which can convert a PLPV system into an LFT system with the delta block that includes key uncertainty and scheduling parameters, 3) micro-analysis, which is a well known robust analysis tool for linear systems. The proposed analysis framework is applied to evaluating the performance of the LPV-fault detection and isolation (FDI) filters of the closed-loop system of a transport aircraft in the presence of unmodeled actuator dynamics and sensor gain uncertainty. The robustness analysis results are compared with nonlinear time simulations.

An Analysis of State and Local Alignment of Teacher Evaluation in Maryland

ERIC Educational Resources Information Center

Peterson, Serene N.

2014-01-01

This study explored the components of Maryland's newly-implemented teacher evaluation framework and compared state requirements with evaluations to three local school systems' evaluation procedures. The study sought to investigate the relationship between three evaluation protocols in comparison to the state requirements. Three local school…

A Proposed Collaborative Framework for Prefabricated Housing Construction Using RFID Technology

NASA Astrophysics Data System (ADS)

Charnwasununth, Phatsaphan; Yabuki, Nobuyoshi; Tongthong, Tanit

Despite the popularity of prefabricated housing construction in Thailand and many other countries, due to the lack of collaboration in current practice, undesired low productivity and a number of mistakes are identified. This research proposes a framework to raise the collaborative level for improving productivity and reducing mistake occurrences at sites. In this framework, RFID system bridges the gap between the real situation and the design, and the proposed system can cope with the unexpected construction conditions by generating proper alternatives. This system is composed of PDAs, RFID readers, laptop PCs, and a desktop PC. Six main modules and a database system are implemented in laptop PCs for recording actual site conditions, generating working alternatives, providing related information, and evaluating the work.

Laboratory Evaluation of Dynamic Traffic Assignment Systems: Requirements, Framework, and System Design

DOT National Transportation Integrated Search

1997-01-01

The success of Advanced Traveler Information Systems (ATIS) and Advanced Traffic Management Systems (ATMS) depends on the availability and dissemination of timely and accurate estimates of current and emerging traffic network conditions. Real-time Dy...

Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda.

PubMed

Hotchkiss, David R; Aqil, Anwer; Lippeveld, Theo; Mukooyo, Edward

2010-07-03

Sound policy, resource allocation and day-to-day management decisions in the health sector require timely information from routine health information systems (RHIS). In most low- and middle-income countries, the RHIS is viewed as being inadequate in providing quality data and continuous information that can be used to help improve health system performance. In addition, there is limited evidence on the effectiveness of RHIS strengthening interventions in improving data quality and use. The purpose of this study is to evaluate the usefulness of the newly developed Performance of Routine Information System Management (PRISM) framework, which consists of a conceptual framework and associated data collection and analysis tools to assess, design, strengthen and evaluate RHIS. The specific objectives of the study are: a) to assess the reliability and validity of the PRISM instruments and b) to assess the validity of the PRISM conceptual framework. Facility- and worker-level data were collected from 110 health care facilities in twelve districts in Uganda in 2004 and 2007 using records reviews, structured interviews and self-administered questionnaires. The analysis procedures include Cronbach's alpha to assess internal consistency of selected instruments, test-retest analysis to assess the reliability and sensitivity of the instruments, and bivariate and multivariate statistical techniques to assess validity of the PRISM instruments and conceptual framework. Cronbach's alpha analysis suggests high reliability (0.7 or greater) for the indices measuring a promotion of a culture of information, RHIS tasks self-efficacy and motivation. The study results also suggest that a promotion of a culture of information influences RHIS tasks self-efficacy, RHIS tasks competence and motivation, and that self-efficacy and the presence of RHIS staff have a direct influence on the use of RHIS information, a key aspect of RHIS performance. The study results provide some empirical support for the reliability and validity of the PRISM instruments and the validity of the PRISM conceptual framework, suggesting that the PRISM approach can be effectively used by RHIS policy makers and practitioners to assess the RHIS and evaluate RHIS strengthening interventions. However, additional studies with larger sample sizes are needed to further investigate the value of the PRISM instruments in exploring the linkages between RHIS data quality and use, and health systems performance.

A simulation framework for the CMS Track Trigger electronics

NASA Astrophysics Data System (ADS)

Amstutz, C.; Magazzù, G.; Weber, M.; Palla, F.

2015-03-01

A simulation framework has been developed to test and characterize algorithms, architectures and hardware implementations of the vastly complex CMS Track Trigger for the high luminosity upgrade of the CMS experiment at the Large Hadron Collider in Geneva. High-level SystemC models of all system components have been developed to simulate a portion of the track trigger. The simulation of the system components together with input data from physics simulations allows evaluating figures of merit, like delays or bandwidths, under realistic conditions. The use of SystemC for high-level modelling allows co-simulation with models developed in Hardware Description Languages, e.g. VHDL or Verilog. Therefore, the simulation framework can also be used as a test bench for digital modules developed for the final system.

Forest resource information system

NASA Technical Reports Server (NTRS)

Mroczynski, R. P. (Principal Investigator)

1978-01-01

The author has identified the following significant results. A benchmark classification evaluation framework was implemented. The FRIS preprocessing activities were refined. Potential geo-based referencing systems were identified as components of FRIS.

Observing System Simulation Experiments for Fun and Profit

NASA Technical Reports Server (NTRS)

Prive, Nikki C.

2015-01-01

Observing System Simulation Experiments can be powerful tools for evaluating and exploring both the behavior of data assimilation systems and the potential impacts of future observing systems. With great power comes great responsibility - given a pure modeling framework, how can we be sure our results are meaningful? The challenges and pitfalls of OSSE calibration and validation will be addressed, as well as issues of incestuousness, selection of appropriate metrics, and experiment design. The use of idealized observational networks to investigate theoretical ideas in a fully complex modeling framework will also be discussed

An Intrusion Detection System Based on Multi-Level Clustering for Hierarchical Wireless Sensor Networks

PubMed Central

Butun, Ismail; Ra, In-Ho; Sankar, Ravi

2015-01-01

In this work, an intrusion detection system (IDS) framework based on multi-level clustering for hierarchical wireless sensor networks is proposed. The framework employs two types of intrusion detection approaches: (1) “downward-IDS (D-IDS)” to detect the abnormal behavior (intrusion) of the subordinate (member) nodes; and (2) “upward-IDS (U-IDS)” to detect the abnormal behavior of the cluster heads. By using analytical calculations, the optimum parameters for the D-IDS (number of maximum hops) and U-IDS (monitoring group size) of the framework are evaluated and presented. PMID:26593915

Overcoming complexities: Damage detection using dictionary learning framework

NASA Astrophysics Data System (ADS)

Alguri, K. Supreet; Melville, Joseph; Deemer, Chris; Harley, Joel B.

2018-04-01

For in situ damage detection, guided wave structural health monitoring systems have been widely researched due to their ability to evaluate large areas and their ability detect many types of damage. These systems often evaluate structural health by recording initial baseline measurements from a pristine (i.e., undamaged) test structure and then comparing later measurements with that baseline. Yet, it is not always feasible to have a pristine baseline. As an alternative, substituting the baseline with data from a surrogate (nearly identical and pristine) structure is a logical option. While effective in some circumstance, surrogate data is often still a poor substitute for pristine baseline measurements due to minor differences between the structures. To overcome this challenge, we present a dictionary learning framework to adapt surrogate baseline data to better represent an undamaged test structure. We compare the performance of our framework with two other surrogate-based damage detection strategies: (1) using raw surrogate data for comparison and (2) using sparse wavenumber analysis, a precursor to our framework for improving the surrogate data. We apply our framework to guided wave data from two 108 mm by 108 mm aluminum plates. With 20 measurements, we show that our dictionary learning framework achieves a 98% accuracy, raw surrogate data achieves a 92% accuracy, and sparse wavenumber analysis achieves a 57% accuracy.

Evaluation Capacity in K-12 School Counseling Programs.

ERIC Educational Resources Information Center

Trevisan, Michael S.

2002-01-01

Presents results of a literature review, organized the Milstein and Cotton evaluation capacity (B. Milstein and D. Cotton, 2000) framework, detailing and analyzing the contextual factors and system features that impact the evaluation capacity of school counseling programs. Discusses implications for evaluation in other settings. (SLD)

Performance Assessment of Communicable Disease Surveillance in Disasters: A Systematic Review

PubMed Central

Babaie, Javad; Ardalan, Ali; Vatandoost, Hasan; Goya, Mohammad Mehdi; Akbarisari, Ali

2015-01-01

Background: This study aimed to identify the indices and frameworks that have been used to assess the performance of communicable disease surveillance (CDS) in response to disasters and other emergencies, including infectious disease outbreaks. Method: In this systematic review, PubMed, Google Scholar, Scopus, ScienceDirect, ProQuest databases and grey literature were searched until the end of 2013. All retrieved titles were examined in accordance with inclusion criteria. Abstracts of the relevant titles were reviewed and eligible abstracts were included in a list for data abstraction. Finally, the study variables were extracted. Results: Sixteen articles and one book were found relevant to our study objectives. In these articles, 31 criteria and 35 indicators were used or suggested for the assessment/evaluation of the performance of surveillance systems in disasters. The Centers for Disease Control (CDC) updated guidelines for the evaluation of public health surveillance systems were the most widely used. Conclusion: Despite the importance of performance assessment in improving CDS in response to disasters, there is a lack of clear and accepted frameworks. There is also no agreement on the use of existing criteria and indices. The only relevant framework is the CDC guideline, which is a common framework for assessing public health surveillance systems as a whole. There is an urgent need to develop appropriate frameworks, criteria, and indices for specifically assessing the performance of CDS in response to disasters and other emergencies, including infectious diseases outbreaks. Key words: Disasters, Emergencies, Communicable Diseases, Surveillance System, Performance Assessment PMID:25774323

A quantitative evaluation of a qualitative risk assessment framework: Examining the assumptions and predictions of the Productivity Susceptibility Analysis (PSA)

PubMed Central

2018-01-01

Qualitative risk assessment frameworks, such as the Productivity Susceptibility Analysis (PSA), have been developed to rapidly evaluate the risks of fishing to marine populations and prioritize management and research among species. Despite being applied to over 1,000 fish populations, and an ongoing debate about the most appropriate method to convert biological and fishery characteristics into an overall measure of risk, the assumptions and predictive capacity of these approaches have not been evaluated. Several interpretations of the PSA were mapped to a conventional age-structured fisheries dynamics model to evaluate the performance of the approach under a range of assumptions regarding exploitation rates and measures of biological risk. The results demonstrate that the underlying assumptions of these qualitative risk-based approaches are inappropriate, and the expected performance is poor for a wide range of conditions. The information required to score a fishery using a PSA-type approach is comparable to that required to populate an operating model and evaluating the population dynamics within a simulation framework. In addition to providing a more credible characterization of complex system dynamics, the operating model approach is transparent, reproducible and can evaluate alternative management strategies over a range of plausible hypotheses for the system. PMID:29856869

Applying the System Component and Operationally Relevant Evaluation (SCORE) Framework to Evaluate Advanced Military Technologies

DTIC Science & Technology

2010-03-01

and charac- terize the actions taken by the soldier (e.g., running, walking, climbing stairs ). Real-time image capture and exchange N The ability of...multimedia information sharing among soldiers in the field, two-way speech translation systems, and autonomous robotic platforms. Key words: Emerging...soldiers in the field, two-way speech translation systems, and autonomous robotic platforms. It has been the foundation for 10 technology evaluations

Using Intervention Mapping to Develop an Oral Health e-Curriculum for Secondary Prevention of Eating Disorders.

PubMed

DeBate, Rita D; Bleck, Jennifer R; Raven, Jessica; Severson, Herb

2017-06-01

Preventing oral-systemic health issues relies on evidence-based interventions across various system-level target groups. Although the use of theory- and evidence-based approaches has been encouraged in developing oral health behavior change programs, the translation of theoretical constructs and principles to behavior change interventions has not been well described. Based on a series of six systematic steps, Intervention Mapping provides a framework for effective decision making with regard to developing, implementing, and evaluating theory- and evidence-informed, system-based behavior change programs. This article describes the application of the Intervention Mapping framework to develop the EAT (evaluating, assessing, and treating) evidence-based intervention with the goal of increasing the capacity of oral health providers to engage in secondary prevention of oral-systemic issues associated with disordered eating behaviors. Examples of data and deliverables for each step are described. In addition, results from evaluation of the intervention via randomized control trial are described, with statistically significant differences observed in behavioral outcomes in the intervention group with effect sizes ranging from r=0.62 to 0.83. These results suggest that intervention mapping, via the six systematic steps, can be useful as a framework for continued development of preventive interventions.

76 FR 37620 - Risk-Based Capital Standards: Advanced Capital Adequacy Framework-Basel II; Establishment of a...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-06-28

... systems. E. Quantitative Methods for Comparing Capital Frameworks The NPR sought comment on how the... industry while assessing levels of capital. This commenter points out maintaining reliable comparative data over time could make quantitative methods for this purpose difficult. For example, evaluating asset...

A common evaluation framework for the African Health Initiative.

PubMed

Bryce, Jennifer; Requejo, Jennifer Harris; Moulton, Lawrence H; Ram, Malathi; Black, Robert E

2013-01-01

The African Health Initiative includes highly diverse partnerships in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia), each of which is working to improve population health by strengthening health systems and to evaluate the results. One aim of the Initiative is to generate cross-site learning that can inform implementation in the five partnerships during the project period and identify lessons that may be generalizable to other countries in the region. Collaborators in the Initiative developed a common evaluation framework as a basis for this cross-site learning. This paper describes the components of the framework; this includes the conceptual model, core metrics to be measured in all sites, and standard guidelines for reporting on the implementation of partnership activities and contextual factors that may affect implementation, or the results it produces. We also describe the systems that have been put in place for data management, data quality assessments, and cross-site analysis of results. The conceptual model for the Initiative highlights points in the causal chain between health system strengthening activities and health impact where evidence produced by the partnerships can contribute to learning. This model represents an important advance over its predecessors by including contextual factors and implementation strength as potential determinants, and explicitly including equity as a component of both outcomes and impact. Specific measurement challenges include the prospective documentation of program implementation and contextual factors. Methodological issues addressed in the development of the framework include the aggregation of data collected using different methods and the challenge of evaluating a complex set of interventions being improved over time based on continuous monitoring and intermediate results.

Has computational creativity successfully made it "Beyond the Fence" in musical theatre?

NASA Astrophysics Data System (ADS)

Jordanous, Anna

2017-10-01

A significant test for software is to task it with replicating human performance, as done recently with creative software and the commercial project Beyond the Fence (undertaken for a television documentary Computer Says Show). The remit of this project was to use computer software as much as possible to produce "the world's first computer-generated musical". Several creative systems were used to generate this musical, which was performed in London's West End in 2016. This paper considers the challenge of evaluating this project. Current computational creativity evaluation methods are ill-suited to evaluating projects that involve creative input from multiple systems and people. Following recent inspiration within computational creativity research from interaction design, here the DECIDE evaluation framework is applied to evaluate the Beyond the Fence project. Evaluation finds that the project was reasonably successful at achieving the task of using computational generation to produce a credible musical. Lessons have been learned for future computational creativity projects though, particularly for affording creative software more agency and enabling software to interact with other creative partners. Upon reflection, the DECIDE framework emerges as a useful evaluation "checklist" (if not a tangible operational methodology) for evaluating multiple creative systems participating in a creative task.

Translation from UML to Markov Model: A Performance Modeling Framework

NASA Astrophysics Data System (ADS)

Khan, Razib Hayat; Heegaard, Poul E.

Performance engineering focuses on the quantitative investigation of the behavior of a system during the early phase of the system development life cycle. Bearing this on mind, we delineate a performance modeling framework of the application for communication system that proposes a translation process from high level UML notation to Continuous Time Markov Chain model (CTMC) and solves the model for relevant performance metrics. The framework utilizes UML collaborations, activity diagrams and deployment diagrams to be used for generating performance model for a communication system. The system dynamics will be captured by UML collaboration and activity diagram as reusable specification building blocks, while deployment diagram highlights the components of the system. The collaboration and activity show how reusable building blocks in the form of collaboration can compose together the service components through input and output pin by highlighting the behavior of the components and later a mapping between collaboration and system component identified by deployment diagram will be delineated. Moreover the UML models are annotated to associate performance related quality of service (QoS) information which is necessary for solving the performance model for relevant performance metrics through our proposed framework. The applicability of our proposed performance modeling framework in performance evaluation is delineated in the context of modeling a communication system.

Models of care for musculoskeletal health: a cross-sectional qualitative study of Australian stakeholders' perspectives on relevance and standardised evaluation.

PubMed

Briggs, Andrew M; Jordan, Joanne E; Speerin, Robyn; Jennings, Matthew; Bragge, Peter; Chua, Jason; Slater, Helen

2015-11-16

The prevalence and impact of musculoskeletal conditions are predicted to rapidly escalate in the coming decades. Effective strategies are required to minimise 'evidence-practice', 'burden-policy' and 'burden-service' gaps and optimise health system responsiveness for sustainable, best-practice healthcare. One mechanism by which evidence can be translated into practice and policy is through Models of Care (MoCs), which provide a blueprint for health services planning and delivery. While evidence supports the effectiveness of musculoskeletal MoCs for improving health outcomes and system efficiencies, no standardised national approach to evaluation in terms of their 'readiness' for implementation and 'success' after implementation, is yet available. Further, the value assigned to MoCs by end users is uncertain. This qualitative study aimed to explore end users' views on the relevance of musculoskeletal MoCs to their work and value of a standardised evaluation approach. A cross-sectional qualitative study was undertaken. Subject matter experts (SMEs) with health, policy and administration and consumer backgrounds were drawn from three Australian states. A semi-structured interview schedule was developed and piloted to explore perceptions about musculoskeletal MoCs including: i) aspects important to their work (or life, for consumers) ii) usefulness of standardised evaluation frameworks to judge 'readiness' and 'success' and iii) challenges associated with standardised evaluation. Verbatim transcripts were analysed by two researchers using a grounded theory approach to derive key themes. Twenty-seven SMEs (n = 19; 70.4 % female) including five (18.5 %) consumers participated in the study. MoCs were perceived as critical for influencing and initiating changes to best-practice healthcare planning and delivery and providing practical guidance on how to implement and evaluate services. A 'readiness' evaluation framework assessing whether critical components across the health system had been considered prior to implementation was strongly supported, while 'success' was perceived as an already familiar evaluation concept. Perceived challenges associated with standardised evaluation included identifying, defining and measuring key 'readiness' and 'success' indicators; impacts of systems and context changes; cost; meaningful stakeholder consultation and developing a widely applicable framework. A standardised evaluation framework that includes a strong focus on 'readiness' is important to ensure successful and sustainable implementation of musculoskeletal MoCs.

Enhancing the Resilience of Interdependent Critical Infrastructure Systems Using a Common Computational Framework

NASA Astrophysics Data System (ADS)

Little, J. C.; Filz, G. M.

2016-12-01

As modern societies become more complex, critical interdependent infrastructure systems become more likely to fail under stress unless they are designed and implemented to be resilient. Hurricane Katrina clearly demonstrated the catastrophic and as yet unpredictable consequences of such failures. Resilient infrastructure systems maintain the flow of goods and services in the face of a broad range of natural and manmade hazards. In this presentation, we illustrate a generic computational framework to facilitate high-level decision-making about how to invest scarce resources most effectively to enhance resilience in coastal protection, transportation, and the economy of a region. Coastal Louisiana, our study area, has experienced the catastrophic effects of several land-falling hurricanes in recent years. In this project, we implement and further refine three process models (a coastal protection model, a transportation model, and an economic model) for the coastal Louisiana region. We upscale essential mechanistic features of the three detailed process models to the systems level and integrate the three reduced-order systems models in a modular fashion. We also evaluate the proposed approach in annual workshops with input from stakeholders. Based on stakeholder inputs, we derive a suite of goals, targets, and indicators for evaluating resilience at the systems level, and assess and enhance resilience using several deterministic scenarios. The unifying framework will be able to accommodate the different spatial and temporal scales that are appropriate for each model. We combine our generic computational framework, which encompasses the entire system of systems, with the targets, and indicators needed to systematically meet our chosen resilience goals. We will start with targets that focus on technical and economic systems, but future work will ensure that targets and indicators are extended to other dimensions of resilience including those in the environmental and social systems. The overall model can be used to optimize decision making in a probabilistic risk-based framework.

Systems-based accident analysis in the led outdoor activity domain: application and evaluation of a risk management framework.

PubMed

Salmon, P; Williamson, A; Lenné, M; Mitsopoulos-Rubens, E; Rudin-Brown, C M

2010-08-01

Safety-compromising accidents occur regularly in the led outdoor activity domain. Formal accident analysis is an accepted means of understanding such events and improving safety. Despite this, there remains no universally accepted framework for collecting and analysing accident data in the led outdoor activity domain. This article presents an application of Rasmussen's risk management framework to the analysis of the Lyme Bay sea canoeing incident. This involved the development of an Accimap, the outputs of which were used to evaluate seven predictions made by the framework. The Accimap output was also compared to an analysis using an existing model from the led outdoor activity domain. In conclusion, the Accimap output was found to be more comprehensive and supported all seven of the risk management framework's predictions, suggesting that it shows promise as a theoretically underpinned approach for analysing, and learning from, accidents in the led outdoor activity domain. STATEMENT OF RELEVANCE: Accidents represent a significant problem within the led outdoor activity domain. This article presents an evaluation of a risk management framework that can be used to understand such accidents and to inform the development of accident countermeasures and mitigation strategies for the led outdoor activity domain.

The behaviour change wheel: a new method for characterising and designing behaviour change interventions.

PubMed

Michie, Susan; van Stralen, Maartje M; West, Robert

2011-04-23

Improving the design and implementation of evidence-based practice depends on successful behaviour change interventions. This requires an appropriate method for characterising interventions and linking them to an analysis of the targeted behaviour. There exists a plethora of frameworks of behaviour change interventions, but it is not clear how well they serve this purpose. This paper evaluates these frameworks, and develops and evaluates a new framework aimed at overcoming their limitations. A systematic search of electronic databases and consultation with behaviour change experts were used to identify frameworks of behaviour change interventions. These were evaluated according to three criteria: comprehensiveness, coherence, and a clear link to an overarching model of behaviour. A new framework was developed to meet these criteria. The reliability with which it could be applied was examined in two domains of behaviour change: tobacco control and obesity. Nineteen frameworks were identified covering nine intervention functions and seven policy categories that could enable those interventions. None of the frameworks reviewed covered the full range of intervention functions or policies, and only a minority met the criteria of coherence or linkage to a model of behaviour. At the centre of a proposed new framework is a 'behaviour system' involving three essential conditions: capability, opportunity, and motivation (what we term the 'COM-B system'). This forms the hub of a 'behaviour change wheel' (BCW) around which are positioned the nine intervention functions aimed at addressing deficits in one or more of these conditions; around this are placed seven categories of policy that could enable those interventions to occur. The BCW was used reliably to characterise interventions within the English Department of Health's 2010 tobacco control strategy and the National Institute of Health and Clinical Excellence's guidance on reducing obesity. Interventions and policies to change behaviour can be usefully characterised by means of a BCW comprising: a 'behaviour system' at the hub, encircled by intervention functions and then by policy categories. Research is needed to establish how far the BCW can lead to more efficient design of effective interventions.

The behaviour change wheel: A new method for characterising and designing behaviour change interventions

PubMed Central

2011-01-01

Background Improving the design and implementation of evidence-based practice depends on successful behaviour change interventions. This requires an appropriate method for characterising interventions and linking them to an analysis of the targeted behaviour. There exists a plethora of frameworks of behaviour change interventions, but it is not clear how well they serve this purpose. This paper evaluates these frameworks, and develops and evaluates a new framework aimed at overcoming their limitations. Methods A systematic search of electronic databases and consultation with behaviour change experts were used to identify frameworks of behaviour change interventions. These were evaluated according to three criteria: comprehensiveness, coherence, and a clear link to an overarching model of behaviour. A new framework was developed to meet these criteria. The reliability with which it could be applied was examined in two domains of behaviour change: tobacco control and obesity. Results Nineteen frameworks were identified covering nine intervention functions and seven policy categories that could enable those interventions. None of the frameworks reviewed covered the full range of intervention functions or policies, and only a minority met the criteria of coherence or linkage to a model of behaviour. At the centre of a proposed new framework is a 'behaviour system' involving three essential conditions: capability, opportunity, and motivation (what we term the 'COM-B system'). This forms the hub of a 'behaviour change wheel' (BCW) around which are positioned the nine intervention functions aimed at addressing deficits in one or more of these conditions; around this are placed seven categories of policy that could enable those interventions to occur. The BCW was used reliably to characterise interventions within the English Department of Health's 2010 tobacco control strategy and the National Institute of Health and Clinical Excellence's guidance on reducing obesity. Conclusions Interventions and policies to change behaviour can be usefully characterised by means of a BCW comprising: a 'behaviour system' at the hub, encircled by intervention functions and then by policy categories. Research is needed to establish how far the BCW can lead to more efficient design of effective interventions. PMID:21513547

Assessing the Spread and Uptake of a Framework for Introducing and Evaluating Advanced Practice Nursing Roles.

PubMed

Boyko, Jennifer A; Carter, Nancy; Bryant-Lukosius, Denise

2016-08-01

Health system researchers must ensure that the products of their work meet the needs of various stakeholder groups (e.g., patients, practitioners, and policy makers). Evidence-based frameworks can support the uptake and spread of research evidence; however, their existence as knowledge translation tools does not ensure their uptake and it is difficult to ascertain their spread into research, practice, and policy using existing methods. The purpose of this article is to report results of a study on the spread and uptake of an evidence-based framework (i.e., the participatory, evidence-based, patient-focused process for advanced practice nursing [PEPPA] framework) into research, practice, and policies relevant to the introduction and evaluation of advanced practice nursing roles. We also reflect on the utility of using a modified citation methodology to evaluate knowledge translation efforts. We searched four databases for literature published between 2004 and 2014 citing the original paper in which the PEPPA framework was published, and carried out an Internet search for grey literature using keywords. Relevant data were extracted from sources and organized using NVivo software. We analysed results descriptively. Our search yielded 164 unique sources of which 69.5% were from published literature and the majority (83.4%) of these were published in nursing journals. Most frequently (71.5%), the framework was used by researchers and students in research studies. A smaller number of citations (11.3%) reflected use of the PEPPA framework in practice settings with a focus on role development, implementation, evaluation, or a combination of these. This study demonstrates that the PEPPA framework has been used to varying degrees as intended, and provides guidance on how to evaluate the spread and uptake of research outputs (e.g., theoretical frameworks). Further research is needed about ways to determine whether evidence-informed research tools such as frameworks have been taken up successfully into practice and policy contexts. © 2016 Sigma Theta Tau International.

Addressing Enterprise-Level Information System Deficiencies

DTIC Science & Technology

2015-03-26

goals and effectiveness. This case study evaluates the Depot Source of Repair (DSOR) team and how it has addressed the USAF’s enterprise- level IS...deficiencies. A framework created from the literature review is used to evaluate the DSOR team’s IS called DSOR II. The case study evaluation ...7 IS Design Evaluation

Teacher Evaluation To Enhance Professional Practice.

ERIC Educational Resources Information Center

Danielson, Charlotte; McGreal, Thomas L.

This book shows how a school district's local teacher evaluation committee can design evaluation systems in which educators can achieve the dual purposes of accountability and professional development and even merge these purposes. A structural framework for designing the evaluation is proposed that locates teachers in one of three tracks: the…

Evaluation 101: How One Department Embraced the Process

ERIC Educational Resources Information Center

Gothard, Katina; Gorham, Jayne

2011-01-01

An evaluation system was designed and implemented by a faculty support department to measure the value, identify gaps, and improve the quality of their training at a midsize community college. The Targeted Evaluation Process (Combs & Falletta, 2000) was selected as the evaluation framework because of its applicability to training and nontraining…

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dale, Virginia H.; Efroymson, Rebecca Ann; Kline, Keith L.

A framework for selecting and evaluating indicators of bioenergy sustainability is presented. This framework is designed to facilitate decision-making about which indicators are useful for assessing sustainability of bioenergy systems and supporting their deployment. Efforts to develop sustainability indicators in the United States and Europe are reviewed. The first steps of the framework for indicator selection are defining the sustainability goals and other goals for a bioenergy project or program, gaining an understanding of the context, and identifying the values of stakeholders. From the goals, context, and stakeholders, the objectives for analysis and criteria for indicator selection can be developed.more » The user of the framework identifies and ranks indicators, applies them in an assessment, and then evaluates their effectiveness, while identifying gaps that prevent goals from being met, assessing lessons learned, and moving toward best practices. The framework approach emphasizes that the selection of appropriate criteria and indicators is driven by the specific purpose of an analysis. Realistic goals and measures of bioenergy sustainability can be developed systematically with the help of the framework presented here.« less

A framework for designing and analyzing binary decision-making strategies in cellular systems†

PubMed Central

Porter, Joshua R.; Andrews, Burton W.; Iglesias, Pablo A.

2015-01-01

Cells make many binary (all-or-nothing) decisions based on noisy signals gathered from their environment and processed through noisy decision-making pathways. Reducing the effect of noise to improve the fidelity of decision-making comes at the expense of increased complexity, creating a tradeoff between performance and metabolic cost. We present a framework based on rate distortion theory, a branch of information theory, to quantify this tradeoff and design binary decision-making strategies that balance low cost and accuracy in optimal ways. With this framework, we show that several observed behaviors of binary decision-making systems, including random strategies, hysteresis, and irreversibility, are optimal in an information-theoretic sense for various situations. This framework can also be used to quantify the goals around which a decision-making system is optimized and to evaluate the optimality of cellular decision-making systems by a fundamental information-theoretic criterion. As proof of concept, we use the framework to quantify the goals of the externally triggered apoptosis pathway. PMID:22370552

Applying air pollution modelling within a multi-criteria decision analysis framework to evaluate UK air quality policies

NASA Astrophysics Data System (ADS)

Chalabi, Zaid; Milojevic, Ai; Doherty, Ruth M.; Stevenson, David S.; MacKenzie, Ian A.; Milner, James; Vieno, Massimo; Williams, Martin; Wilkinson, Paul

2017-10-01

A decision support system for evaluating UK air quality policies is presented. It combines the output from a chemistry transport model, a health impact model and other impact models within a multi-criteria decision analysis (MCDA) framework. As a proof-of-concept, the MCDA framework is used to evaluate and compare idealized emission reduction policies in four sectors (combustion in energy and transformation industries, non-industrial combustion plants, road transport and agriculture) and across six outcomes or criteria (mortality, health inequality, greenhouse gas emissions, biodiversity, crop yield and air quality legal compliance). To illustrate a realistic use of the MCDA framework, the relative importance of the criteria were elicited from a number of stakeholders acting as proxy policy makers. In the prototype decision problem, we show that reducing emissions from industrial combustion (followed very closely by road transport and agriculture) is more advantageous than equivalent reductions from the other sectors when all the criteria are taken into account. Extensions of the MCDA framework to support policy makers in practice are discussed.

An Exploration of Teachers' and Administrators' Perspectives: The Collaborative Process Using the Danielson Framework for Teaching Model

ERIC Educational Resources Information Center

Landolfi, Adrienne M.

2016-01-01

As accountability measures continue to increase within education, public school systems have integrated standards-based evaluation systems to formally assess professional practices among educators. The purpose of this study was to explore the extent in which the communication process between evaluators and teachers impacts teacher performance…

Distributed decision-making in electric power system transmission maintenance scheduling using multi-agent systems (MAS)

NASA Astrophysics Data System (ADS)

Zhang, Zhong

In this work, motivated by the need to coordinate transmission maintenance scheduling among a multiplicity of self-interested entities in restructured power industry, a distributed decision support framework based on multiagent negotiation systems (MANS) is developed. An innovative risk-based transmission maintenance optimization procedure is introduced. Several models for linking condition monitoring information to the equipment's instantaneous failure probability are presented, which enable quantitative evaluation of the effectiveness of maintenance activities in terms of system cumulative risk reduction. Methodologies of statistical processing, equipment deterioration evaluation and time-dependent failure probability calculation are also described. A novel framework capable of facilitating distributed decision-making through multiagent negotiation is developed. A multiagent negotiation model is developed and illustrated that accounts for uncertainty and enables social rationality. Some issues of multiagent negotiation convergence and scalability are discussed. The relationships between agent-based negotiation and auction systems are also identified. A four-step MAS design methodology for constructing multiagent systems for power system applications is presented. A generic multiagent negotiation system, capable of inter-agent communication and distributed decision support through inter-agent negotiations, is implemented. A multiagent system framework for facilitating the automated integration of condition monitoring information and maintenance scheduling for power transformers is developed. Simulations of multiagent negotiation-based maintenance scheduling among several independent utilities are provided. It is shown to be a viable alternative solution paradigm to the traditional centralized optimization approach in today's deregulated environment. This multiagent system framework not only facilitates the decision-making among competing power system entities, but also provides a tool to use in studying competitive industry relative to monopolistic industry.

Microgravity isolation system design: A modern control analysis framework

NASA Technical Reports Server (NTRS)

Hampton, R. D.; Knospe, C. R.; Allaire, P. E.; Grodsinsky, C. M.

1994-01-01

Many acceleration-sensitive, microgravity science experiments will require active vibration isolation from the manned orbiters on which they will be mounted. The isolation problem, especially in the case of a tethered payload, is a complex three-dimensional one that is best suited to modern-control design methods. These methods, although more powerful than their classical counterparts, can nonetheless go only so far in meeting the design requirements for practical systems. Once a tentative controller design is available, it must still be evaluated to determine whether or not it is fully acceptable, and to compare it with other possible design candidates. Realistically, such evaluation will be an inherent part of a necessary iterative design process. In this paper, an approach is presented for applying complex mu-analysis methods to a closed-loop vibration isolation system (experiment plus controller). An analysis framework is presented for evaluating nominal stability, nominal performance, robust stability, and robust performance of active microgravity isolation systems, with emphasis on the effective use of mu-analysis methods.

Framework for quantitative evaluation of 3D vessel segmentation approaches using vascular phantoms in conjunction with 3D landmark localization and registration

NASA Astrophysics Data System (ADS)

Wörz, Stefan; Hoegen, Philipp; Liao, Wei; Müller-Eschner, Matthias; Kauczor, Hans-Ulrich; von Tengg-Kobligk, Hendrik; Rohr, Karl

2016-03-01

We introduce a framework for quantitative evaluation of 3D vessel segmentation approaches using vascular phantoms. Phantoms are designed using a CAD system and created with a 3D printer, and comprise realistic shapes including branches and pathologies such as abdominal aortic aneurysms (AAA). To transfer ground truth information to the 3D image coordinate system, we use a landmark-based registration scheme utilizing fiducial markers integrated in the phantom design. For accurate 3D localization of the markers we developed a novel 3D parametric intensity model that is directly fitted to the markers in the images. We also performed a quantitative evaluation of different vessel segmentation approaches for a phantom of an AAA.

A framework for selecting indicators of bioenergy sustainability

DOE PAGES

Dale, Virginia H.; Efroymson, Rebecca Ann; Kline, Keith L.; ...

2015-05-11

A framework for selecting and evaluating indicators of bioenergy sustainability is presented. This framework is designed to facilitate decision-making about which indicators are useful for assessing sustainability of bioenergy systems and supporting their deployment. Efforts to develop sustainability indicators in the United States and Europe are reviewed. The first steps of the framework for indicator selection are defining the sustainability goals and other goals for a bioenergy project or program, gaining an understanding of the context, and identifying the values of stakeholders. From the goals, context, and stakeholders, the objectives for analysis and criteria for indicator selection can be developed.more » The user of the framework identifies and ranks indicators, applies them in an assessment, and then evaluates their effectiveness, while identifying gaps that prevent goals from being met, assessing lessons learned, and moving toward best practices. The framework approach emphasizes that the selection of appropriate criteria and indicators is driven by the specific purpose of an analysis. Realistic goals and measures of bioenergy sustainability can be developed systematically with the help of the framework presented here.« less

A Web-Based Common Framework to Support the Test and Evaluation Process Any Time, Anywhere, and Anyhow

DOE Office of Scientific and Technical Information (OSTI.GOV)

Schur, Anne; Brown, James C.; Eaton, Sharon L.

Test and evaluation (T and E) is an enterprise. For any product, large or small, performance data is desired on many aspects to evaluate the product?s effectiveness for the intended users. Representing the many T and E facets without bewildering the user is challenging when there is a range of people, from the system developers to the manager of the organization, that want specific feedback. A web-based One-Stop Evaluation Center was created to meet these needs for a particular project. The evaluation center is usable at any time in the systems development lifecycle and streamlines the T and E enterprise.more » This paper discusses a common framework that unifies the T and E process with many stakeholders involved and is flexible to accommodate each stakeholders?specific evaluative processes and content. Our success has translated to many cost savings by enabling quick responses to change and a better line of communication between the users, developers, and managers.« less

Remote sensing of multimodal transportation systems : research brief.

DOT National Transportation Integrated Search

2016-09-01

Remote Sensing of Multimodal Transportation Systems : Rapid condition monitoring and performance evaluations of the vast and vulnerable transportation infrastructure has been elusive. The framework and models developed in this research will enable th...

Participatory Research in Systems of Care for Children’s Mental Health

PubMed Central

Pullmann, Michael D.

2010-01-01

The children’s system of care initiative in the United States requires the participation of caregivers of children with emotional or behavioral problems in conducting research and evaluation. This entails a restructuring of traditional power dynamics among families served by the community mental health system and other system stakeholders, including researchers. However, evidence indicates that system of care research may not currently embrace the different types of knowledge possessed by caregivers and may be frustrated by traditional power hierarchies, resulting in research findings that are not useful for the community. In this paper I examine a framework for power and knowledge and examine how, when viewed through this framework, participatory research in the system of care initiative thus far may be less than authentic. I conclude with improvements suggested by the framework that are expected to shift power to caregivers and result in more useful, actionable research findings for the community. PMID:19533331

Units of Distinction: Creating a Blueprint for Recognition of High-Performing Medical-Surgical Nursing Units.

PubMed

Jeffery, Alvin D; Mosier, Sammie; Baker, Allison; Korwek, Kimberly; Borum, Cindy; Englebright, Jane

2018-02-01

Hospital medical-surgical (M/S) nursing units are responsible for up to 28 million encounters annually, yet receive little attention from professional organizations and national initiatives targeted to improve quality and performance. We sought to develop a framework recognizing high-performing units within our large hospital system. This was a retrospective data analysis of M/S units throughout a 168-hospital system. Measures represented patient experience, employee engagement, staff scheduling, nursing-sensitive patient outcomes, professional practices, and clinical process measures. Four hundred ninety units from 129 hospitals contributed information to test the framework. A manual scoring system identified the top 5% and recognized them as a "Unit of Distinction." Secondary analyses with machine learning provided validation of the proposed framework. Similar to external recognition programs, this framework and process provide a holistic evaluation useful for meaningful recognition and lay the groundwork for benchmarking in improvement efforts.

MushyPeek: A Framework for Online Investigation of Audiovisual Dialogue Phenomena

ERIC Educational Resources Information Center

Edlund, Jens; Beskow, Jonas

2009-01-01

Evaluation of methods and techniques for conversational and multimodal spoken dialogue systems is complex, as is gathering data for the modeling and tuning of such techniques. This article describes MushyPeek, an experiment framework that allows us to manipulate the audiovisual behavior of interlocutors in a setting similar to face-to-face…

Designing for Discovery Learning of Complexity Principles of Congestion by Driving Together in the TrafficJams Simulation

ERIC Educational Resources Information Center

Levy, Sharona T.; Peleg, Ran; Ofeck, Eyal; Tabor, Naamit; Dubovi, Ilana; Bluestein, Shiri; Ben-Zur, Hadar

2018-01-01

We propose and evaluate a framework supporting collaborative discovery learning of complex systems. The framework blends five design principles: (1) individual action: amidst (2) social interactions; challenged with (3) multiple tasks; set in (4) a constrained interactive learning environment that draws attention to (5) highlighted target…

An Information Technology Framework for Strengthening Telehealthcare Service Delivery

PubMed Central

Chen, Chi-Wen; Weng, Yung-Ching; Shang, Rung-Ji; Yu, Hui-Chu; Chung, Yufang; Lai, Feipei

2012-01-01

Abstract Objective: Telehealthcare has been used to provide healthcare service, and information technology infrastructure appears to be essential while providing telehealthcare service. Insufficiencies have been identified, such as lack of integration, need of accommodation of diverse biometric sensors, and accessing diverse networks as different houses have varying facilities, which challenge the promotion of telehealthcare. This study designs an information technology framework to strengthen telehealthcare delivery. Materials and Methods: The proposed framework consists of a system architecture design and a network transmission design. The aim of the framework is to integrate data from existing information systems, to adopt medical informatics standards, to integrate diverse biometric sensors, and to provide different data transmission networks to support a patient's house network despite the facilities. The proposed framework has been evaluated with a case study of two telehealthcare programs, with and without the adoption of the framework. Results: The proposed framework facilitates the functionality of the program and enables steady patient enrollments. The overall patient participations are increased, and the patient outcomes appear positive. The attitudes toward the service and self-improvement also are positive. Conclusions: The findings of this study add up to the construction of a telehealthcare system. Implementing the proposed framework further assists the functionality of the service and enhances the availability of the service and patient acceptances. PMID:23061641

An information technology framework for strengthening telehealthcare service delivery.

PubMed

Chen, Li-Chin; Chen, Chi-Wen; Weng, Yung-Ching; Shang, Rung-Ji; Yu, Hui-Chu; Chung, Yufang; Lai, Feipei

2012-10-01

Telehealthcare has been used to provide healthcare service, and information technology infrastructure appears to be essential while providing telehealthcare service. Insufficiencies have been identified, such as lack of integration, need of accommodation of diverse biometric sensors, and accessing diverse networks as different houses have varying facilities, which challenge the promotion of telehealthcare. This study designs an information technology framework to strengthen telehealthcare delivery. The proposed framework consists of a system architecture design and a network transmission design. The aim of the framework is to integrate data from existing information systems, to adopt medical informatics standards, to integrate diverse biometric sensors, and to provide different data transmission networks to support a patient's house network despite the facilities. The proposed framework has been evaluated with a case study of two telehealthcare programs, with and without the adoption of the framework. The proposed framework facilitates the functionality of the program and enables steady patient enrollments. The overall patient participations are increased, and the patient outcomes appear positive. The attitudes toward the service and self-improvement also are positive. The findings of this study add up to the construction of a telehealthcare system. Implementing the proposed framework further assists the functionality of the service and enhances the availability of the service and patient acceptances.

A framework for the direct evaluation of large deviations in non-Markovian processes

NASA Astrophysics Data System (ADS)

Cavallaro, Massimo; Harris, Rosemary J.

2016-11-01

We propose a general framework to simulate stochastic trajectories with arbitrarily long memory dependence and efficiently evaluate large deviation functions associated to time-extensive observables. This extends the ‘cloning’ procedure of Giardiná et al (2006 Phys. Rev. Lett. 96 120603) to non-Markovian systems. We demonstrate the validity of this method by testing non-Markovian variants of an ion-channel model and the totally asymmetric exclusion process, recovering results obtainable by other means.

Framework of outcome measures recommended for use in the evaluation of childhood obesity treatment interventions: the CoOR framework.

PubMed

Bryant, M; Ashton, L; Nixon, J; Jebb, S; Wright, J; Roberts, K; Brown, J

2014-12-01

Consensus is lacking in determining appropriate outcome measures for assessment of childhood obesity treatments. Inconsistency in the use and reporting of such measures impedes comparisons between treatments and limits consideration of effectiveness. This study aimed to produce a framework of recommended outcome measures: the Childhood obesity treatment evaluation Outcomes Review (CoOR) framework. A systematic review including two searches was conducted to identify (1) existing trial outcome measures and (2) manuscripts describing development/evaluation of outcome measures. Outcomes included anthropometry, diet, eating behaviours, physical activity, sedentary time/behaviour, fitness, physiology, environment, psychological well-being and health-related quality of life. Eligible measures were appraised by the internal team using a system developed from international guidelines, followed by appraisal from national external expert collaborators. A total of 25,486 papers were identified through both searches. Eligible search 1 trial papers cited 417 additional papers linked to outcome measures, of which 56 were eligible. A further 297 outcome development/evaluation papers met eligibility criteria from search 2. Combined, these described 191 outcome measures. After internal and external appraisal, 52 measures across 10 outcomes were recommended for inclusion in the CoOR framework. Application of the CoOR framework will ensure greater consistency in choosing robust outcome measures that are appropriate to population characteristics. © 2014 The Authors. Pediatric Obesity © 2014 International Association for the Study of Obesity.

A Framework for Describing Health Care Delivery Organizations and Systems

PubMed Central

Cohen, Perry D.; Larson, David B.; Marion, Lucy N.; Sills, Marion R.; Solberg, Leif I.; Zerzan, Judy

2015-01-01

Describing, evaluating, and conducting research on the questions raised by comparative effectiveness research and characterizing care delivery organizations of all kinds, from independent individual provider units to large integrated health systems, has become imperative. Recognizing this challenge, the Delivery Systems Committee, a subgroup of the Agency for Healthcare Research and Quality’s Effective Health Care Stakeholders Group, which represents a wide diversity of perspectives on health care, created a draft framework with domains and elements that may be useful in characterizing various sizes and types of care delivery organizations and may contribute to key outcomes of interest. The framework may serve as the door to further studies in areas in which clear definitions and descriptions are lacking. PMID:24922130

Optimization-Based Sensor Fusion of GNSS and IMU Using a Moving Horizon Approach

PubMed Central

Girrbach, Fabian; Hol, Jeroen D.; Bellusci, Giovanni; Diehl, Moritz

2017-01-01

The rise of autonomous systems operating close to humans imposes new challenges in terms of robustness and precision on the estimation and control algorithms. Approaches based on nonlinear optimization, such as moving horizon estimation, have been shown to improve the accuracy of the estimated solution compared to traditional filter techniques. This paper introduces an optimization-based framework for multi-sensor fusion following a moving horizon scheme. The framework is applied to the often occurring estimation problem of motion tracking by fusing measurements of a global navigation satellite system receiver and an inertial measurement unit. The resulting algorithm is used to estimate position, velocity, and orientation of a maneuvering airplane and is evaluated against an accurate reference trajectory. A detailed study of the influence of the horizon length on the quality of the solution is presented and evaluated against filter-like and batch solutions of the problem. The versatile configuration possibilities of the framework are finally used to analyze the estimated solutions at different evaluation times exposing a nearly linear behavior of the sensor fusion problem. PMID:28534857

Optimization-Based Sensor Fusion of GNSS and IMU Using a Moving Horizon Approach.

PubMed

Girrbach, Fabian; Hol, Jeroen D; Bellusci, Giovanni; Diehl, Moritz

2017-05-19

The rise of autonomous systems operating close to humans imposes new challenges in terms of robustness and precision on the estimation and control algorithms. Approaches based on nonlinear optimization, such as moving horizon estimation, have been shown to improve the accuracy of the estimated solution compared to traditional filter techniques. This paper introduces an optimization-based framework for multi-sensor fusion following a moving horizon scheme. The framework is applied to the often occurring estimation problem of motion tracking by fusing measurements of a global navigation satellite system receiver and an inertial measurement unit. The resulting algorithm is used to estimate position, velocity, and orientation of a maneuvering airplane and is evaluated against an accurate reference trajectory. A detailed study of the influence of the horizon length on the quality of the solution is presented and evaluated against filter-like and batch solutions of the problem. The versatile configuration possibilities of the framework are finally used to analyze the estimated solutions at different evaluation times exposing a nearly linear behavior of the sensor fusion problem.

Electronic palliative care coordination systems: Devising and testing a methodology for evaluating documentation

PubMed Central

Allsop, Matthew J; Kite, Suzanne; McDermott, Sarah; Penn, Naomi; Millares-Martin, Pablo; Bennett, Michael I

2016-01-01

Background: The need to improve coordination of care at end of life has driven electronic palliative care coordination systems implementation across the United Kingdom and internationally. No approaches for evaluating electronic palliative care coordination systems use in practice have been developed. Aim: This study outlines and applies an evaluation framework for examining how and when electronic documentation of advance care planning is occurring in end of life care services. Design: A pragmatic, formative process evaluation approach was adopted. The evaluation drew on the Project Review and Objective Evaluation methodology to guide the evaluation framework design, focusing on clinical processes. Setting/participants: Data were extracted from electronic palliative care coordination systems for 82 of 108 general practices across a large UK city. All deaths (n = 1229) recorded on electronic palliative care coordination systems between April 2014 and March 2015 were included to determine the proportion of all deaths recorded, median number of days prior to death that key information was recorded and observations about routine data use. Results: The evaluation identified 26.8% of all deaths recorded on electronic palliative care coordination systems. The median number of days to death was calculated for initiation of an electronic palliative care coordination systems record (31 days), recording a patient’s preferred place of death (8 days) and entry of Do Not Attempt Cardiopulmonary Resuscitation decisions (34 days). Where preferred and actual place of death was documented, these were matching for 75% of patients. Anomalies were identified in coding used during data entry on electronic palliative care coordination systems. Conclusion: This study reports the first methodology for evaluating how and when electronic palliative care coordination systems documentation is occurring. It raises questions about what can be drawn from routine data collected through electronic palliative care coordination systems and outlines considerations for future evaluation. Future evaluations should consider work processes of health professionals using electronic palliative care coordination systems. PMID:27507636

Electronic palliative care coordination systems: Devising and testing a methodology for evaluating documentation.

PubMed

Allsop, Matthew J; Kite, Suzanne; McDermott, Sarah; Penn, Naomi; Millares-Martin, Pablo; Bennett, Michael I

2017-05-01

The need to improve coordination of care at end of life has driven electronic palliative care coordination systems implementation across the United Kingdom and internationally. No approaches for evaluating electronic palliative care coordination systems use in practice have been developed. This study outlines and applies an evaluation framework for examining how and when electronic documentation of advance care planning is occurring in end of life care services. A pragmatic, formative process evaluation approach was adopted. The evaluation drew on the Project Review and Objective Evaluation methodology to guide the evaluation framework design, focusing on clinical processes. Data were extracted from electronic palliative care coordination systems for 82 of 108 general practices across a large UK city. All deaths ( n = 1229) recorded on electronic palliative care coordination systems between April 2014 and March 2015 were included to determine the proportion of all deaths recorded, median number of days prior to death that key information was recorded and observations about routine data use. The evaluation identified 26.8% of all deaths recorded on electronic palliative care coordination systems. The median number of days to death was calculated for initiation of an electronic palliative care coordination systems record (31 days), recording a patient's preferred place of death (8 days) and entry of Do Not Attempt Cardiopulmonary Resuscitation decisions (34 days). Where preferred and actual place of death was documented, these were matching for 75% of patients. Anomalies were identified in coding used during data entry on electronic palliative care coordination systems. This study reports the first methodology for evaluating how and when electronic palliative care coordination systems documentation is occurring. It raises questions about what can be drawn from routine data collected through electronic palliative care coordination systems and outlines considerations for future evaluation. Future evaluations should consider work processes of health professionals using electronic palliative care coordination systems.

Characterizing the reliability of a bioMEMS-based cantilever sensor

NASA Astrophysics Data System (ADS)

Bhalerao, Kaustubh D.

2004-12-01

The cantilever-based BioMEMS sensor represents one instance from many competing ideas of biosensor technology based on Micro Electro Mechanical Systems. The advancement of BioMEMS from laboratory-scale experiments to applications in the field will require standardization of their components and manufacturing procedures as well as frameworks to evaluate their performance. Reliability, the likelihood with which a system performs its intended task, is a compact mathematical description of its performance. The mathematical and statistical foundation of systems-reliability has been applied to the cantilever-based BioMEMS sensor. The sensor is designed to detect one aspect of human ovarian cancer, namely the over-expression of the folate receptor surface protein (FR-alpha). Even as the application chosen is clinically motivated, the objective of this study was to demonstrate the underlying systems-based methodology used to design, develop and evaluate the sensor. The framework development can be readily extended to other BioMEMS-based devices for disease detection and will have an impact in the rapidly growing $30 bn industry. The Unified Modeling Language (UML) is a systems-based framework for design and development of object-oriented information systems which has potential application for use in systems designed to interact with biological environments. The UML has been used to abstract and describe the application of the biosensor, to identify key components of the biosensor, and the technology needed to link them together in a coherent manner. The use of the framework is also demonstrated in computation of system reliability from first principles as a function of the structure and materials of the biosensor. The outcomes of applying the systems-based framework to the study are the following: (1) Characterizing the cantilever-based MEMS device for disease (cell) detection. (2) Development of a novel chemical interface between the analyte and the sensor that provides a degree of selectivity towards the disease. (3) Demonstrating the performance and measuring the reliability of the biosensor prototype, and (4) Identification of opportunities in technological development in order to further refine the proposed biosensor. Application of the methodology to design develop and evaluate the reliability of BioMEMS devices will be beneficial in the streamlining the growth of the BioMEMS industry, while providing a decision-support tool in comparing and adopting suitable technologies from available competing options.

Enhancing TSM&O strategies through life cycle benefit/cost analysis : life cycle benefit/cost analysis & life cycle assessment of adaptive traffic control systems and ramp metering systems.

DOT National Transportation Integrated Search

2015-05-01

The research team developed a comprehensive Benefit/Cost (B/C) analysis framework to evaluate existing and anticipated : intelligent transportation system (ITS) strategies, particularly, adaptive traffic control systems and ramp metering systems, : i...

A framework for analysis of sentinel events in medical student education.

PubMed

Cohen, Daniel M; Clinchot, Daniel M; Werman, Howard A

2013-11-01

Although previous studies have addressed student factors contributing to dismissal or withdrawal from medical school for academic reasons, little information is available regarding institutional factors that may hinder student progress. The authors describe the development and application of a framework for sentinel event (SE) root cause analysis to evaluate cases in which students are dismissed or withdraw because of failure to progress in the medical school curriculum. The SE in medical student education (MSE) framework was piloted at the Ohio State University College of Medicine (OSUCOM) during 2010-2012. Faculty presented cases using the framework during academic oversight committee discussions. Nine SEs in MSE were presented using the framework. Major institution-level findings included the need for improved communication, documentation of cognitive and noncognitive (e.g., mental health) issues, clarification of requirements for remediation and fitness for duty, and additional psychological services. Challenges related to alternative and combined programs were identified as well. The OSUCOM undertook system changes based on the action plans developed through the discussions of these SEs. An SE analysis process appears to be a useful method for making system changes in response to institutional issues identified in evaluation of cases in which students fail to progress in the medical school curriculum. The authors plan to continue to refine the SE in MSE framework and analysis process. Next steps include assessing whether analysis using this framework yields improved student outcomes with universal applications for other institutions.

Disability Policy Evaluation: Combining Logic Models and Systems Thinking

ERIC Educational Resources Information Center

Claes, Claudia; Ferket, Neelke; Vandevelde, Stijn; Verlet, Dries; De Maeyer, Jessica

2017-01-01

Policy evaluation focuses on the assessment of policy-related personal, family, and societal changes or benefits that follow as a result of the interventions, services, and supports provided to those persons to whom the policy is directed. This article describes a systematic approach to policy evaluation based on an evaluation framework and an…

The GRADE Evidence to Decision (EtD) framework for health system and public health decisions.

PubMed

Moberg, Jenny; Oxman, Andrew D; Rosenbaum, Sarah; Schünemann, Holger J; Guyatt, Gordon; Flottorp, Signe; Glenton, Claire; Lewin, Simon; Morelli, Angela; Rada, Gabriel; Alonso-Coello, Pablo

2018-05-29

To describe a framework for people making and using evidence-informed health system and public health recommendations and decisions. We developed the GRADE Evidence to Decision (EtD) framework for health system and public health decisions as part of the DECIDE project, in which we simultaneously developed frameworks for these and other types of healthcare decisions, including clinical recommendations, coverage decisions and decisions about diagnostic tests. Building on GRADE EtD tables, we used an iterative approach, including brainstorming, consultation of the literature and with stakeholders, and an international survey of policy-makers. We applied the framework to diverse examples, conducted workshops and user testing with health system and public health guideline developers and policy-makers, and observed and tested its use in real-life guideline panels. All the GRADE EtD frameworks share the same basic structure, including sections for formulating the question, making an assessment and drawing conclusions. Criteria listed in the assessment section of the health system and public health framework cover the important factors for making these types of decisions; in addition to the effects and economic impact of an option, the priority of the problem, the impact of the option on equity, and its acceptability and feasibility are important considerations that can inform both whether and how to implement an option. Because health system and public health interventions are often complex, detailed implementation considerations should be made when making a decision. The certainty of the evidence is often low or very low, but decision-makers must still act. Monitoring and evaluation are therefore often important considerations for these types of decisions. We illustrate the different components of the EtD framework for health system and public health decisions by presenting their application in a framework adapted from a real-life guideline. This framework provides a structured and transparent approach to support policy-making informed by the best available research evidence, while making the basis for decisions accessible to those whom they will affect. The health system and public health EtD framework can also be used to facilitate dissemination of recommendations and enable decision-makers to adopt, and adapt, recommendations or decisions.

SpaceCubeX: A Framework for Evaluating Hybrid Multi-Core CPU FPGA DSP Architectures

NASA Technical Reports Server (NTRS)

Schmidt, Andrew G.; Weisz, Gabriel; French, Matthew; Flatley, Thomas; Villalpando, Carlos Y.

2017-01-01

The SpaceCubeX project is motivated by the need for high performance, modular, and scalable on-board processing to help scientists answer critical 21st century questions about global climate change, air quality, ocean health, and ecosystem dynamics, while adding new capabilities such as low-latency data products for extreme event warnings. These goals translate into on-board processing throughput requirements that are on the order of 100-1,000 more than those of previous Earth Science missions for standard processing, compression, storage, and downlink operations. To study possible future architectures to achieve these performance requirements, the SpaceCubeX project provides an evolvable testbed and framework that enables a focused design space exploration of candidate hybrid CPU/FPGA/DSP processing architectures. The framework includes ArchGen, an architecture generator tool populated with candidate architecture components, performance models, and IP cores, that allows an end user to specify the type, number, and connectivity of a hybrid architecture. The framework requires minimal extensions to integrate new processors, such as the anticipated High Performance Spaceflight Computer (HPSC), reducing time to initiate benchmarking by months. To evaluate the framework, we leverage a wide suite of high performance embedded computing benchmarks and Earth science scenarios to ensure robust architecture characterization. We report on our projects Year 1 efforts and demonstrate the capabilities across four simulation testbed models, a baseline SpaceCube 2.0 system, a dual ARM A9 processor system, a hybrid quad ARM A53 and FPGA system, and a hybrid quad ARM A53 and DSP system.

Atlanta congestion reduction demonstration. National evaluation : content analysis test plan.

DOT National Transportation Integrated Search

2000-05-30

Commercial Vehicle Information Systems and Networks (CVISN) is the collection of information systems and communication networks that support commercial vehicle operations (CVO.) The National ITS Architecture provides a technical framework that descri...

A framework to analyze emissions implications of ...

EPA Pesticide Factsheets

Future year emissions depend highly on the evolution of the economy, technology and current and future regulatory drivers. A scenario framework was adopted to analyze various technology development pathways and societal change while considering existing regulations and future uncertainty in regulations and evaluate resulting emissions growth patterns. The framework integrates EPA’s energy systems model with an economic Input-Output (I/O) Life Cycle Assessment model. The EPAUS9r MARKAL database is assembled from a set of technologies to represent the U.S. energy system within MARKAL bottom-up technology rich energy modeling framework. The general state of the economy and consequent demands for goods and services from these sectors are taken exogenously in MARKAL. It is important to characterize exogenous inputs about the economy to appropriately represent the industrial sector outlook for each of the scenarios and case studies evaluated. An economic input-output (I/O) model of the US economy is constructed to link up with MARKAL. The I/O model enables user to change input requirements (e.g. energy intensity) for different sectors or the share of consumer income expended on a given good. This gives end-users a mechanism for modeling change in the two dimensions of technological progress and consumer preferences that define the future scenarios. The framework will then be extended to include environmental I/O framework to track life cycle emissions associated

Collaborative mental health services in primary care systems in Latin America: contextualized evaluation needs and opportunities.

PubMed

Sapag, Jaime C; Rush, Brian; Ferris, Lorraine E

2016-02-01

This study examined Latin American evaluation needs regarding the development of a collaborative mental health care (CMHC) evaluation framework as seen by local key health-care leaders and professionals. Potential implementation challenges and opportunities were also identified. This multisite research study used an embedded mixed methods approach in three public health networks in Mexico, Nicaragua and Chile. Local stakeholders participated: decision-makers in key informant interviews, front-line clinicians in focus groups and other stakeholders through a survey. The analysis was conducted within site and then across sites. A total of 22 semi-structured interviews, three focus groups and 27 questionnaires (52% response rate) were conducted. Participants recognized a strong need to evaluate different areas of CMHC in Latin America, including access, types and quality of services, human resources and outcomes related to mental disorders, including addiction. A priority was to evaluate collaboration within the health system, including the referral system. Issues of feasibility, including the weaknesses of information systems, were also identified. Local stakeholders strongly supported the development of a comprehensive evaluation framework for CMHC in Latin America and cited several dimensions and contextual factors critical for inclusion. Implementation must allow flexibility and adaptation to the local context. © 2015 John Wiley & Sons Ltd.

A Framework for Quantitative Evaluation of Care Coordination Effectiveness

ERIC Educational Resources Information Center

Liu, Wei

2017-01-01

The U.S. healthcare system lacks incentives and quantitative evaluation tools to assess coordination in a patient's care transition process. This is needed because poor care coordination has been identified by many studies as one of the major root causes for the U.S. health system's inefficiency, for poor outcomes, and for high cost. Despite…

A Pragmatic Application of the RE-AIM Framework for Evaluating the Implementation of Physical Activity as a Standard of Care in Health Systems

PubMed Central

Galaviz, Karla I.; Lobelo, Felipe; Joy, Elizabeth; Heath, Gregory W.; Hutber, Adrian; Estabrooks, Paul

2018-01-01

Introduction Exercise is Medicine (EIM) is an initiative that seeks to integrate physical activity assessment, prescription, and patient referral as a standard in patient care. Methods to assess this integration have lagged behind its implementation. Purpose and Objectives The purpose of this work is to provide a pragmatic framework to guide health care systems in assessing the implementation and impact of EIM. Evaluation Methods A working group of experts from health care, public health, and implementation science convened to develop an evaluation model based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. The working group aimed to provide pragmatic guidance on operationalizing EIM across the different RE-AIM dimensions based on data typically available in health care settings. Results The Reach of EIM can be determined by the number and proportion of patients that were screened for physical inactivity, received brief counseling and/or a physical activity prescription, and were referred to physical activity resources. Effectiveness can be assessed through self-reported changes in physical activity, cardiometabolic biometric factors, incidence/burden of chronic disease, as well as health care utilization and costs. Adoption includes assessing the number and representativeness of health care settings that adopt any component of EIM, and Implementation involves assessing the extent to which health care teams implement EIM in their clinic. Finally, Maintenance involves assessing the long-term effectiveness (patient level) and sustained implementation (clinic level) of EIM in a given health care setting. Implications for Public Health The availability of a standardized, pragmatic, evaluation framework is critical in determining the impact of implementing EIM as a standard of care across health care systems. PMID:29752803

A common evaluation framework for the African Health Initiative

PubMed Central

2013-01-01

Background The African Health Initiative includes highly diverse partnerships in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia), each of which is working to improve population health by strengthening health systems and to evaluate the results. One aim of the Initiative is to generate cross-site learning that can inform implementation in the five partnerships during the project period and identify lessons that may be generalizable to other countries in the region. Collaborators in the Initiative developed a common evaluation framework as a basis for this cross-site learning. Methods This paper describes the components of the framework; this includes the conceptual model, core metrics to be measured in all sites, and standard guidelines for reporting on the implementation of partnership activities and contextual factors that may affect implementation, or the results it produces. We also describe the systems that have been put in place for data management, data quality assessments, and cross-site analysis of results. Results and conclusions The conceptual model for the Initiative highlights points in the causal chain between health system strengthening activities and health impact where evidence produced by the partnerships can contribute to learning. This model represents an important advance over its predecessors by including contextual factors and implementation strength as potential determinants, and explicitly including equity as a component of both outcomes and impact. Specific measurement challenges include the prospective documentation of program implementation and contextual factors. Methodological issues addressed in the development of the framework include the aggregation of data collected using different methods and the challenge of evaluating a complex set of interventions being improved over time based on continuous monitoring and intermediate results. PMID:23819778

A strategy for monitoring and evaluating massive open online courses.

PubMed

Chapman, S A; Goodman, S; Jawitz, J; Deacon, A

2016-08-01

We argue that the complex, innovative and adaptive nature of Massive Open Online Course (MOOC) initiatives poses particular challenges to monitoring and evaluation, in that any evaluation strategy will need to follow a systems approach. This article aims to guide organizations implementing MOOCs through a series of steps to assist them in developing a strategy to monitor, improve, and judge the merit of their initiatives. We describe how we operationalise our strategy by first defining the different layers of interacting agents in a given MOOC system. We then tailor our approach to these different layers. Specifically, a two-pronged approach was developed, where we suggest that individual projects be assessed through performance monitoring; assessment criteria for which would be defined at the outset to include coverage, participation, quality and student achievement. In contrast, the success of an overall initiative should be considered within a more adaptive, emergent evaluation inquiry framework. We present the inquiry framework we developed for MOOC initiatives, and show how this framework might be used to develop evaluation questions and an assessment methodology. We also define the more fixed indicators and measures for project performance monitoring. Our strategy is described as it was developed to inform the evaluation of a MOOC initiative at the University of Cape Town (UCT), South Africa. Copyright © 2016 Elsevier Ltd. All rights reserved.

A Systems Approach to Designing Effective Clinical Trials Using Simulations

PubMed Central

Fusaro, Vincent A.; Patil, Prasad; Chi, Chih-Lin; Contant, Charles F.; Tonellato, Peter J.

2013-01-01

Background Pharmacogenetics in warfarin clinical trials have failed to show a significant benefit compared to standard clinical therapy. This study demonstrates a computational framework to systematically evaluate pre-clinical trial design of target population, pharmacogenetic algorithms, and dosing protocols to optimize primary outcomes. Methods and Results We programmatically created an end-to-end framework that systematically evaluates warfarin clinical trial designs. The framework includes options to create a patient population, multiple dosing strategies including genetic-based and non-genetic clinical-based, multiple dose adjustment protocols, pharmacokinetic/pharmacodynamics (PK/PD) modeling and international normalization ratio (INR) prediction, as well as various types of outcome measures. We validated the framework by conducting 1,000 simulations of the CoumaGen clinical trial primary endpoints. The simulation predicted a mean time in therapeutic range (TTR) of 70.6% and 72.2% (P = 0.47) in the standard and pharmacogenetic arms, respectively. Then, we evaluated another dosing protocol under the same original conditions and found a significant difference in TTR between the pharmacogenetic and standard arm (78.8% vs. 73.8%; P = 0.0065), respectively. Conclusions We demonstrate that this simulation framework is useful in the pre-clinical assessment phase to study and evaluate design options and provide evidence to optimize the clinical trial for patient efficacy and reduced risk. PMID:23261867

Evaluating the effectiveness of risk minimisation measures: the application of a conceptual framework to Danish real-world dabigatran data.

PubMed

Nyeland, Martin Erik; Laursen, Mona Vestergaard; Callréus, Torbjörn

2017-06-01

For both marketing authorization holders and regulatory authorities, evaluating the effectiveness of risk minimization measures is now an integral part of pharmacovigilance in the European Union. The overall aim of activities in this area is to assess the performance of risk minimization measures implemented in order to ensure a positive benefit-risk balance in patients treated with a medicinal product. Following a review of the relevant literature, we developed a conceptual framework consisting of four domains (data, knowledge, behaviour and outcomes) intended for the evaluation of risk minimization measures put into practice in the Danish health-care system. For the implementation of the framework, four classes of monitoring variables can be named and defined: patient descriptors, performance-related indicators of knowledge, behaviour and outcomes. We reviewed the features of the framework when applied to historical, real-world data following the introduction of dabigatran in Denmark for the prophylactic treatment of patients with non-valvular atrial fibrillation. The application of the framework provided useful graphical displays and an opportunity for a statistical evaluation (interrupted time series analysis) of a regulatory intervention. © 2017 Commonwealth of Australia. Pharmacoepidemiology & Drug Safety © 2017 John Wiley & Sons, Ltd. © 2017 Commonwealth of Australia. Pharmacoepidemiology & Drug Safety © 2017 John Wiley & Sons, Ltd.

A Method for Evaluating Information Security Governance (ISG) Components in Banking Environment

NASA Astrophysics Data System (ADS)

Ula, M.; Ula, M.; Fuadi, W.

2017-02-01

As modern banking increasingly relies on the internet and computer technologies to operate their businesses and market interactions, the threats and security breaches have highly increased in recent years. Insider and outsider attacks have caused global businesses lost trillions of Dollars a year. Therefore, that is a need for a proper framework to govern the information security in the banking system. The aim of this research is to propose and design an enhanced method to evaluate information security governance (ISG) implementation in banking environment. This research examines and compares the elements from the commonly used information security governance frameworks, standards and best practices. Their strength and weakness are considered in its approaches. The initial framework for governing the information security in banking system was constructed from document review. The framework was categorized into three levels which are Governance level, Managerial level, and technical level. The study further conducts an online survey for banking security professionals to get their professional judgment about the ISG most critical components and the importance for each ISG component that should be implemented in banking environment. Data from the survey was used to construct a mathematical model for ISG evaluation, component importance data used as weighting coefficient for the related component in the mathematical model. The research further develops a method for evaluating ISG implementation in banking based on the mathematical model. The proposed method was tested through real bank case study in an Indonesian local bank. The study evidently proves that the proposed method has sufficient coverage of ISG in banking environment and effectively evaluates the ISG implementation in banking environment.

Using frameworks to diagram value in complex policy and environmental interventions to prevent childhood obesity.

PubMed

Swank, Melissa Farrell; Brennan, Laura K; Gentry, Daniel; Kemner, Allison L

2015-01-01

To date, few tools assist policy makers and practitioners in understanding and conveying the implementation costs, potential impacts, and value of policy and environmental changes to address healthy eating, active living, and childhood obesity. For the Evaluation of Healthy Kids, Healthy Communities (HKHC), evaluators considered inputs (resources and investments) that generate costs and savings as well as benefits and harms related to social, economic, environmental, and health-related outcomes in their assessment of 49 HKHC community partnerships funded from 2009 to 2014. Using data collected through individual and group interviews and an online performance monitoring system, evaluators created a socioecological framework to assess investments, resources, costs, savings, benefits, and harms at the individual, organizational, community, and societal levels. Evaluators customized frameworks for 6 focal strategies: active transportation, parks and play spaces, child care physical activity standards, corner stores, farmers' markets, and child care nutrition standards. To illustrate the Value Frameworks, this brief highlights the 38 HKHC communities implementing at least 1 active transportation strategy. Evaluators populated this conceptual Value Framework with themes from the strategy-specific inputs and outputs. The range of factors corresponding to the implementation and impact of the HKHC community partnerships are highlighted along with the inputs and outputs. The Value Frameworks helped evaluators identify gaps in current analysis models (ie, benefit-cost analysis, cost-effectiveness analysis) as well as paint a more complete picture of value for potential obesity prevention strategies. These frameworks provide a comprehensive understanding of investments needed, proposed costs and savings, and potential benefits and harms associated with economic, social, environmental, and health outcomes. This framing also allowed evaluators to demonstrate the interdependence of each socioecological level on the others in these multicomponent interventions. This model can be used by practitioners and community leaders to assess realistic and sustainable strategies to combat childhood obesity.

A Framework for Developing the Structure of Public Health Economic Models.

PubMed

Squires, Hazel; Chilcott, James; Akehurst, Ronald; Burr, Jennifer; Kelly, Michael P

2016-01-01

A conceptual modeling framework is a methodology that assists modelers through the process of developing a model structure. Public health interventions tend to operate in dynamically complex systems. Modeling public health interventions requires broader considerations than clinical ones. Inappropriately simple models may lead to poor validity and credibility, resulting in suboptimal allocation of resources. This article presents the first conceptual modeling framework for public health economic evaluation. The framework presented here was informed by literature reviews of the key challenges in public health economic modeling and existing conceptual modeling frameworks; qualitative research to understand the experiences of modelers when developing public health economic models; and piloting a draft version of the framework. The conceptual modeling framework comprises four key principles of good practice and a proposed methodology. The key principles are that 1) a systems approach to modeling should be taken; 2) a documented understanding of the problem is imperative before and alongside developing and justifying the model structure; 3) strong communication with stakeholders and members of the team throughout model development is essential; and 4) a systematic consideration of the determinants of health is central to identifying the key impacts of public health interventions. The methodology consists of four phases: phase A, aligning the framework with the decision-making process; phase B, identifying relevant stakeholders; phase C, understanding the problem; and phase D, developing and justifying the model structure. Key areas for further research involve evaluation of the framework in diverse case studies and the development of methods for modeling individual and social behavior. This approach could improve the quality of Public Health economic models, supporting efficient allocation of scarce resources. Copyright © 2016 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

[How can the impact of Health Technology Assessment (HTA) in the Austrian healthcare system be assessed? Design of a conceptual framework].

PubMed

Schumacher, I; Zechmeister, I

2012-04-01

In Austria research in Health Technology Assessment (HTA) has been conducted since the 1990s. Research in HTA aims at supporting an adequate and efficient use of health care resources in order to sustain a publicly financed and solidary health care system. Ultimately, HTA research should result in better health of the population. Research results should provide independent information for decision makers. For legitimizing further research resources and for prioritizing future HTA research and guaranteeing the value of future research, HTA research needs itself to undergo evaluation. Aim of the study is to design a conceptual framework for evaluating the impact of HTA research in Austria on the basis of the existing literature. An already existing review which presents methods and concepts how to evaluate HTA-impact was updated by a systematic research including literature of the years 2004-January 2010. Results were analysed in regard to 4 categories: definition of the term impact, target groups and system levels, operationalisation of indicators and evaluation methods. Overall, 19 publications were included. Referring to the 4 categories, an explanation of impact has to take into account HTAs multidisciplinary setting and needs a context related definition. Target groups, system levels, indicators and methods depend on the impact defined. Studies investigated direct and indirect impact and were focused on different target groups like physicians, nurses and decision makers on the micro-, and meso level, as well as politicians and reimbursement institutions on the macro level. Except for one reference all studies applied already known and mostly qualitative methods for measuring the impact of HTA research. Thus, an appropriate pool of instruments seems to be available. There is a lack of information about validity of applied methods and indicators. By adapting adequate methods and concepts a conceptual framework for the Austrian HTA-Impact evaluation has been designed. The paper presents an overview of existing methods for the evaluation of the HTA research. This has been used to identify useful approaches for measuring the HTA-impact in Austria. By providing a context sensitive framework for impact evaluation in Austria the Austrian HTA-research contributes to the international trend of impact-evaluation. © Georg Thieme Verlag KG Stuttgart · New York.

Comparative SWOT analysis of strategic environmental assessment systems in the Middle East and North Africa region.

PubMed

Rachid, G; El Fadel, M

2013-08-15

This paper presents a SWOT analysis of SEA systems in the Middle East North Africa region through a comparative examination of the status, application and structure of existing systems based on country-specific legal, institutional and procedural frameworks. The analysis is coupled with the multi-attribute decision making method (MADM) within an analytical framework that involves both performance analysis based on predefined evaluation criteria and countries' self-assessment of their SEA system through open-ended surveys. The results show heterogenous status with a general delayed progress characterized by varied levels of weaknesses embedded in the legal and administrative frameworks and poor integration with the decision making process. Capitalizing on available opportunities, the paper highlights measures to enhance the development and enactment of SEA in the region. Copyright © 2013 Elsevier Ltd. All rights reserved.

Sociotechnical systems as a framework for regulatory system design and evaluation: Using Work Domain Analysis to examine a new regulatory system.

PubMed

Carden, Tony; Goode, Natassia; Read, Gemma J M; Salmon, Paul M

2017-03-15

Like most work systems, the domain of adventure activities has seen a series of serious incidents and subsequent calls to improve regulation. Safety regulation systems aim to promote safety and reduce accidents. However, there is scant evidence they have led to improved safety outcomes. In fact there is some evidence that the poor integration of regulatory system components has led to adverse safety outcomes in some contexts. Despite this, there is an absence of methods for evaluating regulatory and compliance systems. This article argues that sociotechnical systems theory and methods provide a suitable framework for evaluating regulatory systems. This is demonstrated through an analysis of a recently introduced set of adventure activity regulations. Work Domain Analysis (WDA) was used to describe the regulatory system in terms of its functional purposes, values and priority measures, purpose-related functions, object-related processes and cognitive objects. This allowed judgement to be made on the nature of the new regulatory system and on the constraints that may impact its efficacy following implementation. Importantly, the analysis suggests that the new system's functional purpose of ensuring safe activities is not fully supported in terms of the functions and objects available to fulfil them. Potential improvements to the design of the system are discussed along with the implications for regulatory system design and evaluation across the safety critical domains generally. Copyright © 2017 Elsevier Ltd. All rights reserved.

Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use

ERIC Educational Resources Information Center

Patton, Michael Quinn

2010-01-01

Developmental evaluation (DE) offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority shows how to conduct evaluations within a DE framework. Patton draws on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He…

Reference condition approach to restoration planning

USGS Publications Warehouse

Nestler, J.M.; Theiling, C.H.; Lubinski, S.J.; Smith, D.L.

2010-01-01

Ecosystem restoration planning requires quantitative rigor to evaluate alternatives, define end states, report progress and perform environmental benefits analysis (EBA). Unfortunately, existing planning frameworks are, at best, semi-quantitative. In this paper, we: (1) describe a quantitative restoration planning approach based on a comprehensive, but simple mathematical framework that can be used to effectively apply knowledge and evaluate alternatives, (2) use the approach to derive a simple but precisely defined lexicon based on the reference condition concept and allied terms and (3) illustrate the approach with an example from the Upper Mississippi River System (UMRS) using hydrologic indicators. The approach supports the development of a scaleable restoration strategy that, in theory, can be expanded to ecosystem characteristics such as hydraulics, geomorphology, habitat and biodiversity. We identify three reference condition types, best achievable condition (A BAC), measured magnitude (MMi which can be determined at one or many times and places) and desired future condition (ADFC) that, when used with the mathematical framework, provide a complete system of accounts useful for goal-oriented system-level management and restoration. Published in 2010 by John Wiley & Sons, Ltd.

Evaluating the Relevance, Reliability, and Applicability of CMIP5 Climate Projections for Water Resources and Environmental Planning

NASA Astrophysics Data System (ADS)

Brekke, L. D.; Scott, J.; Ferguson, I. M.; Arnold, J.; Raff, D. A.; Webb, R. S.

2012-12-01

Water managers need to understand the applicability of climate projection information available for decision-support at the scale of their applications. Applicability depends on information reliability and relevance. This need to understand applicability stems from expectations that entities rationalize adaptation investments or decisions to delay investment. It is also occurring at a time when new global climate projections are being released through the World Climate Research Programme Coupled Model Intercomparison Project phase 5 (CMIP5), which introduces new information opportunities and interpretation challenges. This project involves an interagency collaboration to evaluate the applicability of CMIP5 projections for use in water and environmental resources planning. The overarching goal is to develop and demonstrate a framework that involves dual evaluations of relevance and reliability informing an ultimate discussion and judgment of applicability, which is expected to vary with decision-making context. The framework is being developed and demonstrated within the context of reservoir systems management in California's Sacramento and San Joaquin River basins. The relevance evaluation focuses on identifying the climate variables and statistical measures relevant to long-term management questions, which may depend on satisfying multiple objectives. Past studies' results are being considered in this evaluation, along with new results from system sensitivity analyses conducted through this effort. The reliability evaluation focuses on the CMIP5 climate models' ability to simulate past conditions relative to observed references. The evaluation is being conducted across the global domain using a large menu of climate variables and statistical measures, leveraging lessons learned from similar evaluations of CMIP3 climate models. The global focus addresses a broader project goal of producing a web resource that can serve reliability information to applicability discussions around the world, with evaluation results being served through a web-portal similar to that developed by NOAA/CIRES to serve CMIP3 information on future climate extremes (http://www.esrl.noaa.gov/psd/ipcc/extremes/). The framework concludes with an applicability discussion informed by relevance and reliability results. The goal is to observe the discussion process and identify features, choice points, and challenges that might be summarized and shared with other resource management groups facing applicability questions. This presentation will discuss the project framework and preliminary results. In addition to considering CMIP5 21st century projection information, the framework is being developed to support evaluation of CMIP5 decadal predictability experiment simulations and reconcile those simulations with 21st century projections. The presentation will also discuss implications of considering the applicability of bias-corrected and downscaled information within this framework.

The Operation Mechanisms of External Quality Assurance Frameworks of Foreign Higher Education and Implications for Graduate Education

ERIC Educational Resources Information Center

Lin, Mengquan; Chang, Kai; Gong, Le

2016-01-01

The higher education quality evaluation and assurance frameworks and their operating mechanisms of countries such as the United Kingdom, France, and the United States show that higher education systems, traditional culture, and social background all impact quality assurance operating mechanisms. A model analysis of these higher education quality…

The knowledge-based framework for a nuclear power plant operator advisor

DOE Office of Scientific and Technical Information (OSTI.GOV)

Miller, D.W.; Hajek, B.K.

1989-01-01

An important facet in the design, development, and evaluation of aids for complex systems is the identification of the tasks performed by the operator. Operator aids utilizing artificial intelligence, or more specifically knowledge-based systems, require identification of these tasks in the context of a knowledge-based framework. In this context, the operator responses to the plant behavior are to monitor and comprehend the state of the plant, identify normal and abnormal plant conditions, diagnose abnormal plant conditions, predict plant response to specific control actions, and select the best available control action, implement a feasible control action, monitor system response to themore » control action, and correct for any inappropriate responses. These tasks have been identified to formulate a knowledge-based framework for an operator advisor under development at Ohio State University that utilizes the generic task methodology proposed by Chandrasekaran. The paper lays the foundation to identify the responses as a knowledge-based set of tasks in accordance with the expected human operator responses during an event. Initial evaluation of the expert system indicates the potential for an operator aid that will improve the operator's ability to respond to both anticipated and unanticipated events.« less

Modeling and Advanced Control for Sustainable Process Systems

EPA Science Inventory

This book chapter introduces a novel process systems engineering framework that integrates process control with sustainability assessment tools for the simultaneous evaluation and optimization of process operations. The implemented control strategy consists of a biologically-insp...

Assisting allied health in performance evaluation: a systematic review.

PubMed

Lizarondo, Lucylynn; Grimmer, Karen; Kumar, Saravana

2014-11-14

Performance evaluation raises several challenges to allied health practitioners and there is no agreed approach to measuring or monitoring allied health service performance. The aim of this review was to examine the literature on performance evaluation in healthcare to assist in the establishment of a framework that can guide the measurement and evaluation of allied health clinical service performance. This review determined the core elements of a performance evaluation system, tools for evaluating performance, and barriers to the implementation of performance evaluation. A systematic review of the literature was undertaken. Five electronic databases were used to search for relevant articles: MEDLINE, Embase, CINAHL, PsychInfo, and Academic Search Premier. Articles which focussed on any allied health performance evaluation or those which examined performance in health care in general were considered in the review. Content analysis was used to synthesise the findings from individual articles. A total of 37 articles were included in the review. The literature suggests there are core elements involved in performance evaluation which include prioritising clinical areas for measurement, setting goals, selecting performance measures, identifying sources of feedback, undertaking performance measurement, and reporting the results to relevant stakeholders. The literature describes performance evaluation as multi-dimensional, requiring information or data from more than one perspective to provide a rich assessment of performance. A range of tools or instruments are available to capture various perspectives and gather a comprehensive picture of health care quality. Every allied health care delivery system has different performance needs and will therefore require different approaches. However, there are core processes that can be used as a framework to evaluate allied health performance. A careful examination of barriers to performance evaluation and subsequent tailoring of strategies to overcome these barriers should be undertaken to achieve the aims of performance evaluation. The findings of this review should inform the development of a standardised framework that can be used to measure and evaluate allied health performance. Future research should explore the utility and overall impact of such framework in allied health service delivery.

Evaluating the Population Impact on Racial/Ethnic Disparities in HIV in Adulthood of Intervening on Specific Targets: A Conceptual and Methodological Framework.

PubMed

Howe, Chanelle J; Dulin-Keita, Akilah; Cole, Stephen R; Hogan, Joseph W; Lau, Bryan; Moore, Richard D; Mathews, W Christopher; Crane, Heidi M; Drozd, Daniel R; Geng, Elvin; Boswell, Stephen L; Napravnik, Sonia; Eron, Joseph J; Mugavero, Michael J

2018-02-01

Reducing racial/ethnic disparities in human immunodeficiency virus (HIV) disease is a high priority. Reductions in HIV racial/ethnic disparities can potentially be achieved by intervening on important intermediate factors. The potential population impact of intervening on intermediates can be evaluated using observational data when certain conditions are met. However, using standard stratification-based approaches commonly employed in the observational HIV literature to estimate the potential population impact in this setting may yield results that do not accurately estimate quantities of interest. Here we describe a useful conceptual and methodological framework for using observational data to appropriately evaluate the impact on HIV racial/ethnic disparities of interventions. This framework reframes relevant scientific questions in terms of a controlled direct effect and estimates a corresponding proportion eliminated. We review methods and conditions sufficient for accurate estimation within the proposed framework. We use the framework to analyze data on 2,329 participants in the CFAR [Centers for AIDS Research] Network of Integrated Clinical Systems (2008-2014) to evaluate the potential impact of universal prescription of and ≥95% adherence to antiretroviral therapy on racial disparities in HIV virological suppression. We encourage the use of the described framework to appropriately evaluate the potential impact of targeted interventions in addressing HIV racial/ethnic disparities using observational data. © The Author(s) 2017. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Development of hospital disaster resilience: conceptual framework and potential measurement.

PubMed

Zhong, Shuang; Clark, Michele; Hou, Xiang-Yu; Zang, Yu-Li; Fitzgerald, Gerard

2014-11-01

Despite 'hospital resilience' gaining prominence in recent years, it remains poorly defined. This article aims to define hospital resilience, build a preliminary conceptual framework and highlight possible approaches to measurement. Searches were conducted of the commonly used health databases to identify relevant literature and reports. Search terms included 'resilience and framework or model' or 'evaluation or assess or measure and hospital and disaster or emergency or mass casualty and resilience or capacity or preparedness or response or safety'. Articles were retrieved that focussed on disaster resilience frameworks and the evaluation of various hospital capacities. A total of 1480 potentially eligible publications were retrieved initially but the final analysis was conducted on 47 articles, which appeared to contribute to the study objectives. Four disaster resilience frameworks and 11 evaluation instruments of hospital disaster capacity were included. Hospital resilience is a comprehensive concept derived from existing disaster resilience frameworks. It has four key domains: hospital safety; disaster preparedness and resources; continuity of essential medical services; recovery and adaptation. These domains were categorised according to four criteria, namely, robustness, redundancy, resourcefulness and rapidity. A conceptual understanding of hospital resilience is essential for an intellectual basis for an integrated approach to system development. This article (1) defines hospital resilience; (2) constructs conceptual framework (including key domains); (3) proposes comprehensive measures for possible inclusion in an evaluation instrument; and (4) develops a matrix of critical issues to enhance hospital resilience to cope with future disasters. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Real-Time Data Capture and Management Evaluation and Performance Measures : Evaluation Framework

DOT National Transportation Integrated Search

2011-09-01

Through connected vehicle research, the U.S. DOT Intelligent Transportation Systems Joint Program Office (ITS JPO) is leading an effort to assess the potential for systematic and dynamic data capture from vehicles, travelers and the transportation sy...

A surety engineering framework to reduce cognitive systems risks.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Caudell, Thomas P.; Peercy, David Eugene; Caldera, Eva O.

Cognitive science research investigates the advancement of human cognition and neuroscience capabilities. Addressing risks associated with these advancements can counter potential program failures, legal and ethical issues, constraints to scientific research, and product vulnerabilities. Survey results, focus group discussions, cognitive science experts, and surety researchers concur technical risks exist that could impact cognitive science research in areas such as medicine, privacy, human enhancement, law and policy, military applications, and national security (SAND2006-6895). This SAND report documents a surety engineering framework and a process for identifying cognitive system technical, ethical, legal and societal risks and applying appropriate surety methods to reducemore » such risks. The framework consists of several models: Specification, Design, Evaluation, Risk, and Maturity. Two detailed case studies are included to illustrate the use of the process and framework. Several Appendices provide detailed information on existing cognitive system architectures; ethical, legal, and societal risk research; surety methods and technologies; and educing information research with a case study vignette. The process and framework provide a model for how cognitive systems research and full-scale product development can apply surety engineering to reduce perceived and actual risks.« less

Operationalizing the Learning Health Care System in an Integrated Delivery System

PubMed Central

Psek, Wayne A.; Stametz, Rebecca A.; Bailey-Davis, Lisa D.; Davis, Daniel; Darer, Jonathan; Faucett, William A.; Henninger, Debra L.; Sellers, Dorothy C.; Gerrity, Gloria

2015-01-01

Introduction: The Learning Health Care System (LHCS) model seeks to utilize sophisticated technologies and competencies to integrate clinical operations, research and patient participation in order to continuously generate knowledge, improve care, and deliver value. Transitioning from concept to practical application of an LHCS presents many challenges but can yield opportunities for continuous improvement. There is limited literature and practical experience available in operationalizing the LHCS in the context of an integrated health system. At Geisinger Health System (GHS) a multi-stakeholder group is undertaking to enhance organizational learning and develop a plan for operationalizing the LHCS system-wide. We present a framework for operationalizing continuous learning across an integrated delivery system and lessons learned through the ongoing planning process. Framework: The framework focuses attention on nine key LHCS operational components: Data and Analytics; People and Partnerships; Patient and Family Engagement; Ethics and Oversight; Evaluation and Methodology; Funding; Organization; Prioritization; and Deliverables. Definitions, key elements and examples for each are presented. The framework is purposefully broad for application across different organizational contexts. Conclusion: A realistic assessment of the culture, resources and capabilities of the organization related to learning is critical to defining the scope of operationalization. Engaging patients in clinical care and discovery, including quality improvement and comparative effectiveness research, requires a defensible ethical framework that undergirds a system of strong but flexible oversight. Leadership support is imperative for advancement of the LHCS model. Findings from our ongoing work within the proposed framework may inform other organizations considering a transition to an LHCS. PMID:25992388

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ryu, Jun Hyung; Lee, Soo bin; Hodge, Bri-Mathias

The energy system of process industry are faced with a new unprecedented challenge. Renewable energies should be incorporated but single of them cannot meet its energy demand of high degree and a large quantity. This paper investigates a simulation framework to compute the capacity of multiple energy sources including solar, wind power, diesel and batteries. The framework involves actual renewable energy supply and demand profile generation and supply demand matching. Eight configurations of different supply options are evaluated to illustrate the applicability of the proposed framework with some remarks.

Basic test framework for the evaluation of text line segmentation and text parameter extraction.

PubMed

Brodić, Darko; Milivojević, Dragan R; Milivojević, Zoran

2010-01-01

Text line segmentation is an essential stage in off-line optical character recognition (OCR) systems. It is a key because inaccurately segmented text lines will lead to OCR failure. Text line segmentation of handwritten documents is a complex and diverse problem, complicated by the nature of handwriting. Hence, text line segmentation is a leading challenge in handwritten document image processing. Due to inconsistencies in measurement and evaluation of text segmentation algorithm quality, some basic set of measurement methods is required. Currently, there is no commonly accepted one and all algorithm evaluation is custom oriented. In this paper, a basic test framework for the evaluation of text feature extraction algorithms is proposed. This test framework consists of a few experiments primarily linked to text line segmentation, skew rate and reference text line evaluation. Although they are mutually independent, the results obtained are strongly cross linked. In the end, its suitability for different types of letters and languages as well as its adaptability are its main advantages. Thus, the paper presents an efficient evaluation method for text analysis algorithms.

Basic Test Framework for the Evaluation of Text Line Segmentation and Text Parameter Extraction

PubMed Central

Brodić, Darko; Milivojević, Dragan R.; Milivojević, Zoran

2010-01-01

Text line segmentation is an essential stage in off-line optical character recognition (OCR) systems. It is a key because inaccurately segmented text lines will lead to OCR failure. Text line segmentation of handwritten documents is a complex and diverse problem, complicated by the nature of handwriting. Hence, text line segmentation is a leading challenge in handwritten document image processing. Due to inconsistencies in measurement and evaluation of text segmentation algorithm quality, some basic set of measurement methods is required. Currently, there is no commonly accepted one and all algorithm evaluation is custom oriented. In this paper, a basic test framework for the evaluation of text feature extraction algorithms is proposed. This test framework consists of a few experiments primarily linked to text line segmentation, skew rate and reference text line evaluation. Although they are mutually independent, the results obtained are strongly cross linked. In the end, its suitability for different types of letters and languages as well as its adaptability are its main advantages. Thus, the paper presents an efficient evaluation method for text analysis algorithms. PMID:22399932

Land Surface Verification Toolkit (LVT) - A Generalized Framework for Land Surface Model Evaluation

NASA Technical Reports Server (NTRS)

Kumar, Sujay V.; Peters-Lidard, Christa D.; Santanello, Joseph; Harrison, Ken; Liu, Yuqiong; Shaw, Michael

2011-01-01

Model evaluation and verification are key in improving the usage and applicability of simulation models for real-world applications. In this article, the development and capabilities of a formal system for land surface model evaluation called the Land surface Verification Toolkit (LVT) is described. LVT is designed to provide an integrated environment for systematic land model evaluation and facilitates a range of verification approaches and analysis capabilities. LVT operates across multiple temporal and spatial scales and employs a large suite of in-situ, remotely sensed and other model and reanalysis datasets in their native formats. In addition to the traditional accuracy-based measures, LVT also includes uncertainty and ensemble diagnostics, information theory measures, spatial similarity metrics and scale decomposition techniques that provide novel ways for performing diagnostic model evaluations. Though LVT was originally designed to support the land surface modeling and data assimilation framework known as the Land Information System (LIS), it also supports hydrological data products from other, non-LIS environments. In addition, the analysis of diagnostics from various computational subsystems of LIS including data assimilation, optimization and uncertainty estimation are supported within LVT. Together, LIS and LVT provide a robust end-to-end environment for enabling the concepts of model data fusion for hydrological applications. The evolving capabilities of LVT framework are expected to facilitate rapid model evaluation efforts and aid the definition and refinement of formal evaluation procedures for the land surface modeling community.

Classification framework for partially observed dynamical systems

NASA Astrophysics Data System (ADS)

Shen, Yuan; Tino, Peter; Tsaneva-Atanasova, Krasimira

2017-04-01

We present a general framework for classifying partially observed dynamical systems based on the idea of learning in the model space. In contrast to the existing approaches using point estimates of model parameters to represent individual data items, we employ posterior distributions over model parameters, thus taking into account in a principled manner the uncertainty due to both the generative (observational and/or dynamic noise) and observation (sampling in time) processes. We evaluate the framework on two test beds: a biological pathway model and a stochastic double-well system. Crucially, we show that the classification performance is not impaired when the model structure used for inferring posterior distributions is much more simple than the observation-generating model structure, provided the reduced-complexity inferential model structure captures the essential characteristics needed for the given classification task.

The Genome-based Knowledge Management in Cycles model: a complex adaptive systems framework for implementation of genomic applications.

PubMed

Arar, Nedal; Knight, Sara J; Modell, Stephen M; Issa, Amalia M

2011-03-01

The main mission of the Genomic Applications in Practice and Prevention Network™ is to advance collaborative efforts involving partners from across the public health sector to realize the promise of genomics in healthcare and disease prevention. We introduce a new framework that supports the Genomic Applications in Practice and Prevention Network mission and leverages the characteristics of the complex adaptive systems approach. We call this framework the Genome-based Knowledge Management in Cycles model (G-KNOMIC). G-KNOMIC proposes that the collaborative work of multidisciplinary teams utilizing genome-based applications will enhance translating evidence-based genomic findings by creating ongoing knowledge management cycles. Each cycle consists of knowledge synthesis, knowledge evaluation, knowledge implementation and knowledge utilization. Our framework acknowledges that all the elements in the knowledge translation process are interconnected and continuously changing. It also recognizes the importance of feedback loops, and the ability of teams to self-organize within a dynamic system. We demonstrate how this framework can be used to improve the adoption of genomic technologies into practice using two case studies of genomic uptake.

Estimation for aerial detection effectiveness with cooperation efficiency factors of early-warning aircraft in early-warning detection SoS under BSC framework

NASA Astrophysics Data System (ADS)

Zhu, Feng; Hu, Xiaofeng; He, Xiaoyuan; Guo, Rui; Li, Kaiming; Yang, Lu

2017-11-01

In the military field, the performance evaluation of early-warning aircraft deployment or construction is always an important problem needing to be explored. As an effective approach of enterprise management and performance evaluation, Balanced Score Card (BSC) attracts more and more attentions and is studied more and more widely all over the world. It can also bring feasible ideas and technical approaches for studying the issue of the performance evaluation of the deployment or construction of early-warning aircraft which is the important component in early-warning detection system of systems (SoS). Therefore, the deep explored researches are carried out based on the previously research works. On the basis of the characteristics of space exploration and aerial detection effectiveness of early-warning detection SoS and the cardinal principle of BSC are analyzed simply, and the performance evaluation framework of the deployment or construction of early-warning aircraft is given, under this framework, aimed at the evaluation issue of aerial detection effectiveness of early-warning detection SoS with the cooperation efficiency factors of the early-warning aircraft and other land based radars, the evaluation indexes are further designed and the relative evaluation model is further established, especially the evaluation radar chart being also drawn to obtain the evaluation results from a direct sight angle. Finally, some practical computer simulations are launched to prove the validity and feasibility of the research thinking and technologic approaches which are proposed in the paper.

Marginal and internal fit of posterior three-unit fixed zirconia dental prostheses fabricated with two different CAD/CAM systems and materials.

PubMed

Schönberger, Joana; Erdelt, Kurt-Jürgen; Bäumer, Daniel; Beuer, Florian

2017-11-01

The purpose of this in vitro study was to compare the precision of fit of frameworks milled from semi-sintered regular zirconia and high-translucent (HT) zirconia blanks, fabricated with two different CAD/CAM systems. Three-unit, posterior fixed dental prostheses (FDP) frameworks were fabricated for standardized dies (n = 11) with two different laboratory computer-aided design (CAD)/computer-aided manufacturing (CAM) systems (Cercon/Ceramill). The replica technique was used to evaluate the marginal and internal fit under an optical microscope. Evaluation of the data was performed according to prior studies at a level of significance of 5%. The systems showed a statistically significant influence on the internal fit of the frameworks (p ≤ 0.001) and on the marginal fit (p < 0.001). The type of material showed no influence on the marginal fit for the Cercon system (p = 0.636) and on the marginal fit (p = 0.064) and the internal fit (p = 0.316) for the Ceramill system, while regular zirconia from Cercon showed higher internal values than HT zirconia (p = 0.016). Both investigated systems showed clinically acceptable values within the limitations of this in vitro study. However, one showed less internal accuracy when regular zirconia was used.

An Analysis of Principals' Perceptions of the Primary Teaching Evaluation System Used in Eight U.S. States

ERIC Educational Resources Information Center

Dodson, Richard L.

2017-01-01

This research examines how public school principals in eight U.S. states perceive their teacher evaluation systems, which are based on Charlotte Danielson's Framework for Teaching (FfT). States were selected to represent high, middle, and low scorers in the annual "Education Week" "Quality Counts" report (Education Week, 2016).…

How Does Research Evaluation Impact Educational Research? Exploring Intended and Unintended Consequences of Research Assessment in the United Kingdom, 1986-2014

ERIC Educational Resources Information Center

Marques, Marcelo; Powell, Justin J. W.; Zapp, Mike; Biesta, Gert

2017-01-01

Research evaluation systems in many countries aim to improve the quality of higher education. Among the first of such systems, the UK's Research Assessment Exercise (RAE) dating from 1986 is now the Research Excellence Framework (REF). Highly institutionalised, it transforms research to be more accountable. While numerous studies describe the…

Computer-Aided Group Problem Solving for Unified Life Cycle Engineering (ULCE)

DTIC Science & Technology

1989-02-01

defining the problem, generating alternative solutions, evaluating alternatives, selecting alternatives, and implementing the solution. Systems...specialist in group dynamics, assists the group in formulating the problem and selecting a model framework. The analyst provides the group with computer...allocating resources, evaluating and selecting options, making judgments explicit, and analyzing dynamic systems. c. University of Rhode Island Drs. Geoffery

The Getting To Outcomes Demonstration and Evaluation: An Illustration of the Prevention Support System

PubMed Central

Hunter, Sarah B.; Ebener, Patricia; Paddock, Susan M.; Stillman, Lindsey; Imm, Pamela; Wandersman, Abraham

2010-01-01

Communities are increasingly being required by state and federal funders to achieve outcomes and be accountable, yet are often not provided the guidance or the tools needed to successfully meet this challenge. To improve the likelihood of achieving positive outcomes, the Getting To Outcomes (GTO) intervention (manual, training, technical assistance) is designed to provide the necessary guidance and tools, tailored to community needs, in order to build individual capacity and program performance. GTO is an example of a Prevention Support System intervention, which as conceptualized by the Interactive Systems Framework, plays a key role in bridging the gap between prevention science (Prevention Synthesis and Translation System) and prevention practice (Prevention Delivery System). We evaluated the impact of GTO on individual capacity and program performance using survey- and interview-based methods. We tracked the implementation of GTO and gathered user feedback about its utility and acceptability. The evaluation of GTO suggests that it can build individual capacity and program performance and as such demonstrates that the Prevention Support System can successfully fulfill its intended role. Lessons learned from the implementation of GTO relevant to illuminating the framework are discussed. PMID:18278551

A quantitative framework to evaluate modeling of cortical development by neural stem cells

PubMed Central

Stein, Jason L.; de la Torre-Ubieta, Luis; Tian, Yuan; Parikshak, Neelroop N.; Hernandez, Israel A.; Marchetto, Maria C.; Baker, Dylan K.; Lu, Daning; Hinman, Cassidy R.; Lowe, Jennifer K.; Wexler, Eric M.; Muotri, Alysson R.; Gage, Fred H.; Kosik, Kenneth S.; Geschwind, Daniel H.

2014-01-01

Summary Neural stem cells have been adopted to model a wide range of neuropsychiatric conditions in vitro. However, how well such models correspond to in vivo brain has not been evaluated in an unbiased, comprehensive manner. We used transcriptomic analyses to compare in vitro systems to developing human fetal brain and observed strong conservation of in vivo gene expression and network architecture in differentiating primary human neural progenitor cells (phNPCs). Conserved modules are enriched in genes associated with ASD, supporting the utility of phNPCs for studying neuropsychiatric disease. We also developed and validated a machine learning approach called CoNTExT that identifies the developmental maturity and regional identity of in vitro models. We observed strong differences between in vitro models, including hiPSC-derived neural progenitors from multiple laboratories. This work provides a systems biology framework for evaluating in vitro systems and supports their value in studying the molecular mechanisms of human neurodevelopmental disease. PMID:24991955

Security Frameworks for Machine-to-Machine Devices and Networks

NASA Astrophysics Data System (ADS)

Demblewski, Michael

Attacks against mobile systems have escalated over the past decade. There have been increases of fraud, platform attacks, and malware. The Internet of Things (IoT) offers a new attack vector for Cybercriminals. M2M contributes to the growing number of devices that use wireless systems for Internet connection. As new applications and platforms are created, old vulnerabilities are transferred to next-generation systems. There is a research gap that exists between the current approaches for security framework development and the understanding of how these new technologies are different and how they are similar. This gap exists because system designers, security architects, and users are not fully aware of security risks and how next-generation devices can jeopardize safety and personal privacy. Current techniques, for developing security requirements, do not adequately consider the use of new technologies, and this weakens countermeasure implementations. These techniques rely on security frameworks for requirements development. These frameworks lack a method for identifying next generation security concerns and processes for comparing, contrasting and evaluating non-human device security protections. This research presents a solution for this problem by offering a novel security framework that is focused on the study of the "functions and capabilities" of M2M devices and improves the systems development life cycle for the overall IoT ecosystem.

A Cognitive Framework for Understanding Barriers to the Productive Use of a Diabetes Home Telemedicine System

PubMed Central

Kaufman, David R.; Starren, Justin; Patel, Vimla L.; Morin, Philip C.; Hilliman, Charlyn; Pevzner, Jenia; Weinstock, Ruth S.; Goland, Robin; Shea, Steven

2003-01-01

Telemedicine has the potential to transcend geographic and socio-cultural barriers to the delivery of high quality health care to the medically underserved populations. However, there are significant cognitive and usability barriers. This paper presents a multifaceted cognitive evaluation of the IDEATel diabetes education and telemedicine program. The evaluation included a cognitive walkthrough analysis to characterize task complexity and identify potential problems as well as field usability testing in patients’ homes. The study revealed dimensions of the interface that impeded optimal access to system resources. In addition, we found significant obstacles corresponding to perceptual-motoric skills, mental models of the system, and health literacy. The objective of this work is to contribute to a design framework so that participants with a wide range of skills can better manage their chronic illnesses. PMID:14728194

Fuzzy logic based sensor performance evaluation of vehicle mounted metal detector systems

NASA Astrophysics Data System (ADS)

Abeynayake, Canicious; Tran, Minh D.

2015-05-01

Vehicle Mounted Metal Detector (VMMD) systems are widely used for detection of threat objects in humanitarian demining and military route clearance scenarios. Due to the diverse nature of such operational conditions, operational use of VMMD without a proper understanding of its capability boundaries may lead to heavy causalities. Multi-criteria fitness evaluations are crucial for determining capability boundaries of any sensor-based demining equipment. Evaluation of sensor based military equipment is a multi-disciplinary topic combining the efforts of researchers, operators, managers and commanders having different professional backgrounds and knowledge profiles. Information acquired through field tests usually involves uncertainty, vagueness and imprecision due to variations in test and evaluation conditions during a single test or series of tests. This report presents a fuzzy logic based methodology for experimental data analysis and performance evaluation of VMMD. This data evaluation methodology has been developed to evaluate sensor performance by consolidating expert knowledge with experimental data. A case study is presented by implementing the proposed data analysis framework in a VMMD evaluation scenario. The results of this analysis confirm accuracy, practicability and reliability of the fuzzy logic based sensor performance evaluation framework.

Systems theory as a framework for examining a college campus-based support program for the former foster youth.

PubMed

Schelbe, Lisa; Randolph, Karen A; Yelick, Anna; Cheatham, Leah P; Groton, Danielle B

2018-01-01

Increased attention to former foster youth pursuing post-secondary education has resulted in the creation of college campus based support programs to address their need. However, limited empirical evidence and theoretical knowledge exist about these programs. This study seeks to describe the application of systems theory as a framework for examining a college campus based support program for former foster youth. In-depth semi-structured interviews were conducted with 32 program stakeholders including students, mentors, collaborative members, and independent living program staff. Using qualitative data analysis software, holistic coding techniques were employed to analyze interview transcripts. Then applying principles of extended case method using systems theory, data were analyzed. Findings suggest systems theory serves as a framework for understanding the functioning of a college campus based support program. The theory's concepts help delineate program components and roles of stakeholders; outline boundaries between and interactions among stakeholders; and identify program strengths and weakness. Systems theory plays an important role in identifying intervention components and providing a structure through which to identify and understand program elements as a part of the planning process. This study highlights the utility of systems theory as a framework for program planning and evaluation.

Digital evaluation of sitting posture comfort in human-vehicle system under Industry 4.0 framework

NASA Astrophysics Data System (ADS)

Tao, Qing; Kang, Jinsheng; Sun, Wenlei; Li, Zhaobo; Huo, Xiao

2016-09-01

Most of the previous studies on the vibration ride comfort of the human-vehicle system were focused only on one or two aspects of the investigation. A hybrid approach which integrates all kinds of investigation methods in real environment and virtual environment is described. The real experimental environment includes the WBV(whole body vibration) test, questionnaires for human subjective sensation and motion capture. The virtual experimental environment includes the theoretical calculation on simplified 5-DOF human body vibration model, the vibration simulation and analysis within ADAMS/VibrationTM module, and the digital human biomechanics and occupational health analysis in Jack software. While the real experimental environment provides realistic and accurate test results, it also serves as core and validation for the virtual experimental environment. The virtual experimental environment takes full advantages of current available vibration simulation and digital human modelling software, and makes it possible to evaluate the sitting posture comfort in a human-vehicle system with various human anthropometric parameters. How this digital evaluation system for car seat comfort design is fitted in the Industry 4.0 framework is also proposed.

Linking departmental priorities to knowledge management: the experiences of Santa Cruz County's Human Services Department.

PubMed

Lindberg, Arley

2012-01-01

Federal welfare reform, local service collaborations, and the evolution of statewide information systems inspired agency interest in evidence-informed practice and knowledge sharing systems. Four agency leaders, including the Director, Deputy Director, Director of Planning and Evaluation, and Staff Development Program Manager championed the development of a learning organization based on knowledge management throughout the agency. Internal department restructuring helped to strengthen the Planning and Evaluation, Staff Development, and Personnel units, which have become central to supporting knowledge sharing activities. The Four Pillars of Knowledge framework was designed to capture agency directions in relationship to future knowledge management goals. Featuring People, Practice, Technology and Budget, the framework links the agency's services, mission and goals to the process of becoming a learning organization. Built through an iterative process, the framework was created by observing existing activities in each department rather than being designed from the top down. Knowledge management can help the department to fulfill its mission despite reduced resources. Copyright © Taylor & Francis Group, LLC

OECD Reviews of Evaluation and Assessment in Education: Australia

ERIC Educational Resources Information Center

Santiago, Paulo; Donaldson, Graham; Herman, Joan; Shewbridge, Claire

2011-01-01

This report for Australia forms part of the Organisation for Economic Cooperation and Development (OECD) Review on Evaluation and Assessment Frameworks for Improving School Outcomes. The purpose of the Review is to explore how systems of evaluation and assessment can be used to improve the quality, equity and efficiency of school education. The…

Towards an IMU Evaluation Framework for Human Body Tracking.

PubMed

Venek, Verena; Kremser, Wolfgang; Schneider, Cornelia

2018-01-01

Existing full-body tracking systems, which use Inertial Measurement Units (IMUs) as sensing unit, require expert knowledge for setup and data collection. Thus, the daily application for human body tracking is difficult. In particular, in the field of active and assisted living (AAL), tracking human movements would enable novel insights not only into the quantity but also into the quality of human movement, for example by monitoring functional training. While the current market offers a wide range of products with vastly different properties, literature lacks guidelines for choosing IMUs for body tracking applications. Therefore, this paper introduces developments towards an IMU evaluation framework for human body tracking which compares IMUs against five requirement areas that consider device features and data quality. The data quality is assessed by conducting a static and a dynamic error analysis. In a first application to four IMUs of different component consumption, the IMU evaluation framework convinced as promising tool for IMU selection.

Smart Roadside System for Driver Assistance and Safety Warnings: Framework and Applications

PubMed Central

Jang, Jeong Ah; Kim, Hyun Suk; Cho, Han Byeog

2011-01-01

The use of newly emerging sensor technologies in traditional roadway systems can provide real-time traffic services to drivers through Telematics and Intelligent Transport Systems (ITSs). This paper introduces a smart roadside system that utilizes various sensors for driver assistance and traffic safety warnings. This paper shows two road application models for a smart roadside system and sensors: a red-light violation warning system for signalized intersections, and a speed advisory system for highways. Evaluation results for the two services are then shown using a micro-simulation method. In the given real-time applications for drivers, the framework and certain algorithms produce a very efficient solution with respect to the roadway type features and sensor type use. PMID:22164025

An Evaluation of the Chemical Origin of Life as a Context for Teaching Undergraduate Chemistry

ERIC Educational Resources Information Center

Venkataraman, Bhawani

2011-01-01

The chemical origin of life on earth has been used as a conceptual framework in an introductory, undergraduate chemistry course. The course explores the sequence of events through which life is believed to have emerged, from atoms to molecules to macromolecular systems, and uses this framework to teach basic chemical concepts. The results of this…

SupportNet for Frontline Behavioral Health Providers

DTIC Science & Technology

2014-06-30

social -cognitive theory perspective ( Bandura , 1997), the proposed website and integrated treatment would enhance the perceived social environmental...Objective 2: We will evaluate the utility of social cognitive theory as a framework for understanding the stress process for military mental health...healthcare providers. SupportNet, based on the theoretical framework of social cognitive theory , utilizes web-based support system with coaching to

Evaluating the Measuring Properties of the Principal Instructional Management Rating Scale in the Chinese Educational System: Implications for Measuring School Leadership

ERIC Educational Resources Information Center

Antoniou, Panayiotis; Lu, Mohan

2018-01-01

During the last 25 years researchers have proposed a number of conceptual frameworks to measure the various functions of instructional leadership. One of the most frequently used frameworks is the Principal Instructional Management Rating Scale (PIMRS). Despite the great number of studies employing the PIMRS, evidence for its reliability and…

An Experimental Design of a Foundational Framework for the Application of Affective Computing to Soaring Flight Simulation and Training

ERIC Educational Resources Information Center

Moon, Shannon

2017-01-01

In the absence of tools for intelligent tutoring systems for soaring flight simulation training, this study evaluated a framework foundation to measure pilot performance, affect, and physiological response to training in real-time. Volunteers were asked to perform a series of flight tasks selected from Federal Aviation Administration Practical…

SparkMed: a framework for dynamic integration of multimedia medical data into distributed m-Health systems.

PubMed

Constantinescu, Liviu; Kim, Jinman; Feng, David Dagan

2012-01-01

With the advent of 4G and other long-term evolution (LTE) wireless networks, the traditional boundaries of patient record propagation are diminishing as networking technologies extend the reach of hospital infrastructure and provide on-demand mobile access to medical multimedia data. However, due to legacy and proprietary software, storage and decommissioning costs, and the price of centralization and redevelopment, it remains complex, expensive, and often unfeasible for hospitals to deploy their infrastructure for online and mobile use. This paper proposes the SparkMed data integration framework for mobile healthcare (m-Health), which significantly benefits from the enhanced network capabilities of LTE wireless technologies, by enabling a wide range of heterogeneous medical software and database systems (such as the picture archiving and communication systems, hospital information system, and reporting systems) to be dynamically integrated into a cloud-like peer-to-peer multimedia data store. Our framework allows medical data applications to share data with mobile hosts over a wireless network (such as WiFi and 3G), by binding to existing software systems and deploying them as m-Health applications. SparkMed integrates techniques from multimedia streaming, rich Internet applications (RIA), and remote procedure call (RPC) frameworks to construct a Self-managing, Pervasive Automated netwoRK for Medical Enterprise Data (SparkMed). Further, it is resilient to failure, and able to use mobile and handheld devices to maintain its network, even in the absence of dedicated server devices. We have developed a prototype of the SparkMed framework for evaluation on a radiological workflow simulation, which uses SparkMed to deploy a radiological image viewer as an m-Health application for telemedical use by radiologists and stakeholders. We have evaluated our prototype using ten devices over WiFi and 3G, verifying that our framework meets its two main objectives: 1) interactive delivery of medical multimedia data to mobile devices; and 2) attaching to non-networked medical software processes without significantly impacting their performance. Consistent response times of under 500 ms and graphical frame rates of over 5 frames per second were observed under intended usage conditions. Further, overhead measurements displayed linear scalability and low resource requirements.

Integrative evaluation for sustainable decisions of urban wastewater system management under uncertainty

NASA Astrophysics Data System (ADS)

Hadjimichael, A.; Corominas, L.; Comas, J.

2017-12-01

With sustainable development as their overarching goal, urban wastewater system (UWS) managers need to take into account multiple social, economic, technical and environmental facets related to their decisions. In this complex decision-making environment, uncertainty can be formidable. It is present both in the ways the system is interpreted stochastically, but also in its natural ever-shifting behavior. This inherent uncertainty suggests that wiser decisions would be made under an adaptive and iterative decision-making regime. No decision-support framework has been presented in the literature to effectively addresses all these needs. The objective of this work is to describe such a conceptual framework to evaluate and compare alternative solutions for various UWS challenges within an adaptive management structure. Socio-economic aspects such as externalities are taken into account, along with other traditional criteria as necessary. Robustness, reliability and resilience analyses test the performance of the system against present and future variability. A valuation uncertainty analysis incorporates uncertain valuation assumptions in the decision-making process. The framework is demonstrated with an application to a case study presenting a typical problem often faced by managers: poor river water quality, increasing population, and more stringent water quality legislation. The application of the framework made use of: i) a cost-benefit analysis including monetized environmental benefits and damages; ii) a robustness analysis of system performance against future conditions; iii) reliability and resilience analyses of the system given contextual variability; and iv) a valuation uncertainty analysis of model parameters. The results suggest that the installation of bigger volumes would give rise to increased benefits despite larger capital costs, as well as increased robustness and resilience. Population numbers appear to affect the estimated benefits most, followed by electricity prices and climate change projections. The presented framework is expected to be a valuable tool for the next generation of UWS decision-making and the application demonstrates a novel and valuable integration of metrics and methods for UWS analysis.

A Decentralized Compositional Framework for Dependable Decision Process in Self-Managed Cyber Physical Systems

PubMed Central

Hou, Kun-Mean; Zhang, Zhan

2017-01-01

Cyber Physical Systems (CPSs) need to interact with the changeable environment under various interferences. To provide continuous and high quality services, a self-managed CPS should automatically reconstruct itself to adapt to these changes and recover from failures. Such dynamic adaptation behavior introduces systemic challenges for CPS design, advice evaluation and decision process arrangement. In this paper, a formal compositional framework is proposed to systematically improve the dependability of the decision process. To guarantee the consistent observation of event orders for causal reasoning, this work first proposes a relative time-based method to improve the composability and compositionality of the timing property of events. Based on the relative time solution, a formal reference framework is introduced for self-managed CPSs, which includes a compositional FSM-based actor model (subsystems of CPS), actor-based advice and runtime decomposable decisions. To simplify self-management, a self-similar recursive actor interface is proposed for decision (actor) composition. We provide constraints and seven patterns for the composition of reliability and process time requirements. Further, two decentralized decision process strategies are proposed based on our framework, and we compare the reliability with the static strategy and the centralized processing strategy. The simulation results show that the one-order feedback strategy has high reliability, scalability and stability against the complexity of decision and random failure. This paper also shows a way to simplify the evaluation for dynamic system by improving the composability and compositionality of the subsystem. PMID:29120357

A Decentralized Compositional Framework for Dependable Decision Process in Self-Managed Cyber Physical Systems.

PubMed

Zhou, Peng; Zuo, Decheng; Hou, Kun-Mean; Zhang, Zhan

2017-11-09

Cyber Physical Systems (CPSs) need to interact with the changeable environment under various interferences. To provide continuous and high quality services, a self-managed CPS should automatically reconstruct itself to adapt to these changes and recover from failures. Such dynamic adaptation behavior introduces systemic challenges for CPS design, advice evaluation and decision process arrangement. In this paper, a formal compositional framework is proposed to systematically improve the dependability of the decision process. To guarantee the consistent observation of event orders for causal reasoning, this work first proposes a relative time-based method to improve the composability and compositionality of the timing property of events. Based on the relative time solution, a formal reference framework is introduced for self-managed CPSs, which includes a compositional FSM-based actor model (subsystems of CPS), actor-based advice and runtime decomposable decisions. To simplify self-management, a self-similar recursive actor interface is proposed for decision (actor) composition. We provide constraints and seven patterns for the composition of reliability and process time requirements. Further, two decentralized decision process strategies are proposed based on our framework, and we compare the reliability with the static strategy and the centralized processing strategy. The simulation results show that the one-order feedback strategy has high reliability, scalability and stability against the complexity of decision and random failure. This paper also shows a way to simplify the evaluation for dynamic system by improving the composability and compositionality of the subsystem.

Information-Theoretic Approach May Shed a Light to a Better Understanding and Sustaining the Integrity of Ecological-Societal Systems under Changing Climate

NASA Astrophysics Data System (ADS)

Kim, J.

2016-12-01

Considering high levels of uncertainty, epistemological conflicts over facts and values, and a sense of urgency, normal paradigm-driven science will be insufficient to mobilize people and nation toward sustainability. The conceptual framework to bridge the societal system dynamics with that of natural ecosystems in which humanity operates remains deficient. The key to understanding their coevolution is to understand `self-organization.' Information-theoretic approach may shed a light to provide a potential framework which enables not only to bridge human and nature but also to generate useful knowledge for understanding and sustaining the integrity of ecological-societal systems. How can information theory help understand the interface between ecological systems and social systems? How to delineate self-organizing processes and ensure them to fulfil sustainability? How to evaluate the flow of information from data through models to decision-makers? These are the core questions posed by sustainability science in which visioneering (i.e., the engineering of vision) is an essential framework. Yet, visioneering has neither quantitative measure nor information theoretic framework to work with and teach. This presentation is an attempt to accommodate the framework of self-organizing hierarchical open systems with visioneering into a common information-theoretic framework. A case study is presented with the UN/FAO's communal vision of climate-smart agriculture (CSA) which pursues a trilemma of efficiency, mitigation, and resilience. Challenges of delineating and facilitating self-organizing systems are discussed using transdisciplinary toold such as complex systems thinking, dynamic process network analysis and multi-agent systems modeling. Acknowledgments: This study was supported by the Korea Meteorological Administration Research and Development Program under Grant KMA-2012-0001-A (WISE project).

TrustBuilder2: A Reconfigurable Framework for Trust Negotiation

NASA Astrophysics Data System (ADS)

Lee, Adam J.; Winslett, Marianne; Perano, Kenneth J.

To date, research in trust negotiation has focused mainly on the theoretical aspects of the trust negotiation process, and the development of proof of concept implementations. These theoretical works and proofs of concept have been quite successful from a research perspective, and thus researchers must now begin to address the systems constraints that act as barriers to the deployment of these systems. To this end, we present TrustBuilder2, a fully-configurable and extensible framework for prototyping and evaluating trust negotiation systems. TrustBuilder2 leverages a plug-in based architecture, extensible data type hierarchy, and flexible communication protocol to provide a framework within which numerous trust negotiation protocols and system configurations can be quantitatively analyzed. In this paper, we discuss the design and implementation of TrustBuilder2, study its performance, examine the costs associated with flexible authorization systems, and leverage this knowledge to identify potential topics for future research, as well as a novel method for attacking trust negotiation systems.

Design and evaluation of an onboard computer-based information system for aircraft

NASA Technical Reports Server (NTRS)

Rouse, S. H.; Rouse, W. B.; Hammer, J. M.

1982-01-01

Information seeking by human operators of technical systems is considered. Types of information and forms of presentation are discussed and important issues reviewed. This broad discussion provides a framework within which flight management is considered. The design of an onboard computer-based information system for aircraft is discussed. The aiding possibilities of a computer-based system are emphasized. Results of an experimental evaluation of a prototype system are presented. It is concluded that a computer-based information system can substantially lessen the frequency of human errors.

Overview of space propulsion systems for identifying nondestructive evaluation and health monitoring opportunities

NASA Technical Reports Server (NTRS)

Generazio, Edward R.

1991-01-01

The next generation of space propulsion systems will be designed to incorporate advanced health monitoring and nondestructive inspection capabilities. As a guide to help the nondestructive evaluation (NDE) community impact the development of these space propulsion systems, several questions should be addressed. An overview of background and current information on space propulsion systems at both the programmatic and technical levels is provided. A framework is given that will assist the NDE community in addressing key questions raised during the 2 to 5 April 1990 meeting of the Joint Army-Navy-NASA-Air Force (JANNAF) Nondestructive Evaluation Subcommittee (NDES).

A SWOT analysis of the organization and financing of the Danish health care system.

PubMed

Christiansen, Terkel

2002-02-01

The organization and financing of the Danish health care system was evaluated within a framework of a SWOT analysis (analysis of Strengths, Weaknesses, Opportunities and Threats) by a panel of five members with a background in health economics. The present paper describes the methods and materials used for the evaluation: selection of panel members, structure of the evaluation task according to the health care triangle model, selection of background material consisting of documents and literature on the Danish health care system, and a 1-week study visit.

Towards a balanced performance measurement system in a public health care organization.

PubMed

Yuen, Peter P; Ng, Artie W

2012-01-01

This article attempts to devise an integrated performance measurement framework to assess the Hong Kong Hospital Authority (HA) management system by harnessing previous performance measurement systems. An integrated evaluative framework based on the balanced score card (BSC) was developed and applied using the case study method and longitudinal data to evaluate the HA's performance management system. The authors unveil evolving HA performance indicators (P1). Despite the HA staffs explicit quality emphasis, cost control remains the primary focus in their performance measurements. RESEARCH LHNITATIONS/IMPLICATIONS: Data used in this study are from secondary sources, disclosed mostly by HA staff. This study shows public sector staff often attach too much importance to cost control and easily measurable activities at the expense of quality and other less easily measurable attributes'. A balanced performance measurement system, linked to health targets, with a complementary budgeting process that supports pertinent resource allocation is yet to be implemented in Hong Kong's public hospitals.

Coupling between a multi-physics workflow engine and an optimization framework

NASA Astrophysics Data System (ADS)

Di Gallo, L.; Reux, C.; Imbeaux, F.; Artaud, J.-F.; Owsiak, M.; Saoutic, B.; Aiello, G.; Bernardi, P.; Ciraolo, G.; Bucalossi, J.; Duchateau, J.-L.; Fausser, C.; Galassi, D.; Hertout, P.; Jaboulay, J.-C.; Li-Puma, A.; Zani, L.

2016-03-01

A generic coupling method between a multi-physics workflow engine and an optimization framework is presented in this paper. The coupling architecture has been developed in order to preserve the integrity of the two frameworks. The objective is to provide the possibility to replace a framework, a workflow or an optimizer by another one without changing the whole coupling procedure or modifying the main content in each framework. The coupling is achieved by using a socket-based communication library for exchanging data between the two frameworks. Among a number of algorithms provided by optimization frameworks, Genetic Algorithms (GAs) have demonstrated their efficiency on single and multiple criteria optimization. Additionally to their robustness, GAs can handle non-valid data which may appear during the optimization. Consequently GAs work on most general cases. A parallelized framework has been developed to reduce the time spent for optimizations and evaluation of large samples. A test has shown a good scaling efficiency of this parallelized framework. This coupling method has been applied to the case of SYCOMORE (SYstem COde for MOdeling tokamak REactor) which is a system code developed in form of a modular workflow for designing magnetic fusion reactors. The coupling of SYCOMORE with the optimization platform URANIE enables design optimization along various figures of merit and constraints.

Achieving Fairness in Data Fusion Performance Evaluation Development

DTIC Science & Technology

2006-04-30

display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM-YYYY) 2.REPORT TYPE 13. DATES...for and evaluation (T&E) organization is how to affordably achieve fairness in the application of its PE systems. Our PE framework provides a ...as a fusion system design balances probability of mission success with complexity/cost). The key issue for and evaluation (T&E) organization is how to

The Researching on Evaluation of Automatic Voltage Control Based on Improved Zoning Methodology

NASA Astrophysics Data System (ADS)

Xiao-jun, ZHU; Ang, FU; Guang-de, DONG; Rui-miao, WANG; De-fen, ZHU

2018-03-01

According to the present serious phenomenon of increasing size and structure of power system, hierarchically structured automatic voltage control(AVC) has been the researching spot. In the paper, the reduced control model is built and the adaptive reduced control model is researched to improve the voltage control effect. The theories of HCSD, HCVS, SKC and FCM are introduced and the effect on coordinated voltage regulation caused by different zoning methodologies is also researched. The generic framework for evaluating performance of coordinated voltage regulation is built. Finally, the IEEE-96 stsyem is used to divide the network. The 2383-bus Polish system is built to verify that the selection of a zoning methodology affects not only the coordinated voltage regulation operation, but also its robustness to erroneous data and proposes a comprehensive generic framework for evaluating its performance. The New England 39-bus network is used to verify the adaptive reduced control models’ performance.

[Design of warm-acupuncture technique training evaluation device].

PubMed

Gao, Ming; Xu, Gang; Yang, Huayuan; Liu, Tangyi; Tang, Wenchao

2017-01-12

To design a warm-acupuncture teaching instrument to train and evaluate its manipulation. We refer to the principle and technical operation characteristics of traditional warm-acupuncture, as well as the mechanical design and single-chip microcomputer technology. The device is consisted of device noumenon, universal acupoints simulator, vibration reset system and circuit control system, including frame, platform framework, the swing framework, universal acupoints simulator, vibration reset outfit, operation time circuit, acupuncture sensation display, and vibration control circuit, etc. It can be used to train needle inserting with different angles and moxa rubbing and loading. It displays whether a needle point meets the location required. We determine whether the moxa group on a needle handle is easy to fall off through vibration test, and operation time is showed. The device can objectively help warm-acupuncture training and evaluation so as to promote its clinical standardization manipulation.

Curriculum Development in History Using Systems Approach

ERIC Educational Resources Information Center

Acun, Ramazan

2011-01-01

This work provides a conceptual framework for developing coherent history curricula at university level. It can also be used for evaluating existing curricula in terms of coherence. For this purpose, two models that are closely inter-connected called History Education System (Tarih Egitim Sistemi or TES) and History Research System (Tarih…

Developing a holistic policy and intervention framework for global mental health.

PubMed

Khenti, Akwatu; Fréel, Stéfanie; Trainor, Ruth; Mohamoud, Sirad; Diaz, Pablo; Suh, Erica; Bobbili, Sireesha J; Sapag, Jaime C

2016-02-01

There are significant gaps in the accessibility and quality of mental health services around the globe. A wide range of institutions are addressing the challenges, but there is limited reflection and evaluation on the various approaches, how they compare with each other, and conclusions regarding the most effective approach for particular settings. This article presents a framework for global mental health capacity building that could potentially serve as a promising or best practice in the field. The framework is the outcome of a decade of collaborative global health work at the Centre for Addiction and Mental Health (CAMH) (Ontario, Canada). The framework is grounded in scientific evidence, relevant learning and behavioural theories and the underlying principles of health equity and human rights. Grounded in CAMH's research, programme evaluation and practical experience in developing and implementing mental health capacity building interventions, this article presents the iterative learning process and impetus that formed the basis of the framework. A developmental evaluation (Patton M.2010. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford Press.) approach was used to build the framework, as global mental health collaboration occurs in complex or uncertain environments and evolving learning systems. A multilevel framework consists of five central components: (1) holistic health, (2) cultural and socioeconomic relevance, (3) partnerships, (4) collaborative action-based education and learning and (5) sustainability. The framework's practical application is illustrated through the presentation of three international case studies and four policy implications. Lessons learned, limitations and future opportunities are also discussed. The holistic policy and intervention framework for global mental health reflects an iterative learning process that can be applied and scaled up across different settings through appropriate modifications. © The Author 2015. Published by Oxford University Press in association with The London School of Hygiene and Tropical Medicine.

Test and evaluation of vehicle platooning proof-of-concept based on cooperative adaptive cruise control

DOT National Transportation Integrated Search

2017-04-01

This report presents a high-level test and evaluation framework for cooperative driving automation systems that have the potential to significantly improve mobility and enhance traffic flow stability with better safety. It focuses on the test and eva...

FAST: framework for heterogeneous medical image computing and visualization.

PubMed

Smistad, Erik; Bozorgi, Mohammadmehdi; Lindseth, Frank

2015-11-01

Computer systems are becoming increasingly heterogeneous in the sense that they consist of different processors, such as multi-core CPUs and graphic processing units. As the amount of medical image data increases, it is crucial to exploit the computational power of these processors. However, this is currently difficult due to several factors, such as driver errors, processor differences, and the need for low-level memory handling. This paper presents a novel FrAmework for heterogeneouS medical image compuTing and visualization (FAST). The framework aims to make it easier to simultaneously process and visualize medical images efficiently on heterogeneous systems. FAST uses common image processing programming paradigms and hides the details of memory handling from the user, while enabling the use of all processors and cores on a system. The framework is open-source, cross-platform and available online. Code examples and performance measurements are presented to show the simplicity and efficiency of FAST. The results are compared to the insight toolkit (ITK) and the visualization toolkit (VTK) and show that the presented framework is faster with up to 20 times speedup on several common medical imaging algorithms. FAST enables efficient medical image computing and visualization on heterogeneous systems. Code examples and performance evaluations have demonstrated that the toolkit is both easy to use and performs better than existing frameworks, such as ITK and VTK.

Individual performance review in hospital practice: the development of a framework and evaluation of doctors' attitudes to its value and implementation.

PubMed

Trebble, T M; Cruickshank, L; Hockey, P M; Heyworth, N; Powell, T; Clarke, N

2013-11-01

Appraisal, or independent performance review (IPR) is used in human resources management in the commercial and public sectors to evaluate the performance of an employee against agreed local organisational expectations and objectives, and to identify their requirements for development and effective management. IPR for NHS consultants may provide essential information for job planning, contribute towards medical appraisal for revalidation, and facilitate productivity and quality improvement. To develop a framework for IPR for consultants, and to determine attitudes on its value, process and content. Information from commercial, public and voluntary sector models and published and other literature sources were used to develop an IPR framework. This was assessed through a three-cycle action research methodology involving qualitative interviews with 22 consultants (predominantly with medical management roles). The domains of the IPR framework included: (1) performance against objectives; (2) behaviour and leadership; (3) talent management; (4) agreed future objectives. A number of themes were identified from the consultant interviews including: ineffective current appraisal systems reflecting a lack of valid performance data and allotted time; a lack of empowerment of medical managers to address performance issues; IPR as a more explicit system, offering value in evaluating doctors performance; and the dependence of successful implementation on the engagement of the Trust executive. IPR may have value for performance evaluation of consultants, contributing toward job planning and complementing medical appraisal. Support by their employing organisation and engagement with medical managers in design and implementation is likely to be essential.

Framework for Deploying a Virtualized Computing Environment for Collaborative and Secure Data Analytics

PubMed Central

Meyer, Adrian; Green, Laura; Faulk, Ciearro; Galla, Stephen; Meyer, Anne-Marie

2016-01-01

Introduction: Large amounts of health data generated by a wide range of health care applications across a variety of systems have the potential to offer valuable insight into populations and health care systems, but robust and secure computing and analytic systems are required to leverage this information. Framework: We discuss our experiences deploying a Secure Data Analysis Platform (SeDAP), and provide a framework to plan, build and deploy a virtual desktop infrastructure (VDI) to enable innovation, collaboration and operate within academic funding structures. It outlines 6 core components: Security, Ease of Access, Performance, Cost, Tools, and Training. Conclusion: A platform like SeDAP is not simply successful through technical excellence and performance. It’s adoption is dependent on a collaborative environment where researchers and users plan and evaluate the requirements of all aspects. PMID:27683665

Overdenture retaining bar stress distribution: a finite-element analysis.

PubMed

Caetano, Conrado Reinoldes; Mesquita, Marcelo Ferraz; Consani, Rafael Leonardo Xediek; Correr-Sobrinho, Lourenço; Dos Santos, Mateus Bertolini Fernandes

2015-05-01

Evaluate the stress distribution on the peri-implant bone tissue and prosthetic components of bar-clip retaining systems for overdentures presenting different implant inclinations, vertical misfit and framework material. Three-dimensional models of a jaw and an overdenture retained by two implants and a bar-clip attachment were modeled using specific software (SolidWorks 2010). The studied variables were: latero-lateral inclination of one implant (-10°, -5°, 0°, +5°, +10°); vertical misfit on the other implant (50, 100, 200 µm); and framework material (Au type IV, Ag-Pd, Ti cp, Co-Cr). Solid models were imported into mechanical simulation software (ANSYS Workbench 11). All nodes on the bone's external surface were constrained and a displacement was applied to simulate the settling of the framework on the ill-fitted component. Von Mises stress for the prosthetic components and maximum principal stress to the bone tissue were evaluated. The +10° inclination presented the worst biomechanical behavior, promoting the highest stress values on the bar framework and peri-implant bone tissue. The -5° group presented the lowest stress values on the prosthetic components and the lowest stress value on peri-implant bone tissue was observed in -10°. Increased vertical misfit caused an increase on the stress values in all evaluated structures. Stiffer framework materials caused a considerable stress increase in the framework itself, prosthetic screw of the fitted component and peri-implant bone tissue. Inclination of one implant associated with vertical misfit caused a relevant effect on the stress distribution in bar-clip retained overdentures. Different framework materials promoted increased levels of stress in all the evaluated structures.

Framework to Define Structure and Boundaries of Complex Health Intervention Systems: The ALERT Project

PubMed Central

Boriani, Elena; Esposito, Roberto; Frazzoli, Chiara; Fantke, Peter; Hald, Tine; Rüegg, Simon R.

2017-01-01

Health intervention systems are complex and subject to multiple variables in different phases of implementation. This constitutes a concrete challenge for the application of translational science in real life. Complex systems as health-oriented interventions call for interdisciplinary approaches with carefully defined system boundaries. Exploring individual components of such systems from different viewpoints gives a wide overview and helps to understand the elements and the relationships that drive actions and consequences within the system. In this study, we present an application and assessment of a framework with focus on systems and system boundaries of interdisciplinary projects. As an example on how to apply our framework, we analyzed ALERT [an integrated sensors and biosensors’ system (BEST) aimed at monitoring the quality, health, and traceability of the chain of the bovine milk], a multidisciplinary and interdisciplinary project based on the application of measurable biomarkers at strategic points of the milk chain for improved food security (including safety), human, and ecosystem health (1). In fact, the European food safety framework calls for science-based support to the primary producers’ mandate for legal, scientific, and ethical responsibility in food supply. Because of its multidisciplinary and interdisciplinary approach involving human, animal, and ecosystem health, ALERT can be considered as a One Health project. Within the ALERT context, we identified the need to take into account the main actors, interactions, and relationships of stakeholders to depict a simplified skeleton of the system. The framework can provide elements to highlight how and where to improve the project development when project evaluations are required. PMID:28804707

Framework to Define Structure and Boundaries of Complex Health Intervention Systems: The ALERT Project.

PubMed

Boriani, Elena; Esposito, Roberto; Frazzoli, Chiara; Fantke, Peter; Hald, Tine; Rüegg, Simon R

2017-01-01

Health intervention systems are complex and subject to multiple variables in different phases of implementation. This constitutes a concrete challenge for the application of translational science in real life. Complex systems as health-oriented interventions call for interdisciplinary approaches with carefully defined system boundaries. Exploring individual components of such systems from different viewpoints gives a wide overview and helps to understand the elements and the relationships that drive actions and consequences within the system. In this study, we present an application and assessment of a framework with focus on systems and system boundaries of interdisciplinary projects. As an example on how to apply our framework, we analyzed ALERT [an integrated sensors and biosensors' system (BEST) aimed at monitoring the quality, health, and traceability of the chain of the bovine milk], a multidisciplinary and interdisciplinary project based on the application of measurable biomarkers at strategic points of the milk chain for improved food security (including safety), human, and ecosystem health (1). In fact, the European food safety framework calls for science-based support to the primary producers' mandate for legal, scientific, and ethical responsibility in food supply. Because of its multidisciplinary and interdisciplinary approach involving human, animal, and ecosystem health, ALERT can be considered as a One Health project. Within the ALERT context, we identified the need to take into account the main actors, interactions, and relationships of stakeholders to depict a simplified skeleton of the system. The framework can provide elements to highlight how and where to improve the project development when project evaluations are required.

Benchmarking Diagnostic Algorithms on an Electrical Power System Testbed

NASA Technical Reports Server (NTRS)

Kurtoglu, Tolga; Narasimhan, Sriram; Poll, Scott; Garcia, David; Wright, Stephanie

2009-01-01

Diagnostic algorithms (DAs) are key to enabling automated health management. These algorithms are designed to detect and isolate anomalies of either a component or the whole system based on observations received from sensors. In recent years a wide range of algorithms, both model-based and data-driven, have been developed to increase autonomy and improve system reliability and affordability. However, the lack of support to perform systematic benchmarking of these algorithms continues to create barriers for effective development and deployment of diagnostic technologies. In this paper, we present our efforts to benchmark a set of DAs on a common platform using a framework that was developed to evaluate and compare various performance metrics for diagnostic technologies. The diagnosed system is an electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The paper presents the fundamentals of the benchmarking framework, the ADAPT system, description of faults and data sets, the metrics used for evaluation, and an in-depth analysis of benchmarking results obtained from testing ten diagnostic algorithms on the ADAPT electrical power system testbed.

Risk Assessment for Mobile Systems Through a Multilayered Hierarchical Bayesian Network.

PubMed

Li, Shancang; Tryfonas, Theo; Russell, Gordon; Andriotis, Panagiotis

2016-08-01

Mobile systems are facing a number of application vulnerabilities that can be combined together and utilized to penetrate systems with devastating impact. When assessing the overall security of a mobile system, it is important to assess the security risks posed by each mobile applications (apps), thus gaining a stronger understanding of any vulnerabilities present. This paper aims at developing a three-layer framework that assesses the potential risks which apps introduce within the Android mobile systems. A Bayesian risk graphical model is proposed to evaluate risk propagation in a layered risk architecture. By integrating static analysis, dynamic analysis, and behavior analysis in a hierarchical framework, the risks and their propagation through each layer are well modeled by the Bayesian risk graph, which can quantitatively analyze risks faced to both apps and mobile systems. The proposed hierarchical Bayesian risk graph model offers a novel way to investigate the security risks in mobile environment and enables users and administrators to evaluate the potential risks. This strategy allows to strengthen both app security as well as the security of the entire system.

A comprehensive health service evaluation and monitoring framework.

PubMed

Reeve, Carole; Humphreys, John; Wakerman, John

2015-12-01

To develop a framework for evaluating and monitoring a primary health care service, integrating hospital and community services. A targeted literature review of primary health service evaluation frameworks was performed to inform the development of the framework specifically for remote communities. Key principles underlying primary health care evaluation were determined and sentinel indicators developed to operationalise the evaluation framework. This framework was then validated with key stakeholders. The framework includes Donabedian's three seminal domains of structure, process and outcomes to determine health service performance. These in turn are dependent on sustainability, quality of patient care and the determinants of health to provide a comprehensive health service evaluation framework. The principles underpinning primary health service evaluation were pertinent to health services in remote contexts. Sentinel indicators were developed to fit the demographic characteristics and health needs of the population. Consultation with key stakeholders confirmed that the evaluation framework was applicable. Data collected routinely by health services can be used to operationalise the proposed health service evaluation framework. Use of an evaluation framework which links policy and health service performance to health outcomes will assist health services to improve performance as part of a continuous quality improvement cycle. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Machine Learning-based discovery of closures for reduced models of dynamical systems

NASA Astrophysics Data System (ADS)

Pan, Shaowu; Duraisamy, Karthik

2017-11-01

Despite the successful application of machine learning (ML) in fields such as image processing and speech recognition, only a few attempts has been made toward employing ML to represent the dynamics of complex physical systems. Previous attempts mostly focus on parameter calibration or data-driven augmentation of existing models. In this work we present a ML framework to discover closure terms in reduced models of dynamical systems and provide insights into potential problems associated with data-driven modeling. Based on exact closure models for linear system, we propose a general linear closure framework from viewpoint of optimization. The framework is based on trapezoidal approximation of convolution term. Hyperparameters that need to be determined include temporal length of memory effect, number of sampling points, and dimensions of hidden states. To circumvent the explicit specification of memory effect, a general framework inspired from neural networks is also proposed. We conduct both a priori and posteriori evaluations of the resulting model on a number of non-linear dynamical systems. This work was supported in part by AFOSR under the project ``LES Modeling of Non-local effects using Statistical Coarse-graining'' with Dr. Jean-Luc Cambier as the technical monitor.

On the complex interaction between mathematics and the sciences of living systems. Comment on "Move me, astonish me...delight my eyes and brain: The Vienna Integrated Model of top-down and bottom-up processes in Art Perception (VIMAP) and corresponding affective, evaluative, and neurophysiological correlates" by Matthew Pelowski et al.

NASA Astrophysics Data System (ADS)

Bellomo, Nicola; Outada, Nisrine

2017-07-01

Cultural framework: Our comment looks at the general framework given by the interactions between the so-called ;soft; and ;hard; sciences. Specifically, it looks at the development of a mathematics for living systems. Our comment aims at showing how the interesting survey [11] can contribute to the aforementioned challenging task.

A framework for improving the quality of health information on the world-wide-web and bettering public (e-)health: the MedCERTAIN approach.

PubMed

Eysenbach, G; Köhler, C; Yihune, G; Lampe, K; Cross, P; Brickley, D

2001-01-01

There has been considerable debate about the variable quality of health information on the world-wide-web and its impact on public health. While central authorities to regulate, control, censor, or centrally approve information, in-formation providers or websites are neither realistic nor desirable, public health professionals are interested in making systems available that direct patient streams to the best available information sources. National governments and medical societies have also recognized their responsibility to help users to identify "good quality" information sources. But what constitutes good quality, and how can such a system be implemented in a decentralized and democratic manner? This paper presents a model which combines aspects of consumer education, encouragement of best practices among information providers, self-labeling and external evaluations. The model is currently being implemented and evaluated in the MedCERTAIN project, funded by the European Union under the Action Plan for Safer Use of the Internet. The aim is to develop a technical and organisational infrastructure for a pilot system that allows consumers to access metainformation about web-sites and health information providers, including disclosure information from health providers and opinions of external evaluators. The paper explains the general conceptual framework of the model and presents preliminary experiences including results from an expert consensus meeting, where the framework was discussed.

Evaluating Educative Temporary Systems

ERIC Educational Resources Information Center

Dickinson, Gary; Lamoureux, Marvin E.

1975-01-01

The framework of temporary systems theory was the basis for forming theoretical units and propositions pertaining to the outcomes of intensive adult education programs. Empirical indicators and research hypotheses were then developed and applied in the 1972 residential program of the Labour College of Canada. (Author)

Standardized and Repeatable Technology Evaluation for Cybersecurity Acquisition

DTIC Science & Technology

2017-02-01

methodology for evaluating cybersecurity technologies. In this report, we introduce the Department of Defense (DoD)-centric and Independent Technology...Evaluation Capability (DITEC), an experimental decision support service within the U.S. DoD which aims to provide a standardized framework for...13 5.3.1 The Technology Matching Tool: A Recommender System for Security Non - Experts

Temporal efficiency evaluation and small-worldness characterization in temporal networks

PubMed Central

Dai, Zhongxiang; Chen, Yu; Li, Junhua; Fam, Johnson; Bezerianos, Anastasios; Sun, Yu

2016-01-01

Numerous real-world systems can be modeled as networks. To date, most network studies have been conducted assuming stationary network characteristics. Many systems, however, undergo topological changes over time. Temporal networks, which incorporate time into conventional network models, are therefore more accurate representations of such dynamic systems. Here, we introduce a novel generalized analytical framework for temporal networks, which enables 1) robust evaluation of the efficiency of temporal information exchange using two new network metrics and 2) quantitative inspection of the temporal small-worldness. Specifically, we define new robust temporal network efficiency measures by incorporating the time dependency of temporal distance. We propose a temporal regular network model, and based on this plus the redefined temporal efficiency metrics and widely used temporal random network models, we introduce a quantitative approach for identifying temporal small-world architectures (featuring high temporal network efficiency both globally and locally). In addition, within this framework, we can uncover network-specific dynamic structures. Applications to brain networks, international trade networks, and social networks reveal prominent temporal small-world properties with distinct dynamic network structures. We believe that the framework can provide further insight into dynamic changes in the network topology of various real-world systems and significantly promote research on temporal networks. PMID:27682314

Temporal efficiency evaluation and small-worldness characterization in temporal networks

NASA Astrophysics Data System (ADS)

Dai, Zhongxiang; Chen, Yu; Li, Junhua; Fam, Johnson; Bezerianos, Anastasios; Sun, Yu

2016-09-01

Numerous real-world systems can be modeled as networks. To date, most network studies have been conducted assuming stationary network characteristics. Many systems, however, undergo topological changes over time. Temporal networks, which incorporate time into conventional network models, are therefore more accurate representations of such dynamic systems. Here, we introduce a novel generalized analytical framework for temporal networks, which enables 1) robust evaluation of the efficiency of temporal information exchange using two new network metrics and 2) quantitative inspection of the temporal small-worldness. Specifically, we define new robust temporal network efficiency measures by incorporating the time dependency of temporal distance. We propose a temporal regular network model, and based on this plus the redefined temporal efficiency metrics and widely used temporal random network models, we introduce a quantitative approach for identifying temporal small-world architectures (featuring high temporal network efficiency both globally and locally). In addition, within this framework, we can uncover network-specific dynamic structures. Applications to brain networks, international trade networks, and social networks reveal prominent temporal small-world properties with distinct dynamic network structures. We believe that the framework can provide further insight into dynamic changes in the network topology of various real-world systems and significantly promote research on temporal networks.

A software framework for assessing the resilience of drinking water systems to disasters with an example earthquake case study

DOE PAGES

Klise, Katherine A.; Bynum, Michael; Moriarty, Dylan; ...

2017-07-07

Water utilities are vulnerable to a wide variety of human-caused and natural disasters. The Water Network Tool for Resilience (WNTR) is a new open source PythonTM package designed to help water utilities investigate resilience of water distribution systems to hazards and evaluate resilience-enhancing actions. In this paper, the WNTR modeling framework is presented and a case study is described that uses WNTR to simulate the effects of an earthquake on a water distribution system. The case study illustrates that the severity of damage is not only a function of system integrity and earthquake magnitude, but also of the available resourcesmore » and repair strategies used to return the system to normal operating conditions. While earthquakes are particularly concerning since buried water distribution pipelines are highly susceptible to damage, the software framework can be applied to other types of hazards, including power outages and contamination incidents.« less

A software framework for assessing the resilience of drinking water systems to disasters with an example earthquake case study

DOE Office of Scientific and Technical Information (OSTI.GOV)

Klise, Katherine A.; Bynum, Michael; Moriarty, Dylan

Water utilities are vulnerable to a wide variety of human-caused and natural disasters. The Water Network Tool for Resilience (WNTR) is a new open source PythonTM package designed to help water utilities investigate resilience of water distribution systems to hazards and evaluate resilience-enhancing actions. In this paper, the WNTR modeling framework is presented and a case study is described that uses WNTR to simulate the effects of an earthquake on a water distribution system. The case study illustrates that the severity of damage is not only a function of system integrity and earthquake magnitude, but also of the available resourcesmore » and repair strategies used to return the system to normal operating conditions. While earthquakes are particularly concerning since buried water distribution pipelines are highly susceptible to damage, the software framework can be applied to other types of hazards, including power outages and contamination incidents.« less

Relevance and Retrieval Evaluation: Perspectives from Medicine.

ERIC Educational Resources Information Center

Hersh, William

1994-01-01

Discusses topical relevance and situational relevance, which takes into account the impact of the retrieval system on the user, in the context of medicine. Topics addressed include scientific validity; limitations of current evaluation methodology, including recall and precision; and a framework for future retrieval research based on an…

Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation.

PubMed

Wootton, Richard; Vladzymyrskyy, Anton; Zolfo, Maria; Bonnardot, Laurent

2011-01-01

Telemedicine has been used for many years to support doctors in the developing world. Several networks provide services in different settings and in different ways. However, to draw conclusions about which telemedicine networks are successful requires a method of evaluating them. No general consensus or validated framework exists for this purpose. To define a basic method of performance measurement that can be used to improve and compare teleconsultation networks; to employ the proposed framework in an evaluation of three existing networks; to make recommendations about the future implementation and follow-up of such networks. Analysis based on the experience of three telemedicine networks (in operation for 7-10 years) that provide services to doctors in low-resource settings and which employ the same basic design. Although there are many possible indicators and metrics that might be relevant, five measures for each of the three user groups appear to be sufficient for the proposed framework. In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The proposed performance measurement framework was applied to three mature telemedicine networks. Despite their differences in terms of activity, size and objectives, their performance in certain respects is very similar. For example, the time to first reply from an expert is about 24 hours for each network. Although all three networks had systems in place to collect data from the user perspective, none of them collected information about the coordinator's time required or about ease of system usage. They had only limited information about quality and cost. Measuring the performance of a telemedicine network is essential in understanding whether the network is working as intended and what effect it is having. Based on long-term field experience, the suggested framework is a practical tool that will permit organisations to assess the performance of their own networks and to improve them by comparison with others. All telemedicine systems should provide information about setup and running costs because cost-effectiveness is crucial for sustainability.

Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation

PubMed Central

Wootton, Richard; Vladzymyrskyy, Anton; Zolfo, Maria; Bonnardot, Laurent

2011-01-01

Background Telemedicine has been used for many years to support doctors in the developing world. Several networks provide services in different settings and in different ways. However, to draw conclusions about which telemedicine networks are successful requires a method of evaluating them. No general consensus or validated framework exists for this purpose. Objective To define a basic method of performance measurement that can be used to improve and compare teleconsultation networks; to employ the proposed framework in an evaluation of three existing networks; to make recommendations about the future implementation and follow-up of such networks. Methods Analysis based on the experience of three telemedicine networks (in operation for 7–10 years) that provide services to doctors in low-resource settings and which employ the same basic design. Findings Although there are many possible indicators and metrics that might be relevant, five measures for each of the three user groups appear to be sufficient for the proposed framework. In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The proposed performance measurement framework was applied to three mature telemedicine networks. Despite their differences in terms of activity, size and objectives, their performance in certain respects is very similar. For example, the time to first reply from an expert is about 24 hours for each network. Although all three networks had systems in place to collect data from the user perspective, none of them collected information about the coordinator's time required or about ease of system usage. They had only limited information about quality and cost. Conclusion Measuring the performance of a telemedicine network is essential in understanding whether the network is working as intended and what effect it is having. Based on long-term field experience, the suggested framework is a practical tool that will permit organisations to assess the performance of their own networks and to improve them by comparison with others. All telemedicine systems should provide information about setup and running costs because cost-effectiveness is crucial for sustainability. PMID:22162965

NASA System Safety Handbook. Volume 2: System Safety Concepts, Guidelines, and Implementation Examples

NASA Technical Reports Server (NTRS)

Dezfuli, Homayoon; Benjamin, Allan; Everett, Christopher; Feather, Martin; Rutledge, Peter; Sen, Dev; Youngblood, Robert

2015-01-01

This is the second of two volumes that collectively comprise the NASA System Safety Handbook. Volume 1 (NASASP-210-580) was prepared for the purpose of presenting the overall framework for System Safety and for providing the general concepts needed to implement the framework. Volume 2 provides guidance for implementing these concepts as an integral part of systems engineering and risk management. This guidance addresses the following functional areas: 1.The development of objectives that collectively define adequate safety for a system, and the safety requirements derived from these objectives that are levied on the system. 2.The conduct of system safety activities, performed to meet the safety requirements, with specific emphasis on the conduct of integrated safety analysis (ISA) as a fundamental means by which systems engineering and risk management decisions are risk-informed. 3.The development of a risk-informed safety case (RISC) at major milestone reviews to argue that the systems safety objectives are satisfied (and therefore that the system is adequately safe). 4.The evaluation of the RISC (including supporting evidence) using a defined set of evaluation criteria, to assess the veracity of the claims made therein in order to support risk acceptance decisions.

Evaluation of water resources system vulnerability based on co-operative co-evolutionary genetic algorithm and projection pursuit model under the DPSIR framework

NASA Astrophysics Data System (ADS)

Zhao, Y.; Su, X. H.; Wang, M. H.; Li, Z. Y.; Li, E. K.; Xu, X.

2017-08-01

Water resources vulnerability control management is essential because it is related to the benign evolution of socio-economic, environmental and water resources system. Research on water resources system vulnerability is helpful to realization of water resources sustainable utilization. In this study, the DPSIR framework of driving forces-pressure-state-impact-response was adopted to construct the evaluation index system of water resources system vulnerability. Then the co-evolutionary genetic algorithm and projection pursuit were used to establish evaluation model of water resources system vulnerability. Tengzhou City in Shandong Province was selected as a study area. The system vulnerability was analyzed in terms of driving forces, pressure, state, impact and response on the basis of the projection value calculated by the model. The results show that the five components all belong to vulnerability Grade II, the vulnerability degree of impact and state were higher than other components due to the fierce imbalance in supply-demand and the unsatisfied condition of water resources utilization. It is indicated that the influence of high speed socio-economic development and the overuse of the pesticides have already disturbed the benign development of water environment to some extents. While the indexes in response represented lower vulnerability degree than the other components. The results of the evaluation model are coincident with the status of water resources system in the study area, which indicates that the model is feasible and effective.

A framework using cluster-based hybrid network architecture for collaborative virtual surgery.

PubMed

Qin, Jing; Choi, Kup-Sze; Poon, Wai-Sang; Heng, Pheng-Ann

2009-12-01

Research on collaborative virtual environments (CVEs) opens the opportunity for simulating the cooperative work in surgical operations. It is however a challenging task to implement a high performance collaborative surgical simulation system because of the difficulty in maintaining state consistency with minimum network latencies, especially when sophisticated deformable models and haptics are involved. In this paper, an integrated framework using cluster-based hybrid network architecture is proposed to support collaborative virtual surgery. Multicast transmission is employed to transmit updated information among participants in order to reduce network latencies, while system consistency is maintained by an administrative server. Reliable multicast is implemented using distributed message acknowledgment based on cluster cooperation and sliding window technique. The robustness of the framework is guaranteed by the failure detection chain which enables smooth transition when participants join and leave the collaboration, including normal and involuntary leaving. Communication overhead is further reduced by implementing a number of management approaches such as computational policies and collaborative mechanisms. The feasibility of the proposed framework is demonstrated by successfully extending an existing standalone orthopedic surgery trainer into a collaborative simulation system. A series of experiments have been conducted to evaluate the system performance. The results demonstrate that the proposed framework is capable of supporting collaborative surgical simulation.

Benchmarking high performance computing architectures with CMS’ skeleton framework

NASA Astrophysics Data System (ADS)

Sexton-Kennedy, E.; Gartung, P.; Jones, C. D.

2017-10-01

In 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel’s Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta, Mira. Because of this we have revived the 2012 benchmark to test it’s performance and conclusions on these new architectures. This talk will discuss the results of this exercise.

Building an Evaluation Framework for the VIC Model in the NLDAS Testbed

NASA Astrophysics Data System (ADS)

Xia, Y.; Mocko, D. M.; Wang, S.; Pan, M.; Kumar, S.; Peters-Lidard, C. D.; Wei, H.; Ek, M. B.

2017-12-01

Since the second phase of North American Land Data Assimilation System (NLDAS-2) was operationally implemented at NCEP in August 2014, developing the third phase of NLDAS system (NLDAS-3) has been a key task for the NCEP and NASA NLDAS team. The Variable Infiltration Capacity (VIC) model is one major component of the NLDAS system. The current operational NLDAS-2 uses version 4.0.3 (VIC403), research NLDAS-2 uses version 4.0.5 (VIC405), and LIS-based (Land Information System) NLDAS uses version 4.1.2 (VIC412). The purpose of this study is to compressively evaluate three versions and document changes in model behavior towards VIC412 for NLDAS-3. To do that, we develop a relatively comprehensive framework including multiple variables and metrics to assess the performance of different versions. This framework is being incorporated into the NASA Land Verification Toolkit (LVT) for evaluation of other LSMs for NLDAS-3 development. The evaluation results show that there are large and significant improvements for VIC412 in southeastern United States when compared with VIC403 and VIC405. In the other regions, there are very limited improvements or even some degree of deteriorations. Potential reasons are due to: (1) few USGS streamflow observations for soil and hydrologic parameter calibration, (2) the lack of re-calibration of VIC412 in the NLDAS domain, and (3) changes in model physics from VIC403 to VIC412. Overall, the model version upgrade largely/significantly enhances model performance and skill score for all United States except for the Great Plains, suggesting a right direction for VIC model development. Some further efforts are needed for science understanding of land surface physical processes in GP and a re-calibration for VIC412 using reasonable reference datasets is suggested.

Development and Demonstration of a Statistical Data Base System for Library and Network Planning and Evaluation. Fourth Quarterly Report.

ERIC Educational Resources Information Center

Jones, Dennis; And Others

The National Center for Higher Education Management Systems (NCHEMS) has completed the development and demonstration of a library statistical data base. The data base, or management information system, was developed for administrators of public and academic libraries. The system provides administrators with a framework of information and…

Evaluation of Reading Skills in the Wisconsin Prototypic System of Reading Instruction.

ERIC Educational Resources Information Center

Peterson, Joseph M.; Knight, Diana

A system designed to provide a framework for the individualization of reading instruction is discussed. The heart of the system is a mastery checklist of reading skills for each child which provides teachers with a means for discovering the specific skill needs of their students. Initially, the system depended upon individual assessment exercises…

A framework for performing workplace hazard and risk analysis: a participative ergonomics approach.

PubMed

Morag, Ido; Luria, Gil

2013-01-01

Despite the unanimity among researchers about the centrality of workplace analysis based on participatory ergonomics (PE) as a basis for preventive interventions, there is still little agreement about the necessary of a theoretical framework for providing practical guidance. In an effort to develop a conceptual PE framework, the authors, focusing on 20 studies, found five primary dimensions for characterising an analytical structure: (1) extent of workforce involvement; (2) analysis duration; (3) diversity of reporter role types; (4) scope of analysis and (5) supportive information system for analysis management. An ergonomics analysis carried out in a chemical manufacturing plant serves as a case study for evaluating the proposed framework. The study simultaneously demonstrates the five dimensions and evaluates their feasibility. The study showed that managerial leadership was fundamental to the successful implementation of the analysis; that all job holders should participate in analysing their own workplace and simplified reporting methods contributed to a desirable outcome. This paper seeks to clarify the scope of workplace ergonomics analysis by offering a theoretical and structured framework for providing practical advice and guidance. Essential to successfully implementing the analytical framework are managerial involvement, participation of all job holders and simplified reporting methods.

Complementary Evaluation: The Development of a Conceptual Framework to Integrate External and Internal Evaluation in the New Zealand School Context

ERIC Educational Resources Information Center

Mutch, Carol

2012-01-01

One of the themes of current school evaluation research and debate is the extent to which it is possible to integrate internal and external evaluation and accountability and improvement. In this article, the author outlines how New Zealand has attempted to reconcile these differing perspectives and aims. New Zealand has a national system of school…

Adaptive semi-supervised recursive tree partitioning: The ART towards large scale patient indexing in personalized healthcare.

PubMed

Wang, Fei

2015-06-01

With the rapid development of information technologies, tremendous amount of data became readily available in various application domains. This big data era presents challenges to many conventional data analytics research directions including data capture, storage, search, sharing, analysis, and visualization. It is no surprise to see that the success of next-generation healthcare systems heavily relies on the effective utilization of gigantic amounts of medical data. The ability of analyzing big data in modern healthcare systems plays a vital role in the improvement of the quality of care delivery. Specifically, patient similarity evaluation aims at estimating the clinical affinity and diagnostic proximity of patients. As one of the successful data driven techniques adopted in healthcare systems, patient similarity evaluation plays a fundamental role in many healthcare research areas such as prognosis, risk assessment, and comparative effectiveness analysis. However, existing algorithms for patient similarity evaluation are inefficient in handling massive patient data. In this paper, we propose an Adaptive Semi-Supervised Recursive Tree Partitioning (ART) framework for large scale patient indexing such that the patients with similar clinical or diagnostic patterns can be correctly and efficiently retrieved. The framework is designed for semi-supervised settings since it is crucial to leverage experts' supervision knowledge in medical scenario, which are fairly limited compared to the available data. Starting from the proposed ART framework, we will discuss several specific instantiations and validate them on both benchmark and real world healthcare data. Our results show that with the ART framework, the patients can be efficiently and effectively indexed in the sense that (1) similarity patients can be retrieved in a very short time; (2) the retrieval performance can beat the state-of-the art indexing methods. Copyright © 2015. Published by Elsevier Inc.

A Pragmatic Application of the RE-AIM Framework for Evaluating the Implementation of Physical Activity as a Standard of Care in Health Systems.

PubMed

Stoutenberg, Mark; Galaviz, Karla I; Lobelo, Felipe; Joy, Elizabeth; Heath, Gregory W; Hutber, Adrian; Estabrooks, Paul

2018-05-10

Exercise is Medicine (EIM) is an initiative that seeks to integrate physical activity assessment, prescription, and patient referral as a standard in patient care. Methods to assess this integration have lagged behind its implementation. The purpose of this work is to provide a pragmatic framework to guide health care systems in assessing the implementation and impact of EIM. A working group of experts from health care, public health, and implementation science convened to develop an evaluation model based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. The working group aimed to provide pragmatic guidance on operationalizing EIM across the different RE-AIM dimensions based on data typically available in health care settings. The Reach of EIM can be determined by the number and proportion of patients that were screened for physical inactivity, received brief counseling and/or a physical activity prescription, and were referred to physical activity resources. Effectiveness can be assessed through self-reported changes in physical activity, cardiometabolic biometric factors, incidence/burden of chronic disease, as well as health care utilization and costs. Adoption includes assessing the number and representativeness of health care settings that adopt any component of EIM, and Implementation involves assessing the extent to which health care teams implement EIM in their clinic. Finally, Maintenance involves assessing the long-term effectiveness (patient level) and sustained implementation (clinic level) of EIM in a given health care setting. The availability of a standardized, pragmatic, evaluation framework is critical in determining the impact of implementing EIM as a standard of care across health care systems.

An Adaptive Scaffolding E-Learning System for Middle School Students' Physics Learning

ERIC Educational Resources Information Center

Chen, Ching-Huei

2014-01-01

This study presents a framework that utilizes cognitive and motivational aspects of learning to design an adaptive scaffolding e-learning system. It addresses scaffolding processes and conditions for designing adaptive scaffolds. The features and effectiveness of this adaptive scaffolding e-learning system are discussed and evaluated. An…

Designing a Measurement Framework for Response to Intervention in Early Childhood Programs

ERIC Educational Resources Information Center

McConnell, Scott R.; Wackerle-Hollman, Alisha K.; Roloff, Tracy A.; Rodriguez, Michael

2014-01-01

The overall architecture and major components of a measurement system designed and evaluated to support Response to Intervention (RTI) in the areas of language and literacy in early childhood programs are described. Efficient and reliable measurement is essential for implementing any viable RTI system, and implementing such a system in early…

The Impact of the Government Performance and Results Act (GPRA) on Two State Cooperative Extension Systems

ERIC Educational Resources Information Center

Baughman, Sarah; Boyd, Heather H.; Kelsey, Kathleen D.

2012-01-01

The research reported here examined the impact of the Government Performance and Results Act on accountability and evaluation activities in two state Cooperative Extension Systems. Accountability was examined using five dimensions from Koppell's (2005) framework. Findings indicated both Extension systems transferred accountability activities to…

The Dynamic Systems Approach as Metatheory for Developmental Psychology

ERIC Educational Resources Information Center

Witherington, David C.

2007-01-01

The dynamic systems perspective has been touted as an integrative metatheoretical framework for the study of stability and change in development. However, two dynamic systems camps exist with respect to the role higher-order form, once emergent, plays in the process of development. This paper evaluates these two camps in terms of the overarching…

An Evaluation Framework for Selecting Collaboration Systems for Student Teamwork

ERIC Educational Resources Information Center

Shen, Yide; Li, Lei; Zheng, Guangzhi; Guo, Rong

2017-01-01

Collaboration technologies play an increasingly important role in student teamwork in universities. With the proliferation of collaboration systems on the market and the wide range of features they offer, choosing an appropriate system can be an overwhelming task for college students. In this paper, the authors present an empirical study that…

Evaluation Criterion for Quality Assessment of E-Learning Content

ERIC Educational Resources Information Center

Al-Alwani, Abdulkareem

2014-01-01

Research trends related to e-learning systems are oriented towards increasing the efficiency and capacity of the systems, thus they reflect a large variance in performance when considering content conformity and quality standards. The Framework related to standardisation of digital content for e-learning systems is likely to play a significant…

Systematic Planning for Educational Facilities.

ERIC Educational Resources Information Center

McGuffey, Carroll W.

This monograph provides a systematic approach to the problem of planning educational facilities. It first presents a conceptual framework for a general facilities planning and management system called Facilities Resource Allocation Management Evaluation System (FRAMES). The main components of FRAMES are identified as: (1) needs assessment, (2)…

Integration and Evaluation of Automated Pavement Distress Data in INDOT’s Pavement Management System

DOT National Transportation Integrated Search

2017-05-01

This study was in two parts. The first part established and demonstrated a framework for pavement data integration. This is critical for fulfilling QC/QA needs of INDOTs pavement management system, because the precision of the physical location re...

Persuasive Technology in Mobile Applications Promoting Physical Activity: a Systematic Review.

PubMed

Matthews, John; Win, Khin Than; Oinas-Kukkonen, Harri; Freeman, Mark

2016-03-01

Persuasive technology in mobile applications can be used to influence the behaviour of users. A framework known as the Persuasive Systems Design model has been developed for designing and evaluating systems that influence the attitudes or behaviours of users. This paper reviews the current state of mobile applications for health behavioural change with an emphasis on applications that promote physical activity. The inbuilt persuasive features of mobile applications were evaluated using the Persuasive Systems Design model. A database search was conducted to identify relevant articles. Articles were then reviewed using the Persuasive Systems Design model as a framework for analysis. Primary task support, dialogue support, and social support were found to be moderately represented in the selected articles. However, system credibility support was found to have only low levels of representation as a persuasive systems design feature in mobile applications for supporting physical activity. To ensure that available mobile technology resources are best used to improve the wellbeing of people, it is important that the design principles that influence the effectiveness of persuasive technology be understood.

Failure Analysis of a Complex Learning Framework Incorporating Multi-Modal and Semi-Supervised Learning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pullum, Laura L; Symons, Christopher T

2011-01-01

Machine learning is used in many applications, from machine vision to speech recognition to decision support systems, and is used to test applications. However, though much has been done to evaluate the performance of machine learning algorithms, little has been done to verify the algorithms or examine their failure modes. Moreover, complex learning frameworks often require stepping beyond black box evaluation to distinguish between errors based on natural limits on learning and errors that arise from mistakes in implementation. We present a conceptual architecture, failure model and taxonomy, and failure modes and effects analysis (FMEA) of a semi-supervised, multi-modal learningmore » system, and provide specific examples from its use in a radiological analysis assistant system. The goal of the research described in this paper is to provide a foundation from which dependability analysis of systems using semi-supervised, multi-modal learning can be conducted. The methods presented provide a first step towards that overall goal.« less

Clinical time series prediction: Toward a hierarchical dynamical system framework.

PubMed

Liu, Zitao; Hauskrecht, Milos

2015-09-01

Developing machine learning and data mining algorithms for building temporal models of clinical time series is important for understanding of the patient condition, the dynamics of a disease, effect of various patient management interventions and clinical decision making. In this work, we propose and develop a novel hierarchical framework for modeling clinical time series data of varied length and with irregularly sampled observations. Our hierarchical dynamical system framework for modeling clinical time series combines advantages of the two temporal modeling approaches: the linear dynamical system and the Gaussian process. We model the irregularly sampled clinical time series by using multiple Gaussian process sequences in the lower level of our hierarchical framework and capture the transitions between Gaussian processes by utilizing the linear dynamical system. The experiments are conducted on the complete blood count (CBC) panel data of 1000 post-surgical cardiac patients during their hospitalization. Our framework is evaluated and compared to multiple baseline approaches in terms of the mean absolute prediction error and the absolute percentage error. We tested our framework by first learning the time series model from data for the patients in the training set, and then using it to predict future time series values for the patients in the test set. We show that our model outperforms multiple existing models in terms of its predictive accuracy. Our method achieved a 3.13% average prediction accuracy improvement on ten CBC lab time series when it was compared against the best performing baseline. A 5.25% average accuracy improvement was observed when only short-term predictions were considered. A new hierarchical dynamical system framework that lets us model irregularly sampled time series data is a promising new direction for modeling clinical time series and for improving their predictive performance. Copyright © 2014 Elsevier B.V. All rights reserved.

Examining the effectiveness of municipal solid waste management systems: An integrated cost-benefit analysis perspective with a financial cost modeling in Taiwan

DOE Office of Scientific and Technical Information (OSTI.GOV)

Weng, Yu-Chi, E-mail: clyde.weng@gmail.com; Fujiwara, Takeshi

2011-06-15

In order to develop a sound material-cycle society, cost-effective municipal solid waste (MSW) management systems are required for the municipalities in the context of the integrated accounting system for MSW management. Firstly, this paper attempts to establish an integrated cost-benefit analysis (CBA) framework for evaluating the effectiveness of MSW management systems. In this paper, detailed cost/benefit items due to waste problems are particularly clarified. The stakeholders of MSW management systems, including the decision-makers of the municipalities and the citizens, are expected to reconsider the waste problems in depth and thus take wise actions with the aid of the proposed CBAmore » framework. Secondly, focusing on the financial cost, this study develops a generalized methodology to evaluate the financial cost-effectiveness of MSW management systems, simultaneously considering the treatment technological levels and policy effects. The impacts of the influencing factors on the annual total and average financial MSW operation and maintenance (O and M) costs are analyzed in the Taiwanese case study with a demonstrative short-term future projection of the financial costs under scenario analysis. The established methodology would contribute to the evaluation of the current policy measures and to the modification of the policy design for the municipalities.« less

Interpersonal Functions of EFL Teachers' Evaluative Discourse

ERIC Educational Resources Information Center

Lai, Junming

2010-01-01

As an important dimension of interpersonal function in SFL, appraisal has attracted a lot of attentions from the linguists home and abroad. This thesis is an attempt to analyze the interpersonal functions of EFL teachers' evaluative discourse with in the framework of Palladian Systemic-functional Grammar (SFG) and Martin's Appraisal theory. The…

Fuel Cycle Analysis Framework Base Cases for the IAEA/INPRO GAINS Collaborative Project

DOE Office of Scientific and Technical Information (OSTI.GOV)

Brent Dixon

Thirteen countries participated in the Collaborative Project GAINS “Global Architecture of Innovative Nuclear Energy Systems Based on Thermal and Fast Reactors Including a Closed Fuel Cycle”, which was the primary activity within the IAEA/INPRO Program Area B: “Global Vision on Sustainable Nuclear Energy” for the last three years. The overall objective of GAINS was to develop a standard framework for assessing future nuclear energy systems taking into account sustainable development, and to validate results through sample analyses. This paper details the eight scenarios that constitute the GAINS framework base cases for analysis of the transition to future innovative nuclear energymore » systems. The framework base cases provide a reference for users of the framework to start from in developing and assessing their own alternate systems. Each base case is described along with performance results against the GAINS sustainability evaluation metrics. The eight cases include four using a moderate growth projection and four using a high growth projection for global nuclear electricity generation through 2100. The cases are divided into two sets, addressing homogeneous and heterogeneous scenarios developed by GAINS to model global fuel cycle strategies. The heterogeneous world scenario considers three separate nuclear groups based on their fuel cycle strategies, with non-synergistic and synergistic cases. The framework base case analyses results show the impact of these different fuel cycle strategies while providing references for future users of the GAINS framework. A large number of scenario alterations are possible and can be used to assess different strategies, different technologies, and different assumptions about possible futures of nuclear power. Results can be compared to the framework base cases to assess where these alternate cases perform differently versus the sustainability indicators.« less

Freva - Freie Univ Evaluation System Framework for Scientific Infrastructures in Earth System Modeling

NASA Astrophysics Data System (ADS)

Kadow, Christopher; Illing, Sebastian; Kunst, Oliver; Schartner, Thomas; Kirchner, Ingo; Rust, Henning W.; Cubasch, Ulrich; Ulbrich, Uwe

2016-04-01

The Freie Univ Evaluation System Framework (Freva - freva.met.fu-berlin.de) is a software infrastructure for standardized data and tool solutions in Earth system science. Freva runs on high performance computers to handle customizable evaluation systems of research projects, institutes or universities. It combines different software technologies into one common hybrid infrastructure, including all features present in the shell and web environment. The database interface satisfies the international standards provided by the Earth System Grid Federation (ESGF). Freva indexes different data projects into one common search environment by storing the meta data information of the self-describing model, reanalysis and observational data sets in a database. This implemented meta data system with its advanced but easy-to-handle search tool supports users, developers and their plugins to retrieve the required information. A generic application programming interface (API) allows scientific developers to connect their analysis tools with the evaluation system independently of the programming language used. Users of the evaluation techniques benefit from the common interface of the evaluation system without any need to understand the different scripting languages. Facilitation of the provision and usage of tools and climate data automatically increases the number of scientists working with the data sets and identifying discrepancies. The integrated web-shell (shellinabox) adds a degree of freedom in the choice of the working environment and can be used as a gate to the research projects HPC. Plugins are able to integrate their e.g. post-processed results into the database of the user. This allows e.g. post-processing plugins to feed statistical analysis plugins, which fosters an active exchange between plugin developers of a research project. Additionally, the history and configuration sub-system stores every analysis performed with the evaluation system in a database. Configurations and results of the tools can be shared among scientists via shell or web system. Therefore, plugged-in tools benefit from transparency and reproducibility. Furthermore, if configurations match while starting an evaluation plugin, the system suggests to use results already produced by other users - saving CPU/h, I/O, disk space and time. The efficient interaction between different technologies improves the Earth system modeling science framed by Freva.

Freva - Freie Univ Evaluation System Framework for Scientific HPC Infrastructures in Earth System Modeling

NASA Astrophysics Data System (ADS)

Kadow, C.; Illing, S.; Schartner, T.; Grieger, J.; Kirchner, I.; Rust, H.; Cubasch, U.; Ulbrich, U.

2017-12-01

The Freie Univ Evaluation System Framework (Freva - freva.met.fu-berlin.de) is a software infrastructure for standardized data and tool solutions in Earth system science (e.g. www-miklip.dkrz.de, cmip-eval.dkrz.de). Freva runs on high performance computers to handle customizable evaluation systems of research projects, institutes or universities. It combines different software technologies into one common hybrid infrastructure, including all features present in the shell and web environment. The database interface satisfies the international standards provided by the Earth System Grid Federation (ESGF). Freva indexes different data projects into one common search environment by storing the meta data information of the self-describing model, reanalysis and observational data sets in a database. This implemented meta data system with its advanced but easy-to-handle search tool supports users, developers and their plugins to retrieve the required information. A generic application programming interface (API) allows scientific developers to connect their analysis tools with the evaluation system independently of the programming language used. Users of the evaluation techniques benefit from the common interface of the evaluation system without any need to understand the different scripting languages. The integrated web-shell (shellinabox) adds a degree of freedom in the choice of the working environment and can be used as a gate to the research projects HPC. Plugins are able to integrate their e.g. post-processed results into the database of the user. This allows e.g. post-processing plugins to feed statistical analysis plugins, which fosters an active exchange between plugin developers of a research project. Additionally, the history and configuration sub-system stores every analysis performed with the evaluation system in a database. Configurations and results of the tools can be shared among scientists via shell or web system. Furthermore, if configurations match while starting an evaluation plugin, the system suggests to use results already produced by other users - saving CPU/h, I/O, disk space and time. The efficient interaction between different technologies improves the Earth system modeling science framed by Freva.

Probabilistic evaluation of on-line checks in fault-tolerant multiprocessor systems

NASA Technical Reports Server (NTRS)

Nair, V. S. S.; Hoskote, Yatin V.; Abraham, Jacob A.

1992-01-01

The analysis of fault-tolerant multiprocessor systems that use concurrent error detection (CED) schemes is much more difficult than the analysis of conventional fault-tolerant architectures. Various analytical techniques have been proposed to evaluate CED schemes deterministically. However, these approaches are based on worst-case assumptions related to the failure of system components. Often, the evaluation results do not reflect the actual fault tolerance capabilities of the system. A probabilistic approach to evaluate the fault detecting and locating capabilities of on-line checks in a system is developed. The various probabilities associated with the checking schemes are identified and used in the framework of the matrix-based model. Based on these probabilistic matrices, estimates for the fault tolerance capabilities of various systems are derived analytically.

A general Bayesian framework for calibrating and evaluating stochastic models of annual multi-site hydrological data

NASA Astrophysics Data System (ADS)

Frost, Andrew J.; Thyer, Mark A.; Srikanthan, R.; Kuczera, George

2007-07-01

SummaryMulti-site simulation of hydrological data are required for drought risk assessment of large multi-reservoir water supply systems. In this paper, a general Bayesian framework is presented for the calibration and evaluation of multi-site hydrological data at annual timescales. Models included within this framework are the hidden Markov model (HMM) and the widely used lag-1 autoregressive (AR(1)) model. These models are extended by the inclusion of a Box-Cox transformation and a spatial correlation function in a multi-site setting. Parameter uncertainty is evaluated using Markov chain Monte Carlo techniques. Models are evaluated by their ability to reproduce a range of important extreme statistics and compared using Bayesian model selection techniques which evaluate model probabilities. The case study, using multi-site annual rainfall data situated within catchments which contribute to Sydney's main water supply, provided the following results: Firstly, in terms of model probabilities and diagnostics, the inclusion of the Box-Cox transformation was preferred. Secondly the AR(1) and HMM performed similarly, while some other proposed AR(1)/HMM models with regionally pooled parameters had greater posterior probability than these two models. The practical significance of parameter and model uncertainty was illustrated using a case study involving drought security analysis for urban water supply. It was shown that ignoring parameter uncertainty resulted in a significant overestimate of reservoir yield and an underestimation of system vulnerability to severe drought.

Health care services and the transition to young adulthood: challenges and opportunities.

PubMed

Park, M Jane; Adams, Sally H; Irwin, Charles E

2011-01-01

The aim of this study was to examine the potential role of the health care system in the successful transition to young adulthood for all adolescents, with emphasis on adolescents with special health care needs (ASHCN), and to evaluate the system's status in filling that role. Research and conceptual frameworks addressing successful transitions and functioning were reviewed. A framework describing a role for health care services in the transition was presented. The health care system's status in promoting healthy transitions was evaluated, including National Survey of Children with Special Health Care Needs 2005-2006 analyses of key outcomes for ASHCN. Although most national efforts to define skills needed for the transition have focused on career/vocational skills, a few frameworks integrate broader issues such as health, psychosocial development, and civic engagement. Adolescent transitional issues have generally received little attention; however, these have been articulated for ASHCN. Nevertheless, only 2 in 5 ASHCN receive transitional care, and ASHCN fare poorly on other core outcomes. ASHCN with mental health conditions fare worse on outcomes than those with physical health conditions. Our framework for healthy transitions includes the following: 1) adolescents can access a comprehensive health care system, 2) preventable problems are avoided, and 3) chronic problems are managed. The present health care system falls short of accomplishing these. Health care services can potentially play a role in facilitating a healthy transition to young adulthood; however, many gaps exist. Although the health care reform act addresses some gaps, efforts that integrate adolescents' developmental needs and address mental health issues are needed. Copyright © 2011 Academic Pediatric Association. Published by Elsevier Inc. All rights reserved.

Marginal fit and photoelastic stress analysis of CAD-CAM and overcast 3-unit implant-supported frameworks.

PubMed

Presotto, Anna Gabriella Camacho; Bhering, Cláudia Lopes Brilhante; Mesquita, Marcelo Ferraz; Barão, Valentim Adelino Ricardo

2017-03-01

Several studies have shown the superiority of computer-assisted design and computer-assisted manufacturing (CAD-CAM) technology compared with conventional casting. However, an advanced technology exists for casting procedures (the overcasting technique), which may serve as an acceptable and affordable alternative to CAD-CAM technology for fabricating 3-unit implant-supported fixed dental prostheses (FDPs). The purpose of this in vitro study was to evaluate, using quantitative photoelastic analysis, the effect of the prosthetic framework fabrication method (CAD-CAM and overcasting) on the marginal fit and stress transmitted to implants. The correlation between marginal fit and stress was also investigated. Three-unit implant-supported FDP frameworks were made using the CAD-CAM (n=10) and overcasting (n=10) methods. The frameworks were waxed to simulate a mandibular first premolar (PM region) to first molar (M region) FDP using overcast mini-abutment cylinders. The wax patterns were overcast (overcast experimental group) or scanned to obtain the frameworks (CAD-CAM control group). All frameworks were fabricated from cobalt-chromium (CoCr) alloy. The marginal fit was analyzed according to the single-screw test protocol, obtaining an average value for each region (M and PM) and each framework. The frameworks were tightened for the photoelastic model with standardized 10-Ncm torque. Stress was measured by quantitative photoelastic analysis. The results were submitted to the Student t test, 2-way ANOVA, and Pearson correlation test (α=.05). The framework fabrication method (FM) and evaluation site (ES; M and PM regions) did not affect the marginal fit values (P=.559 for FM and P=.065 for ES) and stress (P=.685 for FM and P=.468 for ES) in the implant-supported system. Positive correlations between marginal fit and stress were observed (CAD-CAM: r=0.922; P<.001; overcast: r=0.908; P<.001). CAD-CAM and overcasting methods present similar marginal fit and stress values for 3-unit FDP frameworks. The decreased marginal fit of frameworks induces greater stress in the implant-supported system. Copyright © 2016 Editorial Council for the Journal of Prosthetic Dentistry. Published by Elsevier Inc. All rights reserved.

Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods.

PubMed

Pommier, Jeanine; Guével, Marie-Renée; Jourdan, Didier

2010-01-28

Schools are key settings for health promotion (HP) but the development of suitable approaches for evaluating HP in schools is still a major topic of discussion. This article presents a research protocol of a program developed to evaluate HP. After reviewing HP evaluation issues, the various possible approaches are analyzed and the importance of a realistic evaluation framework and a mixed methods (MM) design are demonstrated. The design is based on a systemic approach to evaluation, taking into account the mechanisms, context and outcomes, as defined in realistic evaluation, adjusted to our own French context using an MM approach. The characteristics of the design are illustrated through the evaluation of a nationwide HP program in French primary schools designed to enhance children's social, emotional and physical health by improving teachers' HP practices and promoting a healthy school environment. An embedded MM design is used in which a qualitative data set plays a supportive, secondary role in a study based primarily on a different quantitative data set. The way the qualitative and quantitative approaches are combined through the entire evaluation framework is detailed. This study is a contribution towards the development of suitable approaches for evaluating HP programs in schools. The systemic approach of the evaluation carried out in this research is appropriate since it takes account of the limitations of traditional evaluation approaches and considers suggestions made by the HP research community.

A Community Framework for Integrative, Coupled Modeling of Human-Earth Systems

NASA Astrophysics Data System (ADS)

Barton, C. M.; Nelson, G. C.; Tucker, G. E.; Lee, A.; Porter, C.; Ullah, I.; Hutton, E.; Hoogenboom, G.; Rogers, K. G.; Pritchard, C.

2017-12-01

We live today in a humanized world, where critical zone dynamics are driven by coupled human and biophysical processes. First generation modeling platforms have been invaluable in providing insight into dynamics of biophysical systems and social systems. But to understand today's humanized planet scientifically and to manage it sustainably, we need integrative modeling of this coupled human-Earth system. To address both scientific and policy questions, we also need modeling that can represent variable combinations of human-Earth system processes at multiple scales. Simply adding more code needed to do this to large, legacy first generation models is impractical, expensive, and will make them even more difficult to evaluate or understand. We need an approach to modeling that mirrors and benefits from the architecture of the complexly coupled systems we hope to model. Building on a series of international workshops over the past two years, we present a community framework to enable and support an ecosystem of diverse models as components that can be interconnected as needed to facilitate understanding of a range of complex human-earth systems interactions. Models are containerized in Docker to make them platform independent. A Basic Modeling Interface and Standard Names ontology (developed by the Community Surface Dynamics Modeling System) is applied to make them interoperable. They are then transformed into RESTful micro-services to allow them to be connected and run in a browser environment. This enables a flexible, multi-scale modeling environment to help address diverse issues with combinations of smaller, focused, component models that are easier to understand and evaluate. We plan to develop, deploy, and maintain this framework for integrated, coupled modeling in an open-source collaborative development environment that can democratize access to advanced technology and benefit from diverse global participation in model development. We also present an initial proof-of-concept of this framework, coupling a widely used agricultural crop model (DSSAT) with a widely used hydrology model (TopoFlow).

Quantitative assessments of municipal waste management systems: using different indicators to compare and rank programs in New York State.

PubMed

Greene, Krista L; Tonjes, David J

2014-04-01

The primary objective of waste management technologies and policies in the United States is to reduce the harmful environmental impacts of waste, particularly those relating to energy consumption and climate change. Performance indicators are frequently used to evaluate the environmental quality of municipal waste systems, as well as to compare and rank programs relative to each other in terms of environmental performance. However, there currently is no consensus on the best indicator for performing these environmental evaluations. The purpose of this study is to examine the common performance indicators used to assess the environmental benefits of municipal waste systems to determine if there is agreement between them regarding which system performs best environmentally. Focus is placed on how indicator selection influences comparisons between municipal waste management programs and subsequent system rankings. The waste systems of ten municipalities in the state of New York, USA, were evaluated using each common performance indicator and Spearman correlations were calculated to see if there was a significant association between system rank orderings. Analyses showed that rank orders of waste systems differ substantially when different indicators are used. Therefore, comparative system assessments based on indicators should be considered carefully, especially those intended to gauge environmental quality. Insight was also gained into specific factors which may lead to one system achieving higher rankings than another. However, despite the insufficiencies of indicators for comparative quality assessments, they do provide important information for waste managers and they can assist in evaluating internal programmatic performance and progress. To enhance these types of assessments, a framework for scoring indicators based on criteria that evaluate their utility and value for system evaluations was developed. This framework was used to construct an improved model for waste system performance assessments. Copyright © 2014 Elsevier Ltd. All rights reserved.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Spogen, L.R.; Cleland, L.L.

An approach to the development of performance based regulations (PBR's) is described. Initially, a framework is constructed that consists of a function hierarchy and associated measures. The function at the top of the hierarchy is described in terms of societal objectives. Decomposition of this function into subordinate functions and their subsequent decompositions yield the function hierarchy. ''Bottom'' functions describe the roles of system components. When measures are identified for the performance of each function and means of aggregating performances to higher levels are established, the framework may be employed for developing PBR's. Consideration of system flexibility and performance uncertainty guidemore » in determining the hierarchical level at which regulations are formulated. Ease of testing compliance is also a factor. To show the viability of the approach, the framework developed by Lawrence Livermore Laboratory for the Nuclear Regulatory Commission for evaluation of material control systems at fixed facilities is presented.« less

Design and Development of a Sharable Clinical Decision Support System Based on a Semantic Web Service Framework.

PubMed

Zhang, Yi-Fan; Gou, Ling; Tian, Yu; Li, Tian-Chang; Zhang, Mao; Li, Jing-Song

2016-05-01

Clinical decision support (CDS) systems provide clinicians and other health care stakeholders with patient-specific assessments or recommendations to aid in the clinical decision-making process. Despite their demonstrated potential for improving health care quality, the widespread availability of CDS systems has been limited mainly by the difficulty and cost of sharing CDS knowledge among heterogeneous healthcare information systems. The purpose of this study was to design and develop a sharable clinical decision support (S-CDS) system that meets this challenge. The fundamental knowledge base consists of independent and reusable knowledge modules (KMs) to meet core CDS needs, wherein each KM is semantically well defined based on the standard information model, terminologies, and representation formalisms. A semantic web service framework was developed to identify, access, and leverage these KMs across diverse CDS applications and care settings. The S-CDS system has been validated in two distinct client CDS applications. Model-level evaluation results confirmed coherent knowledge representation. Application-level evaluation results reached an overall accuracy of 98.66 % and a completeness of 96.98 %. The evaluation results demonstrated the technical feasibility and application prospect of our approach. Compared with other CDS engineering efforts, our approach facilitates system development and implementation and improves system maintainability, scalability and efficiency, which contribute to the widespread adoption of effective CDS within the healthcare domain.

Four Aspects of Civic Education: Teaching the History and Geography of the Land of Israel as a Case Study

ERIC Educational Resources Information Center

Cohen, Aviv

2009-01-01

At the center of this study stands the will to understand the use of the subjects of History and Geography as means of civic education. A new theoretical framework is offered, encompassing different aspects of civic education. With the use of this framework, the Israeli educational system was evaluated as a case study. (Contains 1 table, 1 figure,…

Evaluation of Historical and Projected Agricultural Climate Risk Over the Continental US

NASA Astrophysics Data System (ADS)

Zhu, X.; Troy, T. J.; Devineni, N.

2016-12-01

Food demands are rising due to an increasing population with changing food preferences, which places pressure on agricultural systems. In addition, in the past decade climate extremes have highlighted the vulnerability of our agricultural production to climate variability. Quantitative analyses in the climate-agriculture research field have been performed in many studies. However, climate risk still remains difficult to evaluate at large scales yet shows great potential of help us better understand historical climate change impacts and evaluate the future risk given climate projections. In this study, we developed a framework to evaluate climate risk quantitatively by applying statistical methods such as Bayesian regression, distribution fitting, and Monte Carlo simulation. We applied the framework over different climate regions in the continental US both historically and for modeled climate projections. The relative importance of any major growing season climate index, such as maximum dry period or heavy precipitation, was evaluated to determine what climate indices play a role in affecting crop yields. The statistical modeling framework was applied using county yields, with irrigated and rainfed yields separated to evaluate the different risk. This framework provides estimates of the climate risk facing agricultural production in the near-term that account for the full uncertainty of climate occurrences, range of crop response, and spatial correlation in climate. In particular, the method provides robust estimates of importance of irrigation in mitigating agricultural climate risk. The results of this study can contribute to decision making about crop choice and water use in an uncertain climate.

Achieving the "triple aim" for inborn errors of metabolism: a review of challenges to outcomes research and presentation of a new practice-based evidence framework.

PubMed

Potter, Beth K; Chakraborty, Pranesh; Kronick, Jonathan B; Wilson, Kumanan; Coyle, Doug; Feigenbaum, Annette; Geraghty, Michael T; Karaceper, Maria D; Little, Julian; Mhanni, Aizeddin; Mitchell, John J; Siriwardena, Komudi; Wilson, Brenda J; Syrowatka, Ania

2013-06-01

Across all areas of health care, decision makers are in pursuit of what Berwick and colleagues have called the "triple aim": improving patient experiences with care, improving health outcomes, and managing health system impacts. This is challenging in a rare disease context, as exemplified by inborn errors of metabolism. There is a need for evaluative outcomes research to support effective and appropriate care for inborn errors of metabolism. We suggest that such research should consider interventions at both the level of the health system (e.g., early detection through newborn screening, programs to provide access to treatments) and the level of individual patient care (e.g., orphan drugs, medical foods). We have developed a practice-based evidence framework to guide outcomes research for inborn errors of metabolism. Focusing on outcomes across the triple aim, this framework integrates three priority themes: tailoring care in the context of clinical heterogeneity; a shift from "urgent care" to "opportunity for improvement"; and the need to evaluate the comparative effectiveness of emerging and established therapies. Guided by the framework, a new Canadian research network has been established to generate knowledge that will inform the design and delivery of health services for patients with inborn errors of metabolism and other rare diseases.

Comprehensive planning of data archive in Japanese planetary missions

NASA Astrophysics Data System (ADS)

Yamamoto, Yukio; Shinohara, Iku; Hoshino, Hirokazu; Tateno, Naoki; Hareyama, Makoto; Okada, Naoki; Ebisawa, Ken

Comprehensive planning of data archive in Japanese planetary missions Japan Aerospace Exploration Agency (JAXA) provides HAYABUSA and KAGUYA data as planetary data archives. These data archives, however, were prepared independently. Therefore the inconsistency of data format has occurred, and the knowledge of data archiving activity is not inherited. Recently, the discussion of comprehensive planning of data archive has started to prepare up-coming planetary missions, which indicates the comprehensive plan of data archive is required in several steps. The framework of the comprehensive plan is divided into four items: Preparation, Evaluation, Preservation, and Service. 1. PREPARATION FRAMEWORK Data is classified into several types: raw data, level-0, 1, 2 processing data, ancillary data, and etc. The task of mission data preparation is responsible for instrument teams, but preparations beside mission data and support of data management are essential to make unified conventions and formats over instruments in a mission, and over missions. 2. EVALUATION FRAMEWORK There are two meanings of evaluation: format and quality. The format evaluation is often discussed in the preparation framework. The data quality evaluation which is often called quality assurance (QA) or quality control (QC) must be performed by third party apart from preparation teams. An instrument team has the initiative for the preparation itself, and the third-party group is organized to evaluate the instrument team's activity. 3. PRESERVATION FRAMEWORK The main topic of this framework is document management, archiving structure, and simple access method. The mission produces many documents in the process of the development. Instrument de-velopment is no exception. During long-term development of a mission, many documents are obsoleted and updated repeatedly. A smart system will help instrument team to reduce some troubles of document management and archiving task. JAXA attempts to follow PDS manners to do this management since PDS has highly sophisticated archiving structure. In addition, the access method to archived data must be simple and standard well over a decade. 4. SERVICE FRAMEWORK The service framework including planetary data access protocol, PDAP, has been developed to share a stored data effectively. The sophisticated service framework will work not only for publication data, but also for low-level data. JAXA's data query services is under developed based on PDAP, which means that the low-level data can be published in the same manner as level 2 data. In this presentation, we report the detail structure of these four frameworks adopting upcoming Planet-C, Venus Climate Orbiter, mission.

Evaluation and Verification of Decadal Predictions using the MiKlip Central Evaluation System - a Case Study using the MiKlip Prototype Model Data

NASA Astrophysics Data System (ADS)

Illing, Sebastian; Schuster, Mareike; Kadow, Christopher; Kröner, Igor; Richling, Andy; Grieger, Jens; Kruschke, Tim; Lang, Benjamin; Redl, Robert; Schartner, Thomas; Cubasch, Ulrich

2016-04-01

MiKlip is project for medium-term climate prediction funded by the Federal Ministry of Education and Research in Germany (BMBF) and aims to create a model system that is able provide reliable decadal climate forecasts. During the first project phase of MiKlip the sub-project INTEGRATION located at Freie Universität Berlin developed a framework for scientific infrastructures (FREVA). More information about FREVA can be found in EGU2016-13060. An instance of this framework is used as Central Evaluation System (CES) during the MiKlip project. Throughout the first project phase various sub-projects developed over 25 analysis tools - so called plugins - for the CES. The main focus of these plugins is on the evaluation and verification of decadal climate prediction data, but most plugins are not limited to this scope. They target a wide range of scientific questions. Starting from preprocessing tools like the "LeadtimeSelector", which creates lead-time dependent time-series from decadal hindcast sets, over tracking tools like the "Zykpak" plugin, which can objectively locate and track mid-latitude cyclones, to plugins like "MurCSS" or "SPECS", which calculate deterministic and probabilistic skill metrics. We also integrated some analyses from Model Evaluation Tools (MET), which was developed at NCAR. We will show the theoretical background, technical implementation strategies, and some interesting results of the evaluation of the MiKlip Prototype decadal prediction system for a selected set of these tools.

Generic, Extensible, Configurable Push-Pull Framework for Large-Scale Science Missions

NASA Technical Reports Server (NTRS)

Foster, Brian M.; Chang, Albert Y.; Freeborn, Dana J.; Crichton, Daniel J.; Woollard, David M.; Mattmann, Chris A.

2011-01-01

The push-pull framework was developed in hopes that an infrastructure would be created that could literally connect to any given remote site, and (given a set of restrictions) download files from that remote site based on those restrictions. The Cataloging and Archiving Service (CAS) has recently been re-architected and re-factored in its canonical services, including file management, workflow management, and resource management. Additionally, a generic CAS Crawling Framework was built based on motivation from Apache s open-source search engine project called Nutch. Nutch is an Apache effort to provide search engine services (akin to Google), including crawling, parsing, content analysis, and indexing. It has produced several stable software releases, and is currently used in production services at companies such as Yahoo, and at NASA's Planetary Data System. The CAS Crawling Framework supports many of the Nutch Crawler's generic services, including metadata extraction, crawling, and ingestion. However, one service that was not ported over from Nutch is a generic protocol layer service that allows the Nutch crawler to obtain content using protocol plug-ins that download content using implementations of remote protocols, such as HTTP, FTP, WinNT file system, HTTPS, etc. Such a generic protocol layer would greatly aid in the CAS Crawling Framework, as the layer would allow the framework to generically obtain content (i.e., data products) from remote sites using protocols such as FTP and others. Augmented with this capability, the Orbiting Carbon Observatory (OCO) and NPP (NPOESS Preparatory Project) Sounder PEATE (Product Evaluation and Analysis Tools Elements) would be provided with an infrastructure to support generic FTP-based pull access to remote data products, obviating the need for any specialized software outside of the context of their existing process control systems. This extensible configurable framework was created in Java, and allows the use of different underlying communication middleware (at present, both XMLRPC, and RMI). In addition, the framework is entirely suitable in a multi-mission environment and is supporting both NPP Sounder PEATE and the OCO Mission. Both systems involve tasks such as high-throughput job processing, terabyte-scale data management, and science computing facilities. NPP Sounder PEATE is already using the push-pull framework to accept hundreds of gigabytes of IASI (infrared atmospheric sounding interferometer) data, and is in preparation to accept CRIMS (Cross-track Infrared Microwave Sounding Suite) data. OCO will leverage the framework to download MODIS, CloudSat, and other ancillary data products for use in the high-performance Level 2 Science Algorithm. The National Cancer Institute is also evaluating the framework for use in sharing and disseminating cancer research data through its Early Detection Research Network (EDRN).

A System of Systems (SoS) Approach to Sustainable Energy Planning

NASA Astrophysics Data System (ADS)

Madani, Kaveh; Hadian, Saeed

2015-04-01

The general policy of mandating fossil fuel replacement with "green" energies may not be as effective and environmental-friendly as perceived, due to the secondary impacts of renewable energies on different natural resources. An integrated systems analysis framework is essential to developing sustainable energy supply systems with minimal unintended impacts on valuable natural resources such as water, climate, and ecosystem. This presentation discusses how a system of systems (SoS) framework can be developed to quantitatively evaluate the desirability of different energy supply alternatives with respect to different sustainability criteria under uncertainty. Relative Aggregate Footprint (RAF) scores of a range of renewable and nonrenewable energy alternatives are determined using their performance values under four sustainability criteria, namely carbon footprint, water footprint, land footprint, and cost of energy production. Our results suggest that despite their lower emissions, some renewable energy sources are less promising than non-renewable energy sources from a SoS perspective that considers the trade-offs between carbon footprint of energies and their effects on water, ecosystem, and economic resources. A new framework based on the Modern Portfolio Theory (MPT) is also proposed for analyzing the overall sustainability of different energy mixes for different risk of return levels with respect to the trade-offs involved. It is discussed how the proposed finance-based sustainability evaluation method can help policy makers maximize the energy portfolio's expected sustainability for a given amount of portfolio risk, or equivalently minimize risk for a given level of expected sustainability level, by revising the energy mix.

A COMPARISON OF INTERCELL METRICS ON DISCRETE GLOBAL GRID SYSTEMS

EPA Science Inventory

A discrete global grid system (DGGS) is a spatial data model that aids in global research by serving as a framework for environmental modeling, monitoring and sampling across the earth at multiple spatial scales. Topological and geometric criteria have been proposed to evaluate a...

77 FR 8217 - Evaluating the Usability of Electronic Health Record (EHR) Systems

Federal Register 2010, 2011, 2012, 2013, 2014

2012-02-14

... interface design guidelines for EHRs. Manufacturers interested in participating in this research will be... the usability of health information technology (HIT) systems. NIST research is designed to: (1... develop performance-oriented user interface design guidelines for EHRs, and a framework for assessing the...

ASSESSING THE HYDROGEOLOGIC CLASSIFICATION SYSTEM IN MID-ATLANTIC COASTAL PLAIN STREAMS USING BENTHIC MACROINVERTEBRATES

EPA Science Inventory

Assessing classification systems that describe natural variation across regions is an important first step for developing indicators. We evaluated a hydrogeologic framework for first order streams in the mid-Atlantic Coastal Plain as part of the LIPS-MACS (Landscape Indicators f...

Training Issues Associated with COTS-based Information Intensive Systems.

ERIC Educational Resources Information Center

Farr, John V.; Verma, Dinesh

2002-01-01

A literature review and survey responses from 194 information technology and aerospace contractors identified methods and outcomes of training delivery. Results were used to develop a framework for evaluating and selecting commercial off-the-shelf systems (COTS) for operations and maintenance training. (Contains 11 references.) (SK)

Receptionist and Communication Systems Operation. Florida Vocational Program Guide.

ERIC Educational Resources Information Center

Florida State Univ., Tallahassee. Center for Instructional Development and Services.

This program guide identifies primary considerations in the organization, operation, and evaluation of a receptionist and communication systems operation program. An occupational description and program content are presented. A curriculum framework specifies the exact course title, course number, levels of instruction, major course content,…

Team table: a framework and tool for continuous factory planning

NASA Astrophysics Data System (ADS)

Sihn, Wilfried; Bischoff, Juergen; von Briel, Ralf; Josten, Marcus

2000-10-01

Growing market turbulences and shorter product life cycles require a continuous adaptation of factory structures resulting in a continuous factory planning process. Therefore a new framework is developed which focuses on configuration and data management process integration. This enable an online system performance evaluation based on continuous availability of current data. The use of this framework is especially helpful and will guarantee high cost and time savings, when used in the early stages of the planning, called the concept or rough planning phase. The new framework is supported by a planning round table as a tool for team-based configuration processes integrating the knowledge of all persons involved in planning processes. A case study conducted at a German company shows the advantages which can be achieved by implementing the new framework and methods.

Do health economic evaluations using observational data provide reliable assessment of treatment effects?

PubMed Central

2013-01-01

Economic evaluation in modern health care systems is seen as a transparent scientific framework that can be used to advance progress towards improvements in population health at the best possible value. Despite the perceived superiority that trial-based studies have in terms of internal validity, economic evaluations often employ observational data. In this review, the interface between econometrics and economic evaluation is explored, with emphasis placed on highlighting methodological issues relating to the evaluation of cost-effectiveness within a bivariate framework. Studies that satisfied the eligibility criteria exemplified the use of matching, regression analysis, propensity scores, instrumental variables, as well as difference-in-differences approaches. All studies were reviewed and critically appraised using a structured template. The findings suggest that although state-of-the-art econometric methods have the potential to provide evidence on the causal effects of clinical and policy interventions, their application in economic evaluation is subject to a number of limitations. These range from no credible assessment of key assumptions and scarce evidence regarding the relative performance of different methods, to lack of reporting of important study elements, such as a summary outcome measure and its associated sampling uncertainty. Further research is required to better understand the ways in which observational data should be analysed in the context of the economic evaluation framework. PMID:24229445

Medical-device risk management and public safety: using cost-benefit as a measurement of effectiveness

NASA Astrophysics Data System (ADS)

Hughes, Allen A.

1994-12-01

Public safety can be enhanced through the development of a comprehensive medical device risk management. This can be accomplished through case studies using a framework that incorporates cost-benefit analysis in the evaluation of risk management attributes. This paper presents a framework for evaluating the risk management system for regulatory Class III medical devices. The framework consists of the following sixteen attributes of a comprehensive medical device risk management system: fault/failure analysis, premarket testing/clinical trials, post-approval studies, manufacturer sponsored hospital studies, product labeling, establishment inspections, problem reporting program, mandatory hospital reporting, medical literature surveillance, device/patient registries, device performance monitoring, returned product analysis, autopsy program, emergency treatment funds/interim compensation, product liability, and alternative compensation mechanisms. Review of performance histories for several medical devices can reveal the value of information for many attributes, and also the inter-dependencies of the attributes in generating risk information flow. Such an information flow network is presented as a starting point for enhancing medical device risk management by focusing on attributes with high net benefit values and potential to spur information dissemination.

CURMIS. Curriculum Management Information System (and) Prospectus of a Design to Assist a High School Staff in the Evaluation of Its Program.

ERIC Educational Resources Information Center

Sapone, Carmelo V.

CURMIS (Curriculum Management Information System) is a conceptual system, the framework of which is designed to identify and reveal relationships among complex related interacting phenomena. This paper is a description of the system which will centralize and make conveniently available information needed for developing and monitoring instructional…

A framework to assess management performance in district health systems: a qualitative and quantitative case study in Iran.

PubMed

Tabrizi, Jafar Sadegh; Gholipour, Kamal; Iezadi, Shabnam; Farahbakhsh, Mostafa; Ghiasi, Akbar

2018-01-01

The aim was to design a district health management performance framework for Iran's healthcare system. The mixed-method study was conducted between September 2015 and May 2016 in Tabriz, Iran. In this study, the indicators of district health management performance were obtained by analyzing the 45 semi-structured surveys of experts in the public health system. Content validity of performance indicators which were generated in qualitative part were reviewed and confirmed based on content validity index (CVI). Also content validity ratio (CVR) was calculated using data acquired from a survey of 21 experts in quantitative part. The result of this study indicated that, initially, 81 indicators were considered in framework of district health management performance and, at the end, 53 indicators were validated and confirmed. These indicators were classified in 11 categories which include: human resources and organizational creativity, management and leadership, rules and ethics, planning and evaluation, district managing, health resources management and economics, community participation, quality improvement, research in health system, health information management, epidemiology and situation analysis. The designed framework model can be used to assess the district health management and facilitates performance improvement at the district level.

eClims: An Extensible and Dynamic Integration Framework for Biomedical Information Systems.

PubMed

Savonnet, Marinette; Leclercq, Eric; Naubourg, Pierre

2016-11-01

Biomedical information systems (BIS) require consideration of three types of variability: data variability induced by new high throughput technologies, schema or model variability induced by large scale studies or new fields of research, and knowledge variability resulting from new discoveries. Beyond data heterogeneity, managing variabilities in the context of BIS requires extensible and dynamic integration process. In this paper, we focus on data and schema variabilities and we propose an integration framework based on ontologies, master data, and semantic annotations. The framework addresses issues related to: 1) collaborative work through a dynamic integration process; 2) variability among studies using an annotation mechanism; and 3) quality control over data and semantic annotations. Our approach relies on two levels of knowledge: BIS-related knowledge is modeled using an application ontology coupled with UML models that allow controlling data completeness and consistency, and domain knowledge is described by a domain ontology, which ensures data coherence. A system build with the eClims framework has been implemented and evaluated in the context of a proteomic platform.

Multilayer network decoding versatility and trust

NASA Astrophysics Data System (ADS)

Sarkar, Camellia; Yadav, Alok; Jalan, Sarika

2016-01-01

In the recent years, the multilayer networks have increasingly been realized as a more realistic framework to understand emergent physical phenomena in complex real-world systems. We analyze massive time-varying social data drawn from the largest film industry of the world under a multilayer network framework. The framework enables us to evaluate the versatility of actors, which turns out to be an intrinsic property of lead actors. Versatility in dimers suggests that working with different types of nodes are more beneficial than with similar ones. However, the triangles yield a different relation between type of co-actor and the success of lead nodes indicating the importance of higher-order motifs in understanding the properties of the underlying system. Furthermore, despite the degree-degree correlations of entire networks being neutral, multilayering picks up different values of correlation indicating positive connotations like trust, in the recent years. The analysis of weak ties of the industry uncovers nodes from a lower-degree regime being important in linking Bollywood clusters. The framework and the tools used herein may be used for unraveling the complexity of other real-world systems.

Small drinking water systems under spatiotemporal water quality variability: a risk-based performance benchmarking framework.

PubMed

Bereskie, Ty; Haider, Husnain; Rodriguez, Manuel J; Sadiq, Rehan

2017-08-23

Traditional approaches for benchmarking drinking water systems are binary, based solely on the compliance and/or non-compliance of one or more water quality performance indicators against defined regulatory guidelines/standards. The consequence of water quality failure is dependent on location within a water supply system as well as time of the year (i.e., season) with varying levels of water consumption. Conventional approaches used for water quality comparison purposes fail to incorporate spatiotemporal variability and degrees of compliance and/or non-compliance. This can lead to misleading or inaccurate performance assessment data used in the performance benchmarking process. In this research, a hierarchical risk-based water quality performance benchmarking framework is proposed to evaluate small drinking water systems (SDWSs) through cross-comparison amongst similar systems. The proposed framework (R WQI framework) is designed to quantify consequence associated with seasonal and location-specific water quality issues in a given drinking water supply system to facilitate more efficient decision-making for SDWSs striving for continuous performance improvement. Fuzzy rule-based modelling is used to address imprecision associated with measuring performance based on singular water quality guidelines/standards and the uncertainties present in SDWS operations and monitoring. This proposed R WQI framework has been demonstrated using data collected from 16 SDWSs in Newfoundland and Labrador and Quebec, Canada, and compared to the Canadian Council of Ministers of the Environment WQI, a traditional, guidelines/standard-based approach. The study found that the R WQI framework provides an in-depth state of water quality and benchmarks SDWSs more rationally based on the frequency of occurrence and consequence of failure events.

Simulation Framework for Rapid Entry, Descent, and Landing (EDL) Analysis. Volume 2; Appendices

NASA Technical Reports Server (NTRS)

Murri, Daniel G.

2010-01-01

The NASA Engineering and Safety Center (NESC) was requested to establish the Simulation Framework for Rapid Entry, Descent, and Landing (EDL) Analysis assessment, which involved development of an enhanced simulation architecture using the Program to Optimize Simulated Trajectories II (POST2) simulation tool. The assessment was requested to enhance the capability of the Agency to provide rapid evaluation of EDL characteristics in systems analysis studies, preliminary design, mission development and execution, and time-critical assessments. Many of the new simulation framework capabilities were developed to support the Agency EDL Systems Analysis (EDL-SA) team, that is conducting studies of the technologies and architectures that are required to enable higher mass robotic and human mission to Mars. The appendices to the original report are contained in this document.

Simulation Framework for Rapid Entry, Descent, and Landing (EDL) Analysis, Phase 2 Results

NASA Technical Reports Server (NTRS)

Murri, Daniel G.

2011-01-01

The NASA Engineering and Safety Center (NESC) was requested to establish the Simulation Framework for Rapid Entry, Descent, and Landing (EDL) Analysis assessment, which involved development of an enhanced simulation architecture using the Program to Optimize Simulated Trajectories II simulation tool. The assessment was requested to enhance the capability of the Agency to provide rapid evaluation of EDL characteristics in systems analysis studies, preliminary design, mission development and execution, and time-critical assessments. Many of the new simulation framework capabilities were developed to support the Agency EDL-Systems Analysis (SA) team that is conducting studies of the technologies and architectures that are required to enable human and higher mass robotic missions to Mars. The findings, observations, and recommendations from the NESC are provided in this report.

Simulation Framework for Rapid Entry, Descent, and Landing (EDL) Analysis. Volume 1

NASA Technical Reports Server (NTRS)

Murri, Daniel G.

2010-01-01

The NASA Engineering and Safety Center (NESC) was requested to establish the Simulation Framework for Rapid Entry, Descent, and Landing (EDL) Analysis assessment, which involved development of an enhanced simulation architecture using the Program to Optimize Simulated Trajectories II (POST2) simulation tool. The assessment was requested to enhance the capability of the Agency to provide rapid evaluation of EDL characteristics in systems analysis studies, preliminary design, mission development and execution, and time-critical assessments. Many of the new simulation framework capabilities were developed to support the Agency EDL Systems Analysis (EDL-SA) team, that is conducting studies of the technologies and architectures that are required to enable higher mass robotic and human mission to Mars. The findings of the assessment are contained in this report.

A methodological approach and framework for sustainability assessment in NGO-implemented primary health care programs.

PubMed

Sarriot, Eric G; Winch, Peter J; Ryan, Leo J; Bowie, Janice; Kouletio, Michelle; Swedberg, Eric; LeBan, Karen; Edison, Jay; Welch, Rikki; Pacqué, Michel C

2004-01-01

An estimated 10.8 million children under 5 continue to die each year in developing countries from causes easily treatable or preventable. Non governmental organizations (NGOs) are frontline implementers of low-cost and effective child health interventions, but their progress toward sustainable child health gains is a challenge to evaluate. This paper presents the Child Survival Sustainability Assessment (CSSA) methodology--a framework and process--to map progress towards sustainable child health from the community level and upward. The CSSA was developed with NGOs through a participatory process of research and dialogue. Commitment to sustainability requires a systematic and systemic consideration of human, social and organizational processes beyond a purely biomedical perspective. The CSSA is organized around three interrelated dimensions of evaluation: (1) health and health services; (2) capacity and viability of local organizations; (3) capacity of the community in its social ecological context. The CSSA uses a participatory, action-planning process, engaging a 'local system' of stakeholders in the contextual definition of objectives and indicators. Improved conditions measured in the three dimensions correspond to progress toward a sustainable health situation for the population. This framework opens new opportunities for evaluation and research design and places sustainability at the center of primary health care programming.

Benchmarking high performance computing architectures with CMS’ skeleton framework

DOE PAGES

Sexton-Kennedy, E.; Gartung, P.; Jones, C. D.

2017-11-23

Here, in 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel’s Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta,more » Mira. Because of this we have revived the 2012 benchmark to test it’s performance and conclusions on these new architectures. This talk will discuss the results of this exercise.« less

[The GIPSY-RECPAM model: a versatile approach for integrated evaluation in cardiologic care].

PubMed

Carinci, F

2009-01-01

Tree-structured methodology applied for the GISSI-PSICOLOGIA project, although performed in the framework of earliest GISSI studies, represents a powerful tool to analyze different aspects of cardiologic care. The GISSI-PSICOLOGIA project has delivered a novel methodology based on the joint application of psychometric tools and sophisticated statistical techniques. Its prospective use could allow building effective epidemiological models relevant to the prognosis of the cardiologic patient. The various features of the RECPAM method allow a versatile use in the framework of modern e-health projects. The study used the Cognitive Behavioral Assessment H Form (CBA-H) psychometrics scales. The potential for its future application in the framework of Italian cardiology is relevant and particularly indicated to assist planning of systems for integrated care and routine evaluation of the cardiologic patient.

Benchmarking high performance computing architectures with CMS’ skeleton framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sexton-Kennedy, E.; Gartung, P.; Jones, C. D.

Here, in 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel’s Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta,more » Mira. Because of this we have revived the 2012 benchmark to test it’s performance and conclusions on these new architectures. This talk will discuss the results of this exercise.« less

NIST biometric evaluations and developments

NASA Astrophysics Data System (ADS)

Garris, Michael D.; Wilson, Charles L.

2005-05-01

This paper presents an R&D framework used by the National Institute of Standards and Technology (NIST) for biometric technology testing and evaluation. The focus of this paper is on fingerprint-based verification and identification. Since 9-11 the NIST Image Group has been mandated by Congress to run a program for biometric technology assessment and biometric systems certification. Four essential areas of activity are discussed: 1) developing test datasets, 2) conducting performance assessment; 3) technology development; and 4) standards participation. A description of activities and accomplishments are provided for each of these areas. In the process, methods of performance testing are described and results from specific biometric technology evaluations are presented. This framework is anticipated to have broad applicability to other technology and application domains.

A business rules design framework for a pharmaceutical validation and alert system.

PubMed

Boussadi, A; Bousquet, C; Sabatier, B; Caruba, T; Durieux, P; Degoulet, P

2011-01-01

Several alert systems have been developed to improve the patient safety aspects of clinical information systems (CIS). Most studies have focused on the evaluation of these systems, with little information provided about the methodology leading to system implementation. We propose here an 'agile' business rule design framework (BRDF) supporting both the design of alerts for the validation of drug prescriptions and the incorporation of the end user into the design process. We analyzed the unified process (UP) design life cycle and defined the activities, subactivities, actors and UML artifacts that could be used to enhance the agility of the proposed framework. We then applied the proposed framework to two different sets of data in the context of the Georges Pompidou University Hospital (HEGP) CIS. We introduced two new subactivities into UP: business rule specification and business rule instantiation activity. The pharmacist made an effective contribution to five of the eight BRDF design activities. Validation of the two new subactivities was effected in the context of drug dosage adaption to the patients' clinical and biological contexts. Pilot experiment shows that business rules modeled with BRDF and implemented as an alert system triggered an alert for 5824 of the 71,413 prescriptions considered (8.16%). A business rule design framework approach meets one of the strategic objectives for decision support design by taking into account three important criteria posing a particular challenge to system designers: 1) business processes, 2) knowledge modeling of the context of application, and 3) the agility of the various design steps.

Conceptual framework for development of comprehensive e-health evaluation tool.

PubMed

Khoja, Shariq; Durrani, Hammad; Scott, Richard E; Sajwani, Afroz; Piryani, Usha

2013-01-01

The main objective of this study was to develop an e-health evaluation tool based on a conceptual framework including relevant theories for evaluating use of technology in health programs. This article presents the development of an evaluation framework for e-health programs. The study was divided into three stages: Stage 1 involved a detailed literature search of different theories and concepts on evaluation of e-health, Stage 2 plotted e-health theories to identify relevant themes, and Stage 3 developed a matrix of evaluation themes and stages of e-health programs. The framework identifies and defines different stages of e-health programs and then applies evaluation theories to each of these stages for development of the evaluation tool. This framework builds on existing theories of health and technology evaluation and presents a conceptual framework for developing an e-health evaluation tool to examine and measure different factors that play a definite role in the success of e-health programs. The framework on the horizontal axis divides e-health into different stages of program implementation, while the vertical axis identifies different themes and areas of consideration for e-health evaluation. The framework helps understand various aspects of e-health programs and their impact that require evaluation at different stages of the life cycle. The study led to the development of a new and comprehensive e-health evaluation tool, named the Khoja-Durrani-Scott Framework for e-Health Evaluation.

Evaluation of mobility impacts of advanced information systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Peeta, S.; Poonuru, K.; Sinha, K.

2000-06-01

Advanced technologies under the aegis of advanced traveler information systems and advanced traffic management systems are being employed to address the debilitating traffic congestion problem. Broadly identified under the label intelligent transportation systems (ITS), they focus on enhancing the efficiency of the existing roadway utilization. Though ITS has transitioned from the conceptual framework stage to the operational test phase that analyzes real-world feasibility, studies that systematically quantify the multidimensional real-world impacts of these technologies in terms of mobility, safety, and air quality, are lacking. This paper proposes a simulation-based framework to address the mobility impacts of these technologies through themore » provision of information to travelers. The information provision technologies are labeled as advanced information systems (AIS), and include pretrip information, en route information, variable message signs, and combinations thereof. The primary focus of the paper is to evaluate alternative AIS technologies using the heavily traveled Borman Expressway corridor in northwestern Indiana as a case study. Simulation results provide insights into the mobility impacts of AIS technologies, and contrast the effectiveness of alternative information provision sources and strategies.« less

The interactive systems framework applied to the strategic prevention framework: the Rhode Island experience.

PubMed

Florin, Paul; Friend, Karen B; Buka, Stephen; Egan, Crystelle; Barovier, Linda; Amodei, Brenda

2012-12-01

The Interactive Systems Framework for Dissemination and Implementation (ISF) was introduced as a heuristic systems level model to help bridge the gap between research and practice (Wandersman et al., in Am J Commun Psychol 41:171-181, 2008). This model describes three interacting systems with distinct functions that (1) distill knowledge to develop innovations; (2) provide supportive training and technical assistance for dissemination to; (3) a prevention delivery system responsible for implementation in the field. The Strategic Prevention Framework (SPF) is a major prevention innovation launched by the Center for Substance Abuse Prevention (CSAP) of the Substance Abuse and Mental Health Services Administration (SAMHSA). The SPF offers a structured, sequential, data-driven approach that explicitly targets environmental conditions in the community and aims for change in substance use and problems at the population level. This paper describes how the ISF was applied to the challenges of implementing the SPF in 14 Rhode Island communities, with a focus on the development of a new Training and Technical Assistance Resources Center to support SPF efforts. More specifically, we (1) describe each of the three ISF interacting systems as they evolved in Rhode Island; (2) articulate the lines of communication between the three systems; and (3) examine selected evaluation data to understand relationships between training and technical assistance and SPF implementation and outcomes.

Building a framework for ergonomic research on laparoscopic instrument handles.

PubMed

Li, Zheng; Wang, Guohui; Tan, Juan; Sun, Xulong; Lin, Hao; Zhu, Shaihong

2016-06-01

Laparoscopic surgery carries the advantage of minimal invasiveness, but ergonomic design of the instruments used has progressed slowly. Previous studies have demonstrated that the handle of laparoscopic instruments is vital for both surgical performance and surgeon's health. This review provides an overview of the sub-discipline of handle ergonomics, including an evaluation framework, objective and subjective assessment systems, data collection and statistical analyses. Furthermore, a framework for ergonomic research on laparoscopic instrument handles is proposed to standardize work on instrument design. Copyright © 2016 IJS Publishing Group Ltd. Published by Elsevier Ltd. All rights reserved.

Evaluating building performance in healthcare facilities: an organizational perspective.

PubMed

Steinke, Claudia; Webster, Lynn; Fontaine, Marie

2010-01-01

Using the environment as a strategic tool is one of the most cost-effective and enduring approaches for improving public health; however, it is one that requires multiple perspectives. The purpose of this article is to highlight an innovative methodology that has been developed for conducting comprehensive performance evaluations in public sector health facilities in Canada. The building performance evaluation methodology described in this paper is a government initiative. The project team developed a comprehensive building evaluation process for all new capital health projects that would respond to the aforementioned need for stakeholders to be more accountable and to better integrate the larger organizational strategy of facilities. The Balanced Scorecard, which is a multiparadigmatic, performance-based business framework, serves as the underlying theoretical framework for this initiative. It was applied in the development of the conceptual model entitled the Building Performance Evaluation Scorecard, which provides the following benefits: (1) It illustrates a process to link facilities more effectively to the overall mission and goals of an organization; (2) It is both a measurement and a management system that has the ability to link regional facilities to measures of success and larger business goals; (3) It provides a standardized methodology that ensures consistency in assessing building performance; and (4) It is more comprehensive than traditional building evaluations. The methodology presented in this paper is both a measurement and management system that integrates the principles of evidence-based design with the practices of pre- and post-occupancy evaluation. It promotes accountability and continues throughout the life cycle of a project. The advantage of applying this framework is that it engages health organizations in clarifying a vision and strategy for their facilities and helps translate those strategies into action and measurable performance outcomes.

Review of health information technology usability study methodologies

PubMed Central

Bakken, Suzanne

2011-01-01

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user–task interaction, n=42; stage 2, system–task interaction, n=310; stage 3, user–task–system interaction, n=69; stage 4, user–task–system–environment interaction, n=54; and stage 5, user–task–system–environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation. PMID:21828224

Impact of reconstruction strategies on system performance measures : maximizing safety and mobility while minimizing life-cycle costs : final report, December 8, 2008.

DOT National Transportation Integrated Search

2008-12-08

The objective of this research is to develop a general methodological framework for planning and : evaluating the effectiveness of highway reconstruction strategies on the systems performance : measures, in particular safety, mobility, and the tot...

Evaluating the Diagnostic Validity of a Facet-Based Formative Assessment System

ERIC Educational Resources Information Center

DeBarger, Angela Haydel; DiBello, Louis; Minstrell, Jim; Feng, Mingyu; Stout, William; Pellegrino, James; Haertel, Geneva; Harris, Christopher; Ructinger, Liliana

2011-01-01

This paper describes methods for an alignment study and psychometric analyses of a formative assessment system, Diagnoser Tools for physics. Diagnoser Tools begin with facet clusters as the interpretive framework for designing questions and instructional activities. Thus each question in the diagnostic assessments includes distractors that…

Effective Vehicle-Based Kangaroo Detection for Collision Warning Systems Using Region-Based Convolutional Networks.

PubMed

Saleh, Khaled; Hossny, Mohammed; Nahavandi, Saeid

2018-06-12

Traffic collisions between kangaroos and motorists are on the rise on Australian roads. According to a recent report, it was estimated that there were more than 20,000 kangaroo vehicle collisions that occurred only during the year 2015 in Australia. In this work, we are proposing a vehicle-based framework for kangaroo detection in urban and highway traffic environment that could be used for collision warning systems. Our proposed framework is based on region-based convolutional neural networks (RCNN). Given the scarcity of labeled data of kangaroos in traffic environments, we utilized our state-of-the-art data generation pipeline to generate 17,000 synthetic depth images of traffic scenes with kangaroo instances annotated in them. We trained our proposed RCNN-based framework on a subset of the generated synthetic depth images dataset. The proposed framework achieved a higher average precision (AP) score of 92% over all the testing synthetic depth image datasets. We compared our proposed framework against other baseline approaches and we outperformed it with more than 37% in AP score over all the testing datasets. Additionally, we evaluated the generalization performance of the proposed framework on real live data and we achieved a resilient detection accuracy without any further fine-tuning of our proposed RCNN-based framework.

Premise to Implement a Grading System to Evaluate the Sanitary Level in Food Service Establishments in Milan, Italy

PubMed Central

Razzini, Katia

2015-01-01

The regulatory framework of the official controls on food safety, the criteria and methods from the planning of interventions in the field of official control to the management of information flows, and the standards described in the operation manual of the local competent authorities drafted by the Lombardy Region (2011) were evaluated. A questionnaire consisting of n. 10 questions with multiple answers draft in partnership with EPAM (the Association of Provincial Public Retail and catering businesses in Milan) to n. 107 Food service establishments of Milan shows that 92% of managers approve the introduction of a grading system. The regulatory framework is planned to support the implementation of risk assignment, unfortunately the attribution of risk category of retail and catering businesses is still different among regions. PMID:27800403

Evaluating and Improving Tech Prep: Development, Validation, and Results of the Minnesota Self-Assessment Model.

ERIC Educational Resources Information Center

Pucel, David J.; And Others

1996-01-01

The Minnesota Tech Prep Self-Evaluation System is a framework based on ideas from Deming, Juran, and the Baldridge award. Testing with 17 Minnesota tech prep consortia found it effective in identifying areas needing improvement and promoting dialog among those involved in tech prep implementation. (SK)

Assessing Impact Submissions for REF 2014: An Evaluation

ERIC Educational Resources Information Center

Manville, Catriona; Guthrie, Susan; Henham, Marie-Louise; Garrod, Bryn; Sousa, Sonia; Kirtley, Anne; Castle-Clarke, Sophie; Ling, Tom

2015-01-01

The Research Excellence Framework (REF) is a new system for assessing the quality of research in UK higher education institutions (HEIs). For the first time, part of the assessment included the wider impact of research. RAND Europe was commissioned to evaluate the assessment process of the impact element of REF submissions, and to explore the…

Establishing a Framework for Evaluation and Teacher Incentives: Considerations for Mexico

ERIC Educational Resources Information Center

OECD Publishing (NJ3), 2011

2011-01-01

Countries with underperforming education systems are recognising that effective reform is vital. But what types of programmes are likely to be effective, and how can they be implemented given local norms and conditions? This report focuses on evaluation, assessment and teacher incentives and attempts to answer these important questions for Mexico…

Evaluating How the Computer-Supported Collaborative Learning Community Fosters Critical Reflective Practices

ERIC Educational Resources Information Center

Ma, Ada W.W.

2013-01-01

In recent research, little attention has been paid to issues of methodology and analysis methods to evaluate the quality of the collaborative learning community. To address such issues, an attempt is made to adopt the Activity System Model as an analytical framework to examine the relationship between computer supported collaborative learning…

Theory, Evaluation, and Practice in Widening Participation: A Framework Approach to Assessing Impact

ERIC Educational Resources Information Center

Hayton, Annette; Bengry-Howell, Andrew

2016-01-01

The English higher education (HE) system is deeply stratified, with younger students from more privileged backgrounds comprising the majority of the student population. Over the last 15 years considerable investment has been made to widen participation but attempts to evaluate these initiatives and demonstrate impact have presented a major…

CEOS SEO and GISS Meeting

NASA Technical Reports Server (NTRS)

Killough, Brian; Stover, Shelley

2008-01-01

The Committee on Earth Observation Satellites (CEOS) provides a brief to the Goddard Institute for Space Studies (GISS) regarding the CEOS Systems Engineering Office (SEO) and current work on climate requirements and analysis. A "system framework" is provided for the Global Earth Observation System of Systems (GEOSS). SEO climate-related tasks are outlined including the assessment of essential climate variable (ECV) parameters, use of the "systems framework" to determine relevant informational products and science models and the performance of assessments and gap analyses of measurements and missions for each ECV. Climate requirements, including instruments and missions, measurements, knowledge and models, and decision makers, are also outlined. These requirements would establish traceability from instruments to products and services allowing for benefit evaluation of instruments and measurements. Additionally, traceable climate requirements would provide a better understanding of global climate models.

The Fortune of the Commons: Participatory Evaluation of Small-Scale Fisheries in the Brazilian Amazon

NASA Astrophysics Data System (ADS)

Oviedo, Antonio F. P.; Bursztyn, Marcel

2016-05-01

This paper applies a participatory approach in evaluating small-scale fisheries, focusing on the Arapaima gigas fishery in the Brazilian Amazon. The evaluation uses the social-ecological system (SES) framework, adopted to explain the conditions needed for sustainability and user cooperation in natural resources management, as a more suitable alternative to the `blueprint' or `panaceas' approaches, based only on property rights or governmental intervention. However, managers and users often do not have the necessary information compiled and available for a specific SES while some actions need to be taken immediately. Thus, consensus and negotiation among stakeholders about SES variables may be useful to evaluate system performance and indicate actions to promote sustainability. In the case study, using a consensus-building model, we found that arapaima SES leads to sustainability and is far from being a case of `tragedy of the commons.' More investments in suitable monitoring and enforcement for adaptive management are recommended. Adopting an SES framework based on stakeholders' prospects may be useful until complete interdisciplinary studies become available so as to seek of sustainability in the long term.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Knirsch, Fabian; Engel, Dominik; Neureiter, Christian

In a smart grid, data and information are transported, transmitted, stored, and processed with various stakeholders having to cooperate effectively. Furthermore, personal data is the key to many smart grid applications and therefore privacy impacts have to be taken into account. For an effective smart grid, well integrated solutions are crucial and for achieving a high degree of customer acceptance, privacy should already be considered at design time of the system. To assist system engineers in early design phase, frameworks for the automated privacy evaluation of use cases are important. For evaluation, use cases for services and software architectures needmore » to be formally captured in a standardized and commonly understood manner. In order to ensure this common understanding for all kinds of stakeholders, reference models have recently been developed. In this paper we present a model-driven approach for the automated assessment of such services and software architectures in the smart grid that builds on the standardized reference models. The focus of qualitative and quantitative evaluation is on privacy. For evaluation, the framework draws on use cases from the University of Southern California microgrid.« less

A risk-based framework to assess long-term effects of policy and water supply changes on water resources systems

NASA Astrophysics Data System (ADS)

Hassanzadeh, Elmira; Elshorbagy, Amin; Wheater, Howard; Gober, Patricia

2015-04-01

Climate uncertainty can affect water resources availability and management decisions. Sustainable water resources management therefore requires evaluation of policy and management decisions under a wide range of possible future water supply conditions. This study proposes a risk-based framework to integrate water supply uncertainty into a forward-looking decision making context. To apply this framework, a stochastic reconstruction scheme is used to generate a large ensemble of flow series. For the Rocky Mountain basins considered here, two key characteristics of the annual hydrograph are its annual flow volume and the timing of the seasonal flood peak. These are perturbed to represent natural randomness and potential changes due to future climate. 30-year series of perturbed flows are used as input to the SWAMP model - an integrated water resources model that simulates regional water supply-demand system and estimates economic productivity of water and other sustainability indicators, including system vulnerability and resilience. The simulation results are used to construct 2D-maps of net revenue of a particular water sector; e.g., hydropower, or for all sectors combined. Each map cell represents a risk scenario of net revenue based on a particular annual flow volume, timing of the peak flow, and 200 stochastic realizations of flow series. This framework is demonstrated for a water resources system in the Saskatchewan River Basin (SaskRB) in Saskatchewan, Canada. Critical historical drought sequences, derived from tree-ring reconstructions of several hundred years of annual river flows, are used to evaluate the system's performance (net revenue risk) under extremely low flow conditions and also to locate them on the previously produced 2D risk maps. This simulation and analysis framework is repeated under various reservoir operation strategies (e.g., maximizing flood protection or maximizing water supply security); development proposals, such as irrigation expansion; and change in energy prices. Such risk-based analysis demonstrates relative reduction/increase of risk associated with management and policy decisions and allow decision makers to explore the relative importance of policy versus natural water supply change in a water resources system.

Frameworks for evaluating health research capacity strengthening: a qualitative study

PubMed Central

2013-01-01

Background Health research capacity strengthening (RCS) projects are often complex and hard to evaluate. In order to inform health RCS evaluation efforts, we aimed to describe and compare key characteristics of existing health RCS evaluation frameworks: their process of development, purpose, target users, structure, content and coverage of important evaluation issues. A secondary objective was to explore what use had been made of the ESSENCE framework, which attempts to address one such issue: harmonising the evaluation requirements of different funders. Methods We identified and analysed health RCS evaluation frameworks published by seven funding agencies between 2004 and 2012, using a mixed methods approach involving structured qualitative analyses of documents, a stakeholder survey and consultations with key contacts in health RCS funding agencies. Results The frameworks were intended for use predominantly by the organisations themselves, and most were oriented primarily towards funders’ internal organisational performance requirements. The frameworks made limited reference to theories that specifically concern RCS. Generic devices, such as logical frameworks, were typically used to document activities, outputs and outcomes, but with little emphasis on exploring underlying assumptions or contextual constraints. Usage of the ESSENCE framework appeared limited. Conclusions We believe that there is scope for improving frameworks through the incorporation of more accessible information about how to do evaluation in practice; greater involvement of stakeholders, following evaluation capacity building principles; greater emphasis on explaining underlying rationales of frameworks; and structuring frameworks so that they separate generic and project-specific aspects of health RCS evaluation. The third and fourth of these improvements might assist harmonisation. PMID:24330628

The OSSE Framework at the NASA Global Modeling and Assimilation Office (GMAO)

NASA Astrophysics Data System (ADS)

Moradi, I.; Prive, N.; McCarty, W.; Errico, R. M.; Gelaro, R.

2017-12-01

This abstract summarizes the OSSE framework developed at the Global Modeling and Assimilation Office at the National Aeronautics and Space Administration (NASA/GMAO). Some of the OSSE techniques developed at GMAO including simulation of realistic observations, e.g., adding errors to simulated observations, are now widely used by the community to evaluate the impact of new observations on the weather forecasts. This talk presents some of the recent progresses and challenges in simulating realistic observations, radiative transfer modeling support for the GMAO OSSE activities, assimilation of OSSE observations into data assimilation systems, and evaluating the impact of simulated observations on the forecast skills.

The OSSE Framework at the NASA Global Modeling and Assimilation Office (GMAO)

NASA Technical Reports Server (NTRS)

Moradi, Isaac; Prive, Nikki; McCarty, Will; Errico, Ronald M.; Gelaro, Ron

2017-01-01

This abstract summarizes the OSSE framework developed at the Global Modeling and Assimilation Office at the National Aeronautics and Space Administration (NASA/GMAO). Some of the OSSE techniques developed at GMAO including simulation of realistic observations, e.g., adding errors to simulated observations, are now widely used by the community to evaluate the impact of new observations on the weather forecasts. This talk presents some of the recent progresses and challenges in simulating realistic observations, radiative transfer modeling support for the GMAO OSSE activities, assimilation of OSSE observations into data assimilation systems, and evaluating the impact of simulated observations on the forecast skills.

Stability analysis of host dynamics for hiv

NASA Astrophysics Data System (ADS)

Geetha, V.; Balamuralitharan, S.

2018-04-01

The phenomenon of disease modeling can be easily accomplished through mathematical framework. In this paper the transmission of disease in human is represented mathematically as a nonlinear system. We think about the components of the Human Immunodeficiency Virus (HIV) among the beginning periods of illness. Throughout this paper we have determined those logical representation of a three-compartmental HIV demonstrate for their stability evaluation. We tend to likewise explore the stimulating behavior of the model and acquire those Steady states for the disease-free and the endemic agreement. The framework can be evaluated by reproduction number R0. We additionally clarify the numerical recreation and their outcomes.

A Conceptual Framework for SAHRA Integrated Multi-resolution Modeling in the Rio Grande Basin

NASA Astrophysics Data System (ADS)

Liu, Y.; Gupta, H.; Springer, E.; Wagener, T.; Brookshire, D.; Duffy, C.

2004-12-01

The sustainable management of water resources in a river basin requires an integrated analysis of the social, economic, environmental and institutional dimensions of the problem. Numerical models are commonly used for integration of these dimensions and for communication of the analysis results to stakeholders and policy makers. The National Science Foundation Science and Technology Center for Sustainability of semi-Arid Hydrology and Riparian Areas (SAHRA) has been developing integrated multi-resolution models to assess impacts of climate variability and land use change on water resources in the Rio Grande Basin. These models not only couple natural systems such as surface and ground waters, but will also include engineering, economic and social components that may be involved in water resources decision-making processes. This presentation will describe the conceptual framework being developed by SAHRA to guide and focus the multiple modeling efforts and to assist the modeling team in planning, data collection and interpretation, communication, evaluation, etc. One of the major components of this conceptual framework is a Conceptual Site Model (CSM), which describes the basin and its environment based on existing knowledge and identifies what additional information must be collected to develop technically sound models at various resolutions. The initial CSM is based on analyses of basin profile information that has been collected, including a physical profile (e.g., topographic and vegetative features), a man-made facility profile (e.g., dams, diversions, and pumping stations), and a land use and ecological profile (e.g., demographics, natural habitats, and endangered species). Based on the initial CSM, a Conceptual Physical Model (CPM) is developed to guide and evaluate the selection of a model code (or numerical model) for each resolution to conduct simulations and predictions. A CPM identifies, conceptually, all the physical processes and engineering and socio-economic activities occurring (or to occur) in the real system that the corresponding numerical models are required to address, such as riparian evapotranspiration responses to vegetation change and groundwater pumping impacts on soil moisture contents. Simulation results from different resolution models and observations of the real system will then be compared to evaluate the consistency among the CSM, the CPMs, and the numerical models, and feedbacks will be used to update the models. In a broad sense, the evaluation of the models (conceptual or numerical), as well as the linkages between them, can be viewed as a part of the overall conceptual framework. As new data are generated and understanding improves, the models will evolve, and the overall conceptual framework is refined. The development of the conceptual framework becomes an on-going process. We will describe the current state of this framework and the open questions that have to be addressed in the future.

EMDS 3.0: A modeling framework for coping with complexity in environmental assessment and planning.

Treesearch

K.M. Reynolds

2006-01-01

EMDS 3.0 is implemented as an ArcMap® extension and integrates the logic engine of NetWeaver® to perform landscape evaluations, and the decision modeling engine of Criterium DecisionPlus® for evaluating management priorities. Key features of the system's evaluation component include abilities to (1) reason about large, abstract, multifaceted ecosystem management...

[Beyond the impact factor. Reflections on the book of Stefanie Haustein].

PubMed

Schubert, András

2015-09-20

The excellent book on multidimensional journal evaluation by Stefanie Haustein helps to find the place of the impact factor in the complex system of journal evaluation indicators. By delimiting the dimensions of evaluation and the user groups, the author of the book creates a framework that serves as a novel and useful guidance both for the lay reader and the expert.

An operational structured decision making framework for ...

EPA Pesticide Factsheets

Pressure to develop an operational framework for decision makers to employ the concepts of ecosystem goods and services for assessing changes to human well-being has been increasing since these concepts gained widespread notoriety after the Millennium Ecosystem Assessment Report. Many conceptual frameworks have been proposed, but most do not propose methodologies and tools to make this approach to decision making implementable. Building on common components of existing conceptual frameworks for ecosystem services and human well-being assessment we apply a structured decision making approach to develop a standardized operational framework and suggest tools and methods for completing each step. The structured decision making approach consists of six steps: 1) Clarify the Decision Context 2) Define Objectives and Evaluation Criteria 3) Develop Alternatives 4) Estimate Consequences 5) Evaluate Trade-Offs and Select and 6) Implement and Monitor. These six steps include the following activities, and suggested tools, when applied to ecosystem goods and services and human well-being conceptual frameworks: 1) Characterization of decision specific human beneficiaries using the Final Ecosystem Goods and Services (FEGS) approach and Classification System (FEGS-CS) 2) Determine beneficiaries’ relative priorities for human well-being domains in the Human Well-Being Index (HWBI) through stakeholder engagement and identify beneficiary-relevant metrics of FEGS using the Nat

Innovative use of the integrative review to evaluate evidence of technology transformation in healthcare.

PubMed

Phillips, Andrew B; Merrill, Jacqueline A

2015-12-01

Healthcare is in a period significant transformational activity through the accelerated adoption of healthcare technologies, new reimbursement systems that emphasize shared savings and care coordination, and the common place use of mobile technologies by patients, providers, and others. The complexity of healthcare creates barriers to transformational activity and has the potential to inhibit the desired paths toward change envisioned by policymakers. Methods for understanding how change is occurring within this complex environment are important to the evaluation of delivery system reform and the role of technology in healthcare transformation. This study examines the use on an integrative review methodology to evaluate the healthcare literature for evidence of technology transformation in healthcare. The methodology integrates the evaluation of a broad set of literature with an established evaluative framework to develop a more complete understanding of a particular topic. We applied this methodology and the framework of punctuated equilibrium (PEq) to the analysis of the healthcare literature from 2004 to 2012 for evidence of technology transformation, a time during which technology was at the forefront of healthcare policy. The analysis demonstrated that the established PEq framework applied to the literature showed considerable potential for evaluating the progress of policies that encourage healthcare transformation. Significant inhibitors to change were identified through the integrative review and categorized into ten themes that describe the resistant structure of healthcare delivery: variations in the environment; market complexity; regulations; flawed risks and rewards; change theories; barriers; ethical considerations; competition and sustainability; environmental elements, and internal elements. We hypothesize that the resistant nature of the healthcare system described by this study creates barriers to the direct consumer involvement and engagement necessary for transformational change. Future policies should be directed at removing these barriers by demanding and emphasizing open technologies and unrestricted access to data versus as currently prescribed by technology vendors, practitioners, and policies that perpetuate market equilibrium. Copyright © 2015 Elsevier Inc. All rights reserved.

Surface Pressure Dependencies in the GEOS-Chem-Adjoint System and the Impact of the GEOS-5 Surface Pressure on CO2 Model Forecast

NASA Technical Reports Server (NTRS)

Lee, Meemong; Weidner, Richard

2016-01-01

In the GEOS-Chem Adjoint (GCA) system, the total (wet) surface pressure of the GEOS meteorology is employed as dry surface pressure, ignoring the presence of water vapor. The Jet Propulsion Laboratory (JPL) Carbon Monitoring System (CMS) research team has been evaluating the impact of the above discrepancy on the CO2 model forecast and the CO2 flux inversion. The JPL CMS research utilizes a multi-mission assimilation framework developed by the Multi-Mission Observation Operator (M2O2) research team at JPL extending the GCA system. The GCA-M2O2 framework facilitates mission-generic 3D and 4D-variational assimilations streamlining the interfaces to the satellite data products and prior emission inventories. The GCA-M2O2 framework currently integrates the GCA system version 35h and provides a dry surface pressure setup to allow the CO2 model forecast to be performed with the GEOS-5 surface pressure directly or after converting it to dry surface pressure.

Surface Pressure Dependencies in the Geos-Chem-Adjoint System and the Impact of the GEOS-5 Surface Pressure on CO2 Model Forecast

NASA Technical Reports Server (NTRS)

Lee, Meemong; Weidner, Richard

2016-01-01

In the GEOS-Chem Adjoint (GCA) system, the total (wet) surface pressure of the GEOS meteorology is employed as dry surface pressure, ignoring the presence of water vapor. The Jet Propulsion Laboratory (JPL) Carbon Monitoring System (CMS) research team has been evaluating the impact of the above discrepancy on the CO2 model forecast and the CO2 flux inversion. The JPL CMS research utilizes a multi-mission assimilation framework developed by the Multi-Mission Observation Operator (M2O2) research team at JPL extending the GCA system. The GCA-M2O2 framework facilitates mission-generic 3D and 4D-variational assimilations streamlining the interfaces to the satellite data products and prior emission inventories. The GCA-M2O2 framework currently integrates the GCA system version 35h and provides a dry surface pressure setup to allow the CO2 model forecast to be performed with the GEOS-5 surface pressure directly or after converting it to dry surface pressure.

A framework for performance and data quality assessment of Radio Frequency IDentification (RFID) systems in health care settings.

PubMed

van der Togt, Remko; Bakker, Piet J M; Jaspers, Monique W M

2011-04-01

RFID offers great opportunities to health care. Nevertheless, prior experiences also show that RFID systems have not been designed and tested in response to the particular needs of health care settings and might introduce new risks. The aim of this study is to present a framework that can be used to assess the performance of RFID systems particularly in health care settings. We developed a framework describing a systematic approach that can be used for assessing the feasibility of using an RFID technology in a particular healthcare setting; more specific for testing the impact of environmental factors on the quality of RFID generated data and vice versa. This framework is based on our own experiences with an RFID pilot implementation in an academic hospital in The Netherlands and a literature review concerning RFID test methods and current insights of RFID implementations in healthcare. The implementation of an RFID system within the blood transfusion chain inside a hospital setting was used as a show case to explain the different phases of the framework. The framework consists of nine phases, including an implementation development plan, RFID and medical equipment interference tests, data accuracy- and data completeness tests to be run in laboratory, simulated field and real field settings. The potential risks that RFID technologies may bring to the healthcare setting should be thoroughly evaluated before they are introduced into a vital environment. The RFID performance assessment framework that we present can act as a reference model to start an RFID development, engineering, implementation and testing plan and more specific, to assess the potential risks of interference and to test the quality of the RFID generated data potentially influenced by physical objects in specific health care environments. Copyright © 2010 Elsevier Inc. All rights reserved.

DyKOSMap: A framework for mapping adaptation between biomedical knowledge organization systems.

PubMed

Dos Reis, Julio Cesar; Pruski, Cédric; Da Silveira, Marcos; Reynaud-Delaître, Chantal

2015-06-01

Knowledge Organization Systems (KOS) and their associated mappings play a central role in several decision support systems. However, by virtue of knowledge evolution, KOS entities are modified over time, impacting mappings and potentially turning them invalid. This requires semi-automatic methods to maintain such semantic correspondences up-to-date at KOS evolution time. We define a complete and original framework based on formal heuristics that drives the adaptation of KOS mappings. Our approach takes into account the definition of established mappings, the evolution of KOS and the possible changes that can be applied to mappings. This study experimentally evaluates the proposed heuristics and the entire framework on realistic case studies borrowed from the biomedical domain, using official mappings between several biomedical KOSs. We demonstrate the overall performance of the approach over biomedical datasets of different characteristics and sizes. Our findings reveal the effectiveness in terms of precision, recall and F-measure of the suggested heuristics and methods defining the framework to adapt mappings affected by KOS evolution. The obtained results contribute and improve the quality of mappings over time. The proposed framework can adapt mappings largely automatically, facilitating thus the maintenance task. The implemented algorithms and tools support and minimize the work of users in charge of KOS mapping maintenance. Copyright © 2015 Elsevier Inc. All rights reserved.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Marara, Madeleine; Okello, Nick; Kuhanwa, Zainab

This paper reviews and compares the condition of the environmental impact assessment (EIA) system in three countries in the East Africa region: Kenya, Rwanda and Tanzania. The criteria used for the evaluation and the comparison of each system are based on the elements of the legal, administrative and procedural frameworks, as well as the context in which they operate. These criteria are adapted from the evaluation and quality control criteria derived from a number of literature sources. The study reveals that the EIA systems of Kenya and Tanzania are at a similar stage in their development. The two countries, themore » first to introduce the EIA concept into their jurisdiction in this part of Africa, therefore have more experience than Rwanda in the practice of environmental impact assessment, where the legislation and process requires more time to mature both from the governmental and societal perspective. The analysis of the administrative and procedural frameworks highlights the weakness in the autonomy of the competent authority, in all three countries. Finally a major finding of this study is that the contextual set up i.e. the socio-economic and political situation plays an important role in the performance of an EIA system. The context in developing countries is very different from developed countries where the EIA concept originates. Interpreting EIA conditions in countries like Kenya, Rwanda and Tanzania requires that the analysis for determining the effectiveness of their systems should be undertaken within a relevant framework, taking into account the specific requirements of those countries.« less

Developing a framework for qualitative engineering: Research in design and analysis of complex structural systems

NASA Technical Reports Server (NTRS)

Franck, Bruno M.

1990-01-01

The research is focused on automating the evaluation of complex structural systems, whether for the design of a new system or the analysis of an existing one, by developing new structural analysis techniques based on qualitative reasoning. The problem is to identify and better understand: (1) the requirements for the automation of design, and (2) the qualitative reasoning associated with the conceptual development of a complex system. The long-term objective is to develop an integrated design-risk assessment environment for the evaluation of complex structural systems. The scope of this short presentation is to describe the design and cognition components of the research. Design has received special attention in cognitive science because it is now identified as a problem solving activity that is different from other information processing tasks (1). Before an attempt can be made to automate design, a thorough understanding of the underlying design theory and methodology is needed, since the design process is, in many cases, multi-disciplinary, complex in size and motivation, and uses various reasoning processes involving different kinds of knowledge in ways which vary from one context to another. The objective is to unify all the various types of knowledge under one framework of cognition. This presentation focuses on the cognitive science framework that we are using to represent the knowledge aspects associated with the human mind's abstraction abilities and how we apply it to the engineering knowledge and engineering reasoning in design.

Geologic framework influences on the geomorphology of an anthropogenically modified barrier island: Assessment of dune/beach changes at Fire Island, New York

USGS Publications Warehouse

Lentz, E.E.; Hapke, C.J.

2011-01-01

Antecedent geology plays a crucial role in determining the inner-shelf, nearshore, and onshore geomorphology observed in coastal systems. However, the influence of the geologic framework on a system is difficult to extract when evaluating responses to changes due to storms and anthropogenic modifications, and few studies have quantified the potential for these influences in dune/beach environments. This study evaluates topographic change to the dune/beach system at Fire Island, New York over a ten year period (1998-2008) at two sites representing eastern and western reaches of the island where morphology has been shown to vary. The sites are situated along swaths of coast eroding differentially and where the inner shelf geologic framework differs substantially. Fewer large storms occurred in the first half of the study period, compared with the later part of the study period which includes several severe and prolonged extratropical storms. Additionally, a major beach replenishment project was conducted at one of the study sites. Topographic data from LiDAR and RTK GPS surveys are used to construct high-resolution 3D surfaces, which are used to determine volumetric change and to extract 2D alongshore features and profiles for analysis. The study sites help to further characterize morphologic differences between eastern and western reaches of the island. The western site displays higher sand volumes, lower dunes, and a lower gradient profile slope when compared with the eastern site. In addition to these fundamental morphologic differences, the two sites also differ significantly in their response to coastal storms and in the fact that their replenishment histories are different. The replenished areas show reduced vulnerability to storms through minimal volume loss and shoreline accretion that should be considered when evaluating the response of replenished areas to episodic events. We propose that site-specific differences evident throughout the study period can be linked to alongshore variations in the framework geology of the system. Anthropogenic modifications may have intensified differences already inherent in the system. ?? 2010 Elsevier B.V.

L.E.A.D.: a framework for evidence gathering and use for the prevention of obesity and other complex public health problems.

PubMed

Chatterji, Madhabi; Green, Lawrence W; Kumanyika, Shiriki

2014-02-01

This article summarizes a comprehensive, systems-oriented framework designed to improve the use of a wide variety of evidence sources to address population-wide obesity problems. The L.E.A.D. framework (for Locate the evidence, Evaluate the evidence, Assemble the evidence, and inform Decisions), developed by an expert consensus committee convened by the Institute of Medicine, is broadly applicable to complex, community-wide health problems. The article explains how to use the framework, presenting an evidence typology that helps specify relevant research questions and includes examples of how particular research methodologies and sources of evidence relate to questions that stem from decision-maker needs. The utility of a range of quantitative, qualitative, and mixed method designs and data sources for assembling a broad and credible evidence base is discussed, with a call for ongoing "evidence generation" to fill information gaps using the recommended systems perspective.

An Optimization Framework for Driver Feedback Systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Malikopoulos, Andreas; Aguilar, Juan P.

2013-01-01

Modern vehicles have sophisticated electronic control units that can control engine operation with discretion to balance fuel economy, emissions, and power. These control units are designed for specific driving conditions (e.g., different speed profiles for highway and city driving). However, individual driving styles are different and rarely match the specific driving conditions for which the units were designed. In the research reported here, we investigate driving-style factors that have a major impact on fuel economy and construct an optimization framework to optimize individual driving styles with respect to these driving factors. In this context, we construct a set of polynomialmore » metamodels to reflect the responses produced in fuel economy by changing the driving factors. Then, we compare the optimized driving styles to the original driving styles and evaluate the effectiveness of the optimization framework. Finally, we use this proposed framework to develop a real-time feedback system, including visual instructions, to enable drivers to alter their driving styles in response to actual driving conditions to improve fuel efficiency.« less

Assessing the impact of healthcare research: A systematic review of methodological frameworks.

PubMed

Cruz Rivera, Samantha; Kyte, Derek G; Aiyegbusi, Olalekan Lee; Keeley, Thomas J; Calvert, Melanie J

2017-08-01

Increasingly, researchers need to demonstrate the impact of their research to their sponsors, funders, and fellow academics. However, the most appropriate way of measuring the impact of healthcare research is subject to debate. We aimed to identify the existing methodological frameworks used to measure healthcare research impact and to summarise the common themes and metrics in an impact matrix. Two independent investigators systematically searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), the Excerpta Medica Database (EMBASE), the Cumulative Index to Nursing and Allied Health Literature (CINAHL+), the Health Management Information Consortium, and the Journal of Research Evaluation from inception until May 2017 for publications that presented a methodological framework for research impact. We then summarised the common concepts and themes across methodological frameworks and identified the metrics used to evaluate differing forms of impact. Twenty-four unique methodological frameworks were identified, addressing 5 broad categories of impact: (1) 'primary research-related impact', (2) 'influence on policy making', (3) 'health and health systems impact', (4) 'health-related and societal impact', and (5) 'broader economic impact'. These categories were subdivided into 16 common impact subgroups. Authors of the included publications proposed 80 different metrics aimed at measuring impact in these areas. The main limitation of the study was the potential exclusion of relevant articles, as a consequence of the poor indexing of the databases searched. The measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise research benefit, and to help minimise research waste. This review provides a collective summary of existing methodological frameworks for research impact, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.

Trial Implementation of a Multihazard Risk Assessment Framework for High-Impact Low-Frequency Power Grid Events

DOE Office of Scientific and Technical Information (OSTI.GOV)

Veeramany, Arun; Coles, Garill A.; Unwin, Stephen D.

The Pacific Northwest National Laboratory developed a risk framework for modeling high-impact, low-frequency power grid events to support risk-informed decisions. In this paper, we briefly recap the framework and demonstrate its implementation for seismic and geomagnetic hazards using a benchmark reliability test system. We describe integration of a collection of models implemented to perform hazard analysis, fragility evaluation, consequence estimation, and postevent restoration. We demonstrate the value of the framework as a multihazard power grid risk assessment and management tool. As a result, the research will benefit transmission planners and emergency planners by improving their ability to maintain a resilientmore » grid infrastructure against impacts from major events.« less

Trial Implementation of a Multihazard Risk Assessment Framework for High-Impact Low-Frequency Power Grid Events

DOE PAGES

Veeramany, Arun; Coles, Garill A.; Unwin, Stephen D.; ...

2017-08-25

The Pacific Northwest National Laboratory developed a risk framework for modeling high-impact, low-frequency power grid events to support risk-informed decisions. In this paper, we briefly recap the framework and demonstrate its implementation for seismic and geomagnetic hazards using a benchmark reliability test system. We describe integration of a collection of models implemented to perform hazard analysis, fragility evaluation, consequence estimation, and postevent restoration. We demonstrate the value of the framework as a multihazard power grid risk assessment and management tool. As a result, the research will benefit transmission planners and emergency planners by improving their ability to maintain a resilientmore » grid infrastructure against impacts from major events.« less

IPCC reasons for concern regarding climate change risks

NASA Astrophysics Data System (ADS)

O'Neill, Brian C.; Oppenheimer, Michael; Warren, Rachel; Hallegatte, Stephane; Kopp, Robert E.; Pörtner, Hans O.; Scholes, Robert; Birkmann, Joern; Foden, Wendy; Licker, Rachel; Mach, Katharine J.; Marbaix, Phillippe; Mastrandrea, Michael D.; Price, Jeff; Takahashi, Kiyoshi; van Ypersele, Jean-Pascal; Yohe, Gary

2017-01-01

The reasons for concern framework communicates scientific understanding about risks in relation to varying levels of climate change. The framework, now a cornerstone of the IPCC assessments, aggregates global risks into five categories as a function of global mean temperature change. We review the framework's conceptual basis and the risk judgments made in the most recent IPCC report, confirming those judgments in most cases in the light of more recent literature and identifying their limitations. We point to extensions of the framework that offer complementary climate change metrics to global mean temperature change and better account for possible changes in social and ecological system vulnerability. Further research should systematically evaluate risks under alternative scenarios of future climatic and societal conditions.

Different CAD/CAM-processing routes for zirconia restorations: influence on fitting accuracy.

PubMed

Kohorst, Philipp; Junghanns, Janet; Dittmer, Marc P; Borchers, Lothar; Stiesch, Meike

2011-08-01

The aim of the present in vitro study was to evaluate the influence of different processing routes on the fitting accuracy of four-unit zirconia fixed dental prostheses (FDPs) fabricated by computer-aided design/computer-aided manufacturing (CAD/CAM). Three groups of zirconia frameworks with ten specimens each were fabricated. Frameworks of one group (CerconCAM) were produced by means of a laboratory CAM-only system. The other frameworks were made with different CAD/CAM systems; on the one hand by in-laboratory production (CerconCAD/CAM) and on the other hand by centralized production in a milling center (Compartis) after forwarding geometrical data. Frameworks were then veneered with the recommended ceramics, and marginal accuracy was determined using a replica technique. Horizontal marginal discrepancy, vertical marginal discrepancy, absolute marginal discrepancy, and marginal gap were evaluated. Statistical analyses were performed by one-way analysis of variance (ANOVA), with the level of significance chosen at 0.05. Mean horizontal discrepancies ranged between 22 μm (CerconCAM) and 58 μm (Compartis), vertical discrepancies ranged between 63 μm (CerconCAD/CAM) and 162 μm (CerconCAM), and absolute marginal discrepancies ranged between 94 μm (CerconCAD/CAM) and 181 μm (CerconCAM). The marginal gap varied between 72 μm (CerconCAD/CAM) and 112 μm (CerconCAM, Compartis). Statistical analysis revealed that, with all measurements, the marginal accuracy of the zirconia FDPs was significantly influenced by the processing route used (p < 0.05). Within the limitations of this study, all restorations showed a clinically acceptable marginal accuracy; however, the results suggest that the CAD/CAM systems are more precise than the CAM-only system for the manufacture of four-unit FDPs.

An evidence-based framework to measure quality of allied health care.

PubMed

Grimmer, Karen; Lizarondo, Lucylynn; Kumar, Saravana; Bell, Erica; Buist, Michael; Weinstein, Philip

2014-02-26

There is no standard way of describing the complexities of allied health (AH) care, or its quality. AH is an umbrella term which excludes medicine and nursing, and variably includes disciplines which provide therapy, diagnostic, or scientific services. This paper outlines a framework for a standard approach to evaluate the quality of AH therapy services. A realist synthesis framework describing what AH does, how it does it, and what is achieved, was developed. This was populated by the findings of a systematic review of literature published since 1980 reporting concepts of quality relevant to AH. Articles were included on quality measurement concepts, theories, debates, and/or hypothetical frameworks. Of 139 included articles, 21 reported on descriptions of quality potentially relevant to AH. From these, 24 measures of quality were identified, with 15 potentially relating to what AH does, 17 to how AH delivers care, 8 relating to short term functional outcomes, and 9 relating to longer term functional and health system outcomes. A novel evidence-based quality framework was proposed to address the complexity of AH therapies. This should assist in better evaluation of AH processes and outcomes, costs, and evidence-based engagement of AH providers in healthcare teams.

Patient assessment of medication information leaflets and validation of the Evaluative Linguistic Framework (ELF).

PubMed

Hirsh, Di; Clerehan, Rosemary; Staples, Margaret; Osborne, Richard H; Buchbinder, Rachelle

2009-11-01

To obtain patient feedback about the structure and quality of medication information leaflets and validate the usefulness of the Evaluative Linguistic Framework (ELF) for improving written communication with patients. Triangulated feedback about a set of rheumatoid arthritis (RA) medication leaflets, some developed with knowledge of the ELF, was obtained from 27 people with RA from interviews, focus group discussion and self-administered questionnaires. The principal elements of the framework were investigated: overall generic structure and functions of each stage, interpersonal relationship between writer and reader, technicality of language and density of information. Participant assessments of the leaflets aligned with the framework in terms of what constituted a good leaflet. While the main purpose of the leaflets was identified as being information provision, participants also wanted clear instructions, benefits to be highlighted and side effects to be comprehensively listed. For comprehensiveness and user-friendliness, leaflets developed with guidance of the ELF were consistently preferred. According to people with RA, leaflets generated from a linguistic framework are clearer and more effective in communicating information about medications. The ELF is a user-friendly, structured analytic system that can assist with the development of effective high quality patient information materials.

Using the systems framework for postural control to analyze the components of balance evaluated in standardized balance measures: a scoping review.

PubMed

Sibley, Kathryn M; Beauchamp, Marla K; Van Ooteghem, Karen; Straus, Sharon E; Jaglal, Susan B

2015-01-01

To identify components of postural control included in standardized balance measures for adult populations. Electronic searches of MEDLINE, EMBASE, and CINAHL databases using keyword combinations of postural balance/equilibrium, psychometrics/reproducibility of results/predictive value of tests/validation studies, instrument construction/instrument validation, geriatric assessment/disability evaluation, gray literature, and hand searches. Inclusion criteria were measures with a stated objective to assess balance, adult populations (18y and older), at least 1 psychometric evaluation, 1 standing task, a standardized protocol and evaluation criteria, and published in English. Two reviewers independently identified studies for inclusion. Sixty-six measures were included. A research assistant extracted descriptive characteristics and 2 reviewers independently coded components of balance in each measure using the Systems Framework for Postural Control, a widely recognized model of balance. Components of balance evaluated in these measures were underlying motor systems (100% of measures), anticipatory postural control (71%), dynamic stability (67%), static stability (64%), sensory integration (48%), functional stability limits (27%), reactive postural control (23%), cognitive influences (17%), and verticality (8%). Thirty-four measures evaluated 3 or fewer components of balance, and 1 measure-the Balance Evaluation Systems Test-evaluated all components of balance. Several standardized balance measures provide only partial information on postural control and omit important components of balance related to avoiding falls. As such, the choice of measure(s) may limit the overall interpretation of an individual's balance ability. Continued work is necessary to increase the implementation of comprehensive balance assessment in research and practice. Copyright © 2015 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Transportation Impact Evaluation System

DOT National Transportation Integrated Search

1979-11-01

This report specifies a framework for spatial analysis and the general modelling steps required. It also suggests available urban and regional data sources, along with some typical existing urban and regional models. The goal is to develop a computer...

Examining the effectiveness of municipal solid waste management systems: an integrated cost-benefit analysis perspective with a financial cost modeling in Taiwan.

PubMed

Weng, Yu-Chi; Fujiwara, Takeshi

2011-06-01

In order to develop a sound material-cycle society, cost-effective municipal solid waste (MSW) management systems are required for the municipalities in the context of the integrated accounting system for MSW management. Firstly, this paper attempts to establish an integrated cost-benefit analysis (CBA) framework for evaluating the effectiveness of MSW management systems. In this paper, detailed cost/benefit items due to waste problems are particularly clarified. The stakeholders of MSW management systems, including the decision-makers of the municipalities and the citizens, are expected to reconsider the waste problems in depth and thus take wise actions with the aid of the proposed CBA framework. Secondly, focusing on the financial cost, this study develops a generalized methodology to evaluate the financial cost-effectiveness of MSW management systems, simultaneously considering the treatment technological levels and policy effects. The impacts of the influencing factors on the annual total and average financial MSW operation and maintenance (O&M) costs are analyzed in the Taiwanese case study with a demonstrative short-term future projection of the financial costs under scenario analysis. The established methodology would contribute to the evaluation of the current policy measures and to the modification of the policy design for the municipalities. Crown Copyright © 2011. Published by Elsevier Ltd. All rights reserved.

Automated software system for checking the structure and format of ACM SIG documents

NASA Astrophysics Data System (ADS)

Mirza, Arsalan Rahman; Sah, Melike

2017-04-01

Microsoft (MS) Office Word is one of the most commonly used software tools for creating documents. MS Word 2007 and above uses XML to represent the structure of MS Word documents. Metadata about the documents are automatically created using Office Open XML (OOXML) syntax. We develop a new framework, which is called ADFCS (Automated Document Format Checking System) that takes the advantage of the OOXML metadata, in order to extract semantic information from MS Office Word documents. In particular, we develop a new ontology for Association for Computing Machinery (ACM) Special Interested Group (SIG) documents for representing the structure and format of these documents by using OWL (Web Ontology Language). Then, the metadata is extracted automatically in RDF (Resource Description Framework) according to this ontology using the developed software. Finally, we generate extensive rules in order to infer whether the documents are formatted according to ACM SIG standards. This paper, introduces ACM SIG ontology, metadata extraction process, inference engine, ADFCS online user interface, system evaluation and user study evaluations.

A software tool for ecosystem services assessments

NASA Astrophysics Data System (ADS)

Riegels, Niels; Klinting, Anders; Butts, Michael; Middelboe, Anne Lise; Mark, Ole

2017-04-01

The EU FP7 DESSIN project is developing methods and tools for assessment of ecosystem services (ESS) and associated economic values, with a focus on freshwater ESS in urban settings. Although the ESS approach has gained considerable visibility over the past ten years, operationalizing the approach remains a challenge. Therefore, DESSSIN is also supporting development of a free software tool to support users implementing the DESSIN ESS evaluation framework. The DESSIN ESS evaluation framework is a structured approach to measuring changes in ecosystem services. The main purpose of the framework is to facilitate the application of the ESS approach in the appraisal of projects that have impacts on freshwater ecosystems and their services. The DESSIN framework helps users evaluate changes in ESS by linking biophysical, economic, and sustainability assessments sequentially. It was developed using the Common International Classification of Ecosystem Services (CICES) and the DPSIR (Drivers, Pressures, States, Impacts, Responses) adaptive management cycle. The former is a standardized system for the classification of ESS developed by the European Union to enhance the consistency and comparability of ESS assessments. The latter is a well-known concept to disentangle the biophysical and social aspects of a system under study. As part of its analytical component, the DESSIN framework also integrates elements of the Final Ecosystem Goods and Services-Classification System (FEGS-CS) of the US Environmental Protection Agency (USEPA). As implemented in the software tool, the DESSIN framework consists of five parts: • In part I of the evaluation, the ecosystem is defined and described and the local stakeholders are identified. In addition, administrative details and objectives of the assessment are defined. • In part II, drivers and pressures are identified. Once these first two elements of the DPSIR scheme have been characterized, the claimed/expected capabilities of a proposed project can be estimated to determine whether the project affects drivers, pressures, states or a combination of these. • In part III, information about impacts on drivers, pressures, and states is used to identify ESS impacted by a proposed project. Potential beneficiaries of impacted ESS are also identified. • In part IV, changes in ESS are estimated. These estimates include changes in the provision of ESS, the use of ESS, and the value of ESS. • A sustainability assessment in Part V estimates the broader impact of a proposed project according to social, environmental, governance and other criteria. The ESS evaluation software tool is designed to assist an evaluation or study leader carrying out an ESS assessment. The tool helps users move through the logic of the ESS evaluation and make sense of relationships between elements of the DPSIR framework, the CICES classification scheme, and the FEGS approach. The tool also provides links to useful indicators and assessment methods in order to help users quantify changes in ESS and ESS values. The software tool is developed in collaboration with the DESSIN user group, who will use the software to estimate changes in ESS resulting from the implementation of green technologies addressing water quality and water scarcity issues. Although the software is targeted to this user group, it will be made available for free to the public after the conclusion of the project.

Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods

PubMed Central

2010-01-01

Background Schools are key settings for health promotion (HP) but the development of suitable approaches for evaluating HP in schools is still a major topic of discussion. This article presents a research protocol of a program developed to evaluate HP. After reviewing HP evaluation issues, the various possible approaches are analyzed and the importance of a realistic evaluation framework and a mixed methods (MM) design are demonstrated. Methods/Design The design is based on a systemic approach to evaluation, taking into account the mechanisms, context and outcomes, as defined in realistic evaluation, adjusted to our own French context using an MM approach. The characteristics of the design are illustrated through the evaluation of a nationwide HP program in French primary schools designed to enhance children's social, emotional and physical health by improving teachers' HP practices and promoting a healthy school environment. An embedded MM design is used in which a qualitative data set plays a supportive, secondary role in a study based primarily on a different quantitative data set. The way the qualitative and quantitative approaches are combined through the entire evaluation framework is detailed. Discussion This study is a contribution towards the development of suitable approaches for evaluating HP programs in schools. The systemic approach of the evaluation carried out in this research is appropriate since it takes account of the limitations of traditional evaluation approaches and considers suggestions made by the HP research community. PMID:20109202

Implementing disability evaluation and welfare services based on the framework of the international classification of functioning, disability and health: experiences in Taiwan

PubMed Central

2013-01-01

Background Before 2007, the disability evaluation was based on the medical model in Taiwan. According to the People with Disabilities Rights Protection Act, from 2012 the assessment of a person’s eligibility for disability benefits has to be determined based on the International Classification of Functioning, Disability, and Health (ICF) framework nationwide. The purposes of this study were to: 1) design the evaluation tools for disability eligibility system based on the ICF/ICF-Children and Youth; 2) compare the differences of grades of disability between the old and new evaluation systems; 3) analyse the outcome of the new disability evaluation system. Methods To develop evaluation tools and procedure for disability determination, we formed an implementation taskforce, including 199 professional experts, and conducted a small-scale field trial to examine the feasibility of evaluation tools in Phase I. To refine the evaluation tools and process and to compare the difference of the grades of disability between new and old systems, 7,329 persons with disabilities were randomly recruited in a national population-based study in Phase II. To implement the new system smoothly and understand the impact of the new system, the collaboration mechanism was established and data of 168,052 persons who applied for the disability benefits was extracted from the information system and analysed in Phase III. Results The measures of the 43 categories for body function/structure components, the Functioning Scale of Disability Evaluation System for activities/participation components, and the needs assessment have been developed and used in the field after several revisions. In Phase II, there was 49.7% agreement of disability grades between the old and new systems. In Phase III, 110,667 persons with a disability received their welfare services through the new system. Among them, 77% received basic social welfare support, 89% financial support, 24% allowance for assistive technology, 7% caregiver support, 8% nursing care and rehabilitation services at home, and 47% were issued parking permits for persons with disability. Conclusion This study demonstrated that disability evaluation system based on the ICF could provide a common language between disability assessment, needs assessment and welfare services. However, the proposed assessment protocol and tools require additional testing and validation. PMID:24125482

Framework for modelling the cost-effectiveness of systemic interventions aimed to reduce youth delinquency.

PubMed

Schawo, Saskia J; van Eeren, Hester; Soeteman, Djira I; van der Veldt, Marie-Christine; Noom, Marc J; Brouwer, Werner; Busschbach, Jan J V; Hakkaart, Leona

2012-12-01

Many interventions initiated within and financed from the health care sector are not necessarily primarily aimed at improving health. This poses important questions regarding the operationalisation of economic evaluations in such contexts. We investigated whether assessing cost-effectiveness using state-of-the-art methods commonly applied in health care evaluations is feasible and meaningful when evaluating interventions aimed at reducing youth delinquency. A probabilistic Markov model was constructed to create a framework for the assessment of the cost-effectiveness of systemic interventions in delinquent youth. For illustrative purposes, Functional Family Therapy (FFT), a systemic intervention aimed at improving family functioning and, primarily, reducing delinquent activity in youths, was compared to Treatment as Usual (TAU). "Criminal activity free years" (CAFYs) were introduced as central outcome measure. Criminal activity may e.g. be based on police contacts or committed crimes. In absence of extensive data and for illustrative purposes the current study based criminal activity on available literature on recidivism. Furthermore, a literature search was performed to deduce the model's structure and parameters. Common cost-effectiveness methodology could be applied to interventions for youth delinquency. Model characteristics and parameters were derived from literature and ongoing trial data. The model resulted in an estimate of incremental costs/CAFY and included long-term effects. Illustrative model results point towards dominance of FFT compared to TAU. Using a probabilistic model and the CAFY outcome measure to assess cost-effectiveness of systemic interventions aimed to reduce delinquency is feasible. However, the model structure is limited to three states and the CAFY measure was defined rather crude. Moreover, as the model parameters are retrieved from literature the model results are illustrative in the absence of empirical data. The current model provides a framework to assess the cost-effectiveness of systemic interventions, while taking into account parameter uncertainty and long-term effectiveness. The framework of the model could be used to assess the cost-effectiveness of systemic interventions alongside (clinical) trial data. Consequently, it is suitable to inform reimbursement decisions, since the value for money of systemic interventions can be demonstrated using a decision analytic model. Future research could be focussed on testing the current model based on extensive empirical data, improving the outcome measure and finding appropriate values for that outcome.

Tryout and Evaluation of Prototype LMS (Learning Mastery System) Training System Under Exclusive Use Agreement.

ERIC Educational Resources Information Center

McDonald, Cheryl; Hylton, John A.

In 1970-1971 Learning Mastery System (LMS) materials were made available to schools within the state of California under an Exclusive Use Agreement. The LMS is a set of materials and procedures prepared by the Southwest Regional Laboratory (SWRL) as an objectives-based framework to assist in managing the learning activities of existing reading…

A Framework for Evaluation and Optimization of Relevance and Novelty-Based Retrieval

ERIC Educational Resources Information Center

Lad, Abhimanyu

2011-01-01

There has been growing interest in building and optimizing retrieval systems with respect to relevance and novelty of information, which together more realistically reflect the usefulness of a system as perceived by the user. How to combine these criteria into a single metric that can be used to measure as well as optimize retrieval systems is an…

Health services research evaluation principles. Broadening a general framework for evaluating health information technology.

PubMed

Sockolow, P S; Crawford, P R; Lehmann, H P

2012-01-01

Our forthcoming national experiment in increased health information technology (HIT) adoption funded by the American Recovery and Reinvestment Act of 2009 will require a comprehensive approach to evaluating HIT. The quality of evaluation studies of HIT to date reveals a need for broader evaluation frameworks that limits the generalizability of findings and the depth of lessons learned. Develop an informatics evaluation framework for health information technology (HIT) integrating components of health services research (HSR) evaluation and informatics evaluation to address identified shortcomings in available HIT evaluation frameworks. A systematic literature review updated and expanded the exhaustive review by Ammenwerth and deKeizer (AdK). From retained studies, criteria were elicited and organized into classes within a framework. The resulting Health Information Technology Research-based Evaluation Framework (HITREF) was used to guide clinician satisfaction survey construction, multi-dimensional analysis of data, and interpretation of findings in an evaluation of a vanguard community health care EHR. The updated review identified 128 electronic health record (EHR) evaluation studies and seven evaluation criteria not in AdK: EHR Selection/Development/Training; Patient Privacy Concerns; Unintended Consequences/ Benefits; Functionality; Patient Satisfaction with EHR; Barriers/Facilitators to Adoption; and Patient Satisfaction with Care. HITREF was used productively and was a complete evaluation framework which included all themes that emerged. We can recommend to future EHR evaluators that they consider adding a complete, research-based HIT evaluation framework, such as HITREF, to their evaluation tools suite to monitor HIT challenges as the federal government strives to increase HIT adoption.

From Determinism and Probability to Chaos: Chaotic Evolution towards Philosophy and Methodology of Chaotic Optimization

PubMed Central

2015-01-01

We present and discuss philosophy and methodology of chaotic evolution that is theoretically supported by chaos theory. We introduce four chaotic systems, that is, logistic map, tent map, Gaussian map, and Hénon map, in a well-designed chaotic evolution algorithm framework to implement several chaotic evolution (CE) algorithms. By comparing our previous proposed CE algorithm with logistic map and two canonical differential evolution (DE) algorithms, we analyse and discuss optimization performance of CE algorithm. An investigation on the relationship between optimization capability of CE algorithm and distribution characteristic of chaotic system is conducted and analysed. From evaluation result, we find that distribution of chaotic system is an essential factor to influence optimization performance of CE algorithm. We propose a new interactive EC (IEC) algorithm, interactive chaotic evolution (ICE) that replaces fitness function with a real human in CE algorithm framework. There is a paired comparison-based mechanism behind CE search scheme in nature. A simulation experimental evaluation is conducted with a pseudo-IEC user to evaluate our proposed ICE algorithm. The evaluation result indicates that ICE algorithm can obtain a significant better performance than or the same performance as interactive DE. Some open topics on CE, ICE, fusion of these optimization techniques, algorithmic notation, and others are presented and discussed. PMID:25879067

From determinism and probability to chaos: chaotic evolution towards philosophy and methodology of chaotic optimization.

PubMed

Pei, Yan

2015-01-01

We present and discuss philosophy and methodology of chaotic evolution that is theoretically supported by chaos theory. We introduce four chaotic systems, that is, logistic map, tent map, Gaussian map, and Hénon map, in a well-designed chaotic evolution algorithm framework to implement several chaotic evolution (CE) algorithms. By comparing our previous proposed CE algorithm with logistic map and two canonical differential evolution (DE) algorithms, we analyse and discuss optimization performance of CE algorithm. An investigation on the relationship between optimization capability of CE algorithm and distribution characteristic of chaotic system is conducted and analysed. From evaluation result, we find that distribution of chaotic system is an essential factor to influence optimization performance of CE algorithm. We propose a new interactive EC (IEC) algorithm, interactive chaotic evolution (ICE) that replaces fitness function with a real human in CE algorithm framework. There is a paired comparison-based mechanism behind CE search scheme in nature. A simulation experimental evaluation is conducted with a pseudo-IEC user to evaluate our proposed ICE algorithm. The evaluation result indicates that ICE algorithm can obtain a significant better performance than or the same performance as interactive DE. Some open topics on CE, ICE, fusion of these optimization techniques, algorithmic notation, and others are presented and discussed.

The Translational Science Benefits Model: A New Framework for Assessing the Health and Societal Benefits of Clinical and Translational Sciences

PubMed Central

Sarli, Cathy C.; Suiter, Amy M.; Carothers, Bobbi J.; Combs, Todd B.; Allen, Jae L.; Beers, Courtney E.; Evanoff, Bradley A.

2017-01-01

Abstract We report the development of the Translational Science Benefits Model (TSBM), a framework designed to support institutional assessment of clinical and translational research outcomes to measure clinical and community health impacts beyond bibliometric measures. The TSBM includes 30 specific and potentially measurable indicators that reflect benefits that accrue from clinical and translational science research such as products, system characteristics, or activities. Development of the TSBM was based on literature review, a modified Delphi method, and in‐house expert panel feedback. Three case studies illustrate the feasibility and face validity of the TSBM for identification of clinical and community health impacts that result from translational science activities. Future plans for the TSBM include further pilot testing and a resource library that will be freely available for evaluators, translational scientists, and academic institutions who wish to implement the TSBM framework in their own evaluation efforts. PMID:28887873

The search for relevant outcome measures for cost-utility analysis of systemic family interventions in adolescents with substance use disorder and delinquent behavior: a systematic literature review.

PubMed

Schawo, S; Bouwmans, C; van der Schee, E; Hendriks, V; Brouwer, W; Hakkaart, L

2017-09-19

Systemic family interventions have shown to be effective in adolescents with substance use disorder and delinquent behavior. The interventions target interactions between the adolescent and involved systems (i.e. youth, family, peers, neighbors, school, work, and society). Next to effectiveness considerations, economic aspects have gained attention. However, conventional generic quality of life measures used in health economic evaluations may not be able to capture the broad effects of systemic interventions. This study aims to identify existing outcome measures, which capture the broad effects of systemic family interventions, and allow use in a health economic framework. We based our systematic review on clinical studies in the field. Our goal was to identify effectiveness studies of psychosocial interventions for adolescents with substance use disorder and delinquent behavior and to distill the instruments used in these studies to measure effects. Searched databases were PubMed, Education Resource Information Center (ERIC), Cochrane and Psychnet (PsycBOOKSc, PsycCRITIQUES, print). Identified instruments were ranked according to the number of systems covered (comprehensiveness). In addition, their use for health economic analyses was evaluated according to suitability characteristics such as brevity, accessibility, psychometric properties, etc. One thousand three hundred seventy-eight articles were found and screened for eligibility. Eighty articles were selected, 8 instruments were identified covering 5 or more systems. The systematic review identified instruments from the clinical field suitable to evaluate systemic family interventions in a health economic framework. None of them had preference-weights available. Hence, a next step could be to attach preference-weights to one of the identified instruments to allow health economic evaluations of systemic family interventions.

The national response for preventing healthcare-associated infections: system capacity and sustainability for improvement.

PubMed

Mendel, Peter; Weinberg, Daniel A; Gall, Elizabeth M; Leuschner, Kristin J; Kahn, Katherine L

2014-02-01

Strengthening capacity across the healthcare system for improvement is critical to ensuring that past efforts and investments establish a foundation for sustaining progress in patient safety. The objective of this analysis was to identify key system capacity issues for sustainability from evaluation of the Action Plan to prevent healthcare-associated infections, a major national initiative launched by the US Department of Health and Human Services in 2009. The analysis involves the review and synthesis of results across the components of a 3-year evaluation of the Action Plan, as described in the evaluation framework and detailed in separate analyses elsewhere in this special issue. Data collection methods included interviews with government and private stakeholders, document and literature reviews, and observations of meetings and conferences at multiple time points. Key developments in healthcare-associated infection prevention system capacity were extracted on the basis of "major activities" identified through multiple methods and organized into the level of progress based on perspectives of multiple stakeholders. Activities within each level were then examined and compared according to our evaluation's framework of 4 system functions and 5 system properties. Key system capacity and sustainability issues for the Action Plan to be addressed centered on coordination and alignment (among participating agencies, with other federal initiatives, and across levels of healthcare), infrastructure for data and accountability (including more efficient technologies and unintended consequences), cultural embedding of prevention practices, and uncertainty and variability in resources. Sustainability depends on improvements across system functions and properties and how they reinforce each other. Change is more robust if different system elements support and incentivize behavior in similar directions.

The TEF modeling and analysis approach to advance thermionic space power technology

NASA Astrophysics Data System (ADS)

Marshall, Albert C.

1997-01-01

Thermionics space power systems have been proposed as advanced power sources for future space missions that require electrical power levels significantly above the capabilities of current space power systems. The Defense Special Weapons Agency's (DSWA) Thermionic Evaluation Facility (TEF) is carrying out both experimental and analytical research to advance thermionic space power technology to meet this expected need. A Modeling and Analysis (M&A) project has been created at the TEF to develop analysis tools, evaluate concepts, and guide research. M&A activities are closely linked to the TEF experimental program, providing experiment support and using experimental data to validate models. A planning exercise has been completed for the M&A project, and a strategy for implementation was developed. All M&A activities will build on a framework provided by a system performance model for a baseline Thermionic Fuel Element (TFE) concept. The system model is composed of sub-models for each of the system components and sub-systems. Additional thermionic component options and model improvements will continue to be incorporated in the basic system model during the course of the program. All tasks are organized into four focus areas: 1) system models, 2) thermionic research, 3) alternative concepts, and 4) documentation and integration. The M&A project will provide a solid framework for future thermionic system development.

Quality Evaluation of Zirconium Dioxide Frameworks Produced in Five Dental Laboratories from Different Countries.

PubMed

Schneebeli, Esther; Brägger, Urs; Scherrer, Susanne S; Keller, Andrea; Wittneben, Julia G; Hicklin, Stefan P

2017-07-01

The aim of this study was to assess and compare quality as well as economic aspects of CAD/CAM high strength ceramic three-unit FDP frameworks ordered from dental laboratories located in emerging countries and Switzerland. The master casts of six cases were sent to five dental laboratories located in Thailand (Bangkok), China (Peking and Shenzhen), Turkey (Izmir), and Switzerland (Bern). Each laboratory was using a different CAD/CAM system. The clinical fit of the frameworks was qualitatively assessed, and the thickness of the framework material, the connector height, the width, and the diameter were evaluated using a measuring sensor. The analysis of the internal fit of the frameworks was performed by means of a replica technique, whereas the inner and outer surfaces of the frameworks were evaluated for traces of postprocessing and damage to the intaglio surface with light and electronic microscopes. Groups (dental laboratories and cases) were compared for statistically significant differences using Mann-Whitney U-tests after Bonferroni correction. An acceptable clinical fit was found at 97.9% of the margins produced in laboratory E, 87.5% in B, 93.7% in C, 79.2% in A, and 62.5% in D. The mean framework thicknesses were not statistically significantly different for the premolar regions; however, for the molar area 4/8 of the evaluated sites were statistically significantly different. Circumference, surface, and width of the connectors produced in the different laboratories were statistically significantly different but not the height. There were great differences in the designs for the pontic and connector regions, and some of the frameworks would not be recommended for clinical use. Traces of heavy postprocessing were found in frameworks from some of the laboratories. The prices per framework ranged from US$177 to US$896. By ordering laboratory work in developing countries, a considerable price reduction was obtained compared to the price level in Switzerland. Despite the use of the standardized CAD/CAM chains of production in all laboratories, a large variability in the quality aspects, such as clinical marginal fit, connector and pontic design, as well as postprocessing traces was noted. Recommended sound handling of postprocessing was not applied in all laboratories. Dentists should be aware of the true and factitious advantages of CAD/CAM production chains and not lose control over the process. © 2015 by the American College of Prosthodontists.

A Hypermedia System To Aid in Preservice Teacher Education: Instructional Design and Evaluation.

ERIC Educational Resources Information Center

Lambdin, Diana V.; And Others

This research investigated how use of an interactive videodisk information system, the Strategic Teaching Framework (STF), helped preservice teachers expand their visions of teaching, learning, and assessment in mathematics, and helped develop their skills in translating that vision into action in the classroom. STF consisted of videos of…

Careers in Academe: The Academic Labour Market as an Eco-System

ERIC Educational Resources Information Center

Baruch, Yehuda

2013-01-01

Purpose: This paper aims to explore the contrast between stable and dynamic labour markets in academe in light of career theories that were originally developed for business environments. Design/methodology/approach: A conceptual design, offering the eco-system as a framework. Findings: It evaluates their relevance and applicability to dynamic and…

A Strategic Governance Review for Multi-Organizational Systems of Education, Training, and Professional Development.

ERIC Educational Resources Information Center

Daley, Glenn A.; Levy, Dina G.; Kaganoff, Tessa; Augustine, Catherine A.; Benjamin, Roger; Bikson, Tora K.; Gates, Susan M.; Moini, Joy S.

A study was conducted to develop tools for describing and evaluating external governance in the U.S. Department of Defense's (DoD's) multiorganizational system of education, training, and professional development. An exploratory conceptual framework and taxonomy for understanding external governance in the DoD context was developed, and the…

Patients' views on priority setting in neurosurgery: A qualitative study.

PubMed

Gunaratnam, Caroline; Bernstein, Mark

2016-01-01

Accountability for Reasonableness is an ethical framework which has been implemented in various health care systems to improve and evaluate the fairness of priority setting. This framework is grounded on four mandatory conditions: relevance, publicity, appeals, and enforcement. There have been few studies which have evaluated the patient stakeholders' acceptance of this framework; certainly no studies have been done on patients' views on the prioritization system for allocating patients for operating time in a system with pressure on the resource of inpatient beds. The aim of this study is to examine neurosurgical patients' views on the prioritization of patients for operating theater (OT) time on a daily basis at a tertiary and quaternary referral neurosurgery center. Semi-structured face-to-face interviews were conducted with thirty-seven patients, recruited from the neurosurgery clinic at Toronto Western Hospital. Family members and friends who accompanied the patient to their clinic visit were encouraged to contribute to the discussion. Interviews were audio recorded, transcribed verbatim, and subjected to thematic analysis using open and axial coding. Overall, patients are supportive of the concept of a priority-setting system based on fairness, but felt that a few changes would help to improve the fairness of the current system. These changes include lowering the level of priority given to volume-funded cases and providing scheduled surgeries that were previously canceled a higher level of prioritization. Good communication, early notification, and rescheduling canceled surgeries as soon as possible were important factors that directly reflected the patients' confidence level in their doctor, the hospital, and the health care system. This study is the first clinical qualitative study of patients' perspective on a prioritization system used for allocating neurosurgical patients for OT time on a daily basis in a socialized not-for-profit health care system with fixed resources.

Active Wireless System for Structural Health Monitoring Applications.

PubMed

Perera, Ricardo; Pérez, Alberto; García-Diéguez, Marta; Zapico-Valle, José Luis

2017-12-11

The use of wireless sensors in Structural Health Monitoring (SHM) has increased significantly in the last years. Piezoelectric-based lead zirconium titanate (PZT) sensors have been on the rise in SHM due to their superior sensing abilities. They are applicable in different technologies such as electromechanical impedance (EMI)-based SHM. This work develops a flexible wireless smart sensor (WSS) framework based on the EMI method using active sensors for full-scale and autonomous SHM. In contrast to passive sensors, the self-sensing properties of the PZTs allow interrogating with or exciting a structure when desired. The system integrates the necessary software and hardware within a service-oriented architecture approach able to provide in a modular way the services suitable to satisfy the key requirements of a WSS. The framework developed in this work has been validated on different experimental applications. Initially, the reliability of the EMI method when carried out with the proposed wireless sensor system is evaluated by comparison with the wireless counterpart. Afterwards, the performance of the system is evaluated in terms of software stability and reliability of functioning.

Comparison and Evaluation of End-User Interfaces for Online Public Access Catalogs.

ERIC Educational Resources Information Center

Zumer, Maja

End-user interfaces for the online public access catalogs (OPACs) of OhioLINK, a system linking major university and research libraries in Ohio, and its 16 member libraries, accessible through the Internet, are compared and evaluated from the user-oriented perspective. A common, systematic framework was used for the scientific observation of the…

Conceptual Framework for Therapeutic Training with Biofeedback in Virtual Reality: First Evaluation of a Relaxation Simulator

ERIC Educational Resources Information Center

Fominykh, Mikhail; Prasolova-Førland, Ekaterina; Stiles, Tore C.; Krogh, Anne Berit; Linde, Mattias

2018-01-01

This paper presents a concept for designing low-cost therapeutic training with biofeedback and virtual reality. We completed the first evaluation of a prototype--a mobile learning application for relaxation training, primarily for adolescents suffering from tension-type headaches. The system delivers visual experience on a head-mounted display. A…

Impacts of Performance Pay under the Teacher Incentive Fund: Study Design Report

ERIC Educational Resources Information Center

Glazerman, Steven; Chiang, Hanley; Wellington, Alison; Constantine, Jill; Player, Dan

2011-01-01

The body of research on the design, implementation, and effects of performance-based compensation systems has influenced the design and evaluation of the 2010 Teacher Incentive Fund (TIF) grants. In the sections presented here, the authors describe the key components of 2010 TIF grants and the conceptual framework for the evaluation. The remainder…

Evaluation methodology for query-based scene understanding systems

NASA Astrophysics Data System (ADS)

Huster, Todd P.; Ross, Timothy D.; Culbertson, Jared L.

2015-05-01

In this paper, we are proposing a method for the principled evaluation of scene understanding systems in a query-based framework. We can think of a query-based scene understanding system as a generalization of typical sensor exploitation systems where instead of performing a narrowly defined task (e.g., detect, track, classify, etc.), the system can perform general user-defined tasks specified in a query language. Examples of this type of system have been developed as part of DARPA's Mathematics of Sensing, Exploitation, and Execution (MSEE) program. There is a body of literature on the evaluation of typical sensor exploitation systems, but the open-ended nature of the query interface introduces new aspects to the evaluation problem that have not been widely considered before. In this paper, we state the evaluation problem and propose an approach to efficiently learn about the quality of the system under test. We consider the objective of the evaluation to be to build a performance model of the system under test, and we rely on the principles of Bayesian experiment design to help construct and select optimal queries for learning about the parameters of that model.

A modular Human Exposure Model (HEM) framework to ...

EPA Pesticide Factsheets

Life Cycle Impact Analysis (LCIA) has proven to be a valuable tool for systematically comparing processes and products, and has been proposed for use in Chemical Alternatives Analysis (CAA). The exposure assessment portion of the human health impact scores of LCIA has historically focused on far-field sources (environmentally mediated exposures) while research has shown that use related exposures, (near-field exposures) typically dominate population exposure. Characterizing the human health impacts of chemicals in consumer products over the life cycle of these products requires an evaluation of both near-field as well far-field sources. Assessing the impacts of the near-field exposures requires bridging the scientific and technical gaps that currently prevent the harmonious use of the best available methods and tools from the fields of LCIA and human health exposure and risk assessment. The U.S. EPA’s Chemical Safety and Sustainability LC-HEM project is developing the Human Exposure Model (HEM) to assess near-field exposures to chemicals that occur to various populations over the life cycle of a commercial product. The HEM will be a publically available, web-based, modular system which will allow for the evaluation of chemical/product impacts in a LCIA framework to support CAA. We present here an overview of the framework for the modular HEM system. The framework includes a data flow diagram of in-progress and future planned modules, the definition of each mod

Estimating yield gaps at the cropping system level.

PubMed

Guilpart, Nicolas; Grassini, Patricio; Sadras, Victor O; Timsina, Jagadish; Cassman, Kenneth G

2017-05-01

Yield gap analyses of individual crops have been used to estimate opportunities for increasing crop production at local to global scales, thus providing information crucial to food security. However, increases in crop production can also be achieved by improving cropping system yield through modification of spatial and temporal arrangement of individual crops. In this paper we define the cropping system yield potential as the output from the combination of crops that gives the highest energy yield per unit of land and time, and the cropping system yield gap as the difference between actual energy yield of an existing cropping system and the cropping system yield potential. Then, we provide a framework to identify alternative cropping systems which can be evaluated against the current ones. A proof-of-concept is provided with irrigated rice-maize systems at four locations in Bangladesh that represent a range of climatic conditions in that country. The proposed framework identified (i) realistic alternative cropping systems at each location, and (ii) two locations where expected improvements in crop production from changes in cropping intensity (number of crops per year) were 43% to 64% higher than from improving the management of individual crops within the current cropping systems. The proposed framework provides a tool to help assess food production capacity of new systems ( e.g. with increased cropping intensity) arising from climate change, and assess resource requirements (water and N) and associated environmental footprint per unit of land and production of these new systems. By expanding yield gap analysis from individual crops to the cropping system level and applying it to new systems, this framework could also be helpful to bridge the gap between yield gap analysis and cropping/farming system design.

A model to evaluate quality and effectiveness of disease management.

PubMed

Lemmens, K M M; Nieboer, A P; van Schayck, C P; Asin, J D; Huijsman, R

2008-12-01

Disease management has emerged as a new strategy to enhance quality of care for patients suffering from chronic conditions, and to control healthcare costs. So far, however, the effects of this strategy remain unclear. Although current models define the concept of disease management, they do not provide a systematic development or an explanatory theory of how disease management affects the outcomes of care. The objective of this paper is to present a framework for valid evaluation of disease-management initiatives. The evaluation model is built on two pillars of disease management: patient-related and professional-directed interventions. The effectiveness of these interventions is thought to be affected by the organisational design of the healthcare system. Disease management requires a multifaceted approach; hence disease-management programme evaluations should focus on the effects of multiple interventions, namely patient-related, professional-directed and organisational interventions. The framework has been built upon the conceptualisation of these disease-management interventions. Analysis of the underlying mechanisms of these interventions revealed that learning and behavioural theories support the core assumptions of disease management. The evaluation model can be used to identify the components of disease-management programmes and the mechanisms behind them, making valid comparison feasible. In addition, this model links the programme interventions to indicators that can be used to evaluate the disease-management programme. Consistent use of this framework will enable comparisons among disease-management programmes and outcomes in evaluation research.

Managing health and safety risks: Implications for tailoring health and safety management system practices.

PubMed

Willmer, D R; Haas, E J

2016-01-01

As national and international health and safety management system (HSMS) standards are voluntarily accepted or regulated into practice, organizations are making an effort to modify and integrate strategic elements of a connected management system into their daily risk management practices. In high-risk industries such as mining, that effort takes on added importance. The mining industry has long recognized the importance of a more integrated approach to recognizing and responding to site-specific risks, encouraging the adoption of a risk-based management framework. Recently, the U.S. National Mining Association led the development of an industry-specific HSMS built on the strategic frameworks of ANSI: Z10, OHSAS 18001, The American Chemistry Council's Responsible Care, and ILO-OSH 2001. All of these standards provide strategic guidance and focus on how to incorporate a plan-do-check-act cycle into the identification, management and evaluation of worksite risks. This paper details an exploratory study into whether practices associated with executing a risk-based management framework are visible through the actions of an organization's site-level management of health and safety risks. The results of this study show ways that site-level leaders manage day-to-day risk at their operations that can be characterized according to practices associated with a risk-based management framework. Having tangible operational examples of day-to-day risk management can serve as a starting point for evaluating field-level risk assessment efforts and their alignment to overall company efforts at effective risk mitigation through a HSMS or other processes.

Development of a spatially universal framework for classifying stream assemblages with application to conservation planning for Great Lakes lotic fish communities

USGS Publications Warehouse

McKenna, James E.; Schaeffer, Jeffrey S.; Stewart, Jana S.; Slattery, Michael T.

2015-01-01

Classifications are typically specific to particular issues or areas, leading to patchworks of subjectively defined spatial units. Stream conservation is hindered by the lack of a universal habitat classification system and would benefit from an independent hydrology-guided spatial framework of units encompassing all aquatic habitats at multiple spatial scales within large regions. We present a system that explicitly separates the spatial framework from any particular classification developed from the framework. The framework was constructed from landscape variables that are hydrologically and biologically relevant, covered all space within the study area, and was nested hierarchically and spatially related at scales ranging from the stream reach to the entire region; classifications may be developed from any subset of the 9 basins, 107 watersheds, 459 subwatersheds, or 10,000s of valley segments or stream reaches. To illustrate the advantages of this approach, we developed a fish-guided classification generated from a framework for the Great Lakes region that produced a mosaic of habitat units which, when aggregated, formed larger patches of more general conditions at progressively broader spatial scales. We identified greater than 1,200 distinct fish habitat types at the valley segment scale, most of which were rare. Comparisons of biodiversity and species assemblages are easily examined at any scale. This system can identify and quantify habitat types, evaluate habitat quality for conservation and/or restoration, and assist managers and policymakers with prioritization of protection and restoration efforts. Similar spatial frameworks and habitat classifications can be developed for any organism in any riverine ecosystem.

Multivendor Spectral-Domain Optical Coherence Tomography Dataset, Observer Annotation Performance Evaluation, and Standardized Evaluation Framework for Intraretinal Cystoid Fluid Segmentation.

PubMed

Wu, Jing; Philip, Ana-Maria; Podkowinski, Dominika; Gerendas, Bianca S; Langs, Georg; Simader, Christian; Waldstein, Sebastian M; Schmidt-Erfurth, Ursula M

2016-01-01

Development of image analysis and machine learning methods for segmentation of clinically significant pathology in retinal spectral-domain optical coherence tomography (SD-OCT), used in disease detection and prediction, is limited due to the availability of expertly annotated reference data. Retinal segmentation methods use datasets that either are not publicly available, come from only one device, or use different evaluation methodologies making them difficult to compare. Thus we present and evaluate a multiple expert annotated reference dataset for the problem of intraretinal cystoid fluid (IRF) segmentation, a key indicator in exudative macular disease. In addition, a standardized framework for segmentation accuracy evaluation, applicable to other pathological structures, is presented. Integral to this work is the dataset used which must be fit for purpose for IRF segmentation algorithm training and testing. We describe here a multivendor dataset comprised of 30 scans. Each OCT scan for system training has been annotated by multiple graders using a proprietary system. Evaluation of the intergrader annotations shows a good correlation, thus making the reproducibly annotated scans suitable for the training and validation of image processing and machine learning based segmentation methods. The dataset will be made publicly available in the form of a segmentation Grand Challenge.

Multivendor Spectral-Domain Optical Coherence Tomography Dataset, Observer Annotation Performance Evaluation, and Standardized Evaluation Framework for Intraretinal Cystoid Fluid Segmentation

PubMed Central

Wu, Jing; Philip, Ana-Maria; Podkowinski, Dominika; Gerendas, Bianca S.; Langs, Georg; Simader, Christian

2016-01-01

Development of image analysis and machine learning methods for segmentation of clinically significant pathology in retinal spectral-domain optical coherence tomography (SD-OCT), used in disease detection and prediction, is limited due to the availability of expertly annotated reference data. Retinal segmentation methods use datasets that either are not publicly available, come from only one device, or use different evaluation methodologies making them difficult to compare. Thus we present and evaluate a multiple expert annotated reference dataset for the problem of intraretinal cystoid fluid (IRF) segmentation, a key indicator in exudative macular disease. In addition, a standardized framework for segmentation accuracy evaluation, applicable to other pathological structures, is presented. Integral to this work is the dataset used which must be fit for purpose for IRF segmentation algorithm training and testing. We describe here a multivendor dataset comprised of 30 scans. Each OCT scan for system training has been annotated by multiple graders using a proprietary system. Evaluation of the intergrader annotations shows a good correlation, thus making the reproducibly annotated scans suitable for the training and validation of image processing and machine learning based segmentation methods. The dataset will be made publicly available in the form of a segmentation Grand Challenge. PMID:27579177

Application-level regression testing framework using Jenkins

DOE PAGES

Budiardja, Reuben; Bouvet, Timothy; Arnold, Galen

2017-09-26

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integratemore » with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.« less

Application-level regression testing framework using Jenkins

DOE Office of Scientific and Technical Information (OSTI.GOV)

Budiardja, Reuben; Bouvet, Timothy; Arnold, Galen

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integratemore » with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.« less

A Framework for Evaluating Induction into Teaching.

ERIC Educational Resources Information Center

Schlechty, Phillip C.

1985-01-01

Indicators and characteristics of effective induction systems are described using efforts of the Charlotte-Mecklenburg Schools' Career Development Program as an example. Fundamental changes needed in the way teacher education is conceptualized by school personnel are discussed. (DF)

Development of Chemical Process Design and Control for Sustainability

EPA Science Inventory

This contribution describes a novel process systems engineering framework that couples advanced control with sustainability evaluation and decision making for the optimization of process operations to minimize environmental impacts associated with products, materials, and energy....

Managing complexity in simulations of land surface and near-surface processes

DOE PAGES

Coon, Ethan T.; Moulton, J. David; Painter, Scott L.

2016-01-12

Increasing computing power and the growing role of simulation in Earth systems science have led to an increase in the number and complexity of processes in modern simulators. We present a multiphysics framework that specifies interfaces for coupled processes and automates weak and strong coupling strategies to manage this complexity. Process management is enabled by viewing the system of equations as a tree, where individual equations are associated with leaf nodes and coupling strategies with internal nodes. A dynamically generated dependency graph connects a variable to its dependencies, streamlining and automating model evaluation, easing model development, and ensuring models aremore » modular and flexible. Additionally, the dependency graph is used to ensure that data requirements are consistent between all processes in a given simulation. Here we discuss the design and implementation of these concepts within the Arcos framework, and demonstrate their use for verification testing and hypothesis evaluation in numerical experiments.« less

Evaluating probabilistic dengue risk forecasts from a prototype early warning system for Brazil.

PubMed

Lowe, Rachel; Coelho, Caio As; Barcellos, Christovam; Carvalho, Marilia Sá; Catão, Rafael De Castro; Coelho, Giovanini E; Ramalho, Walter Massa; Bailey, Trevor C; Stephenson, David B; Rodó, Xavier

2016-02-24

Recently, a prototype dengue early warning system was developed to produce probabilistic forecasts of dengue risk three months ahead of the 2014 World Cup in Brazil. Here, we evaluate the categorical dengue forecasts across all microregions in Brazil, using dengue cases reported in June 2014 to validate the model. We also compare the forecast model framework to a null model, based on seasonal averages of previously observed dengue incidence. When considering the ability of the two models to predict high dengue risk across Brazil, the forecast model produced more hits and fewer missed events than the null model, with a hit rate of 57% for the forecast model compared to 33% for the null model. This early warning model framework may be useful to public health services, not only ahead of mass gatherings, but also before the peak dengue season each year, to control potentially explosive dengue epidemics.

Examining the functionality of the DeLone and McLean information system success model as a framework for synthesis in nursing information and communication technology research.

PubMed

Booth, Richard G

2012-06-01

In this review, studies examining information and communication technology used by nurses in clinical practice were examined. Overall, a total of 39 studies were assessed spanning a time period from 1995 to 2008. The impacts of the various health information and communication technology evaluated by individual studies were synthesized using the DeLone and McLean's six-dimensional framework for evaluating information systems success (ie, System Quality, Information Quality, Service Quality, Use, User Satisfaction, and Net Benefits). Overall, the majority of researchers reported results related to the overall Net Benefits (positive, negative, and indifferent) of the health information and communication technology used by nurses. Attitudes and user satisfaction with technology were also commonly measured attributes. The current iteration of DeLone and McLean model is effective at synthesizing basic elements of health information and communication technology use by nurses. Regardless, the current model lacks the sociotechnical sensitivity to capture deeper nurse-technology relationalities. Limitations and recommendations are provided for researchers considering using the DeLone and McLean model for evaluating health information and communication technology used by nurses.

Combining Cryptography with EEG Biometrics

PubMed Central

Kazanavičius, Egidijus; Woźniak, Marcin

2018-01-01

Cryptographic frameworks depend on key sharing for ensuring security of data. While the keys in cryptographic frameworks must be correctly reproducible and not unequivocally connected to the identity of a user, in biometric frameworks this is different. Joining cryptography techniques with biometrics can solve these issues. We present a biometric authentication method based on the discrete logarithm problem and Bose-Chaudhuri-Hocquenghem (BCH) codes, perform its security analysis, and demonstrate its security characteristics. We evaluate a biometric cryptosystem using our own dataset of electroencephalography (EEG) data collected from 42 subjects. The experimental results show that the described biometric user authentication system is effective, achieving an Equal Error Rate (ERR) of 0.024.

Combining Cryptography with EEG Biometrics.

PubMed

Damaševičius, Robertas; Maskeliūnas, Rytis; Kazanavičius, Egidijus; Woźniak, Marcin

2018-01-01

Cryptographic frameworks depend on key sharing for ensuring security of data. While the keys in cryptographic frameworks must be correctly reproducible and not unequivocally connected to the identity of a user, in biometric frameworks this is different. Joining cryptography techniques with biometrics can solve these issues. We present a biometric authentication method based on the discrete logarithm problem and Bose-Chaudhuri-Hocquenghem (BCH) codes, perform its security analysis, and demonstrate its security characteristics. We evaluate a biometric cryptosystem using our own dataset of electroencephalography (EEG) data collected from 42 subjects. The experimental results show that the described biometric user authentication system is effective, achieving an Equal Error Rate (ERR) of 0.024.

A case study of evaluating informatics impact on diffusion of scientific knowledge.

PubMed

Katz, Susan B

2008-11-06

This case study poster uses a newly developed framework to evaluate an informatics effort in its public health context. The electronic clearance system being evaluated provides the potential for increasing the speed and quality of scientific diffusion of knowledge, and thus translation of research into practice. A graphical logic model and tabular results of the evaluation are presented. Public health history suggests potential benefits of more timely and coordinated diffusion of scientific information.

Providing a Science Base for the Evaluation of Tobacco Products

PubMed Central

Berman, Micah L.; Connolly, Greg; Cummings, K. Michael; Djordjevic, Mirjana V.; Hatsukami, Dorothy K.; Henningfield, Jack E.; Myers, Matthew; O'Connor, Richard J.; Parascandola, Mark; Rees, Vaughan; Rice, Jerry M.

2015-01-01

Objective Evidence-based tobacco regulation requires a comprehensive scientific framework to guide the evaluation of new tobacco products and health-related claims made by product manufacturers. Methods The Tobacco Product Assessment Consortium (TobPRAC) employed an iterative process involving consortia investigators, consultants, a workshop of independent scientists and public health experts, and written reviews in order to develop a conceptual framework for evaluating tobacco products. Results The consortium developed a four-phased framework for the scientific evaluation of tobacco products. The four phases addressed by the framework are: (1) pre-market evaluation, (2) pre-claims evaluation, (3) post-market activities, and (4) monitoring and re-evaluation. For each phase, the framework proposes the use of validated testing procedures that will evaluate potential harms at both the individual and population level. Conclusions While the validation of methods for evaluating tobacco products is an ongoing and necessary process, the proposed framework need not wait for fully validated methods to be used in guiding tobacco product regulation today. PMID:26665160

Disaster Metrics: A Comprehensive Framework for Disaster Evaluation Typologies.

PubMed

Wong, Diana F; Spencer, Caroline; Boyd, Lee; Burkle, Frederick M; Archer, Frank

2017-10-01

Introduction The frequency of disasters is increasing around the world with more people being at risk. There is a moral imperative to improve the way in which disaster evaluations are undertaken and reported with the aim of reducing preventable mortality and morbidity in future events. Disasters are complex events and undertaking disaster evaluations is a specialized area of study at an international level. Hypothesis/Problem While some frameworks have been developed to support consistent disaster research and evaluation, they lack validation, consistent terminology, and standards for reporting across the different phases of a disaster. There is yet to be an agreed, comprehensive framework to structure disaster evaluation typologies. The aim of this paper is to outline an evolving comprehensive framework for disaster evaluation typologies. It is anticipated that this new framework will facilitate an agreement on identifying, structuring, and relating the various evaluations found in the disaster setting with a view to better understand the process, outcomes, and impacts of the effectiveness and efficiency of interventions. Research was undertaken in two phases: (1) a scoping literature review (peer-reviewed and "grey literature") was undertaken to identify current evaluation frameworks and typologies used in the disaster setting; and (2) a structure was developed that included the range of typologies identified in Phase One and suggests possible relationships in the disaster setting. No core, unifying framework to structure disaster evaluation and research was identified in the literature. The authors propose a "Comprehensive Framework for Disaster Evaluation Typologies" that identifies, structures, and suggests relationships for the various typologies detected. The proposed Comprehensive Framework for Disaster Evaluation Typologies outlines the different typologies of disaster evaluations that were identified in this study and brings them together into a single framework. This unique, unifying framework has relevance at an international level and is expected to benefit the disaster, humanitarian, and development sectors. The next step is to undertake a validation process that will include international leaders with experience in evaluation, in general, and disasters specifically. This work promotes an environment for constructive dialogue on evaluations in the disaster setting to strengthen the evidence base for interventions across the disaster spectrum. It remains a work in progress. Wong DF , Spencer C , Boyd L , Burkle FM Jr. , Archer F . Disaster metrics: a comprehensive framework for disaster evaluation typologies. Prehosp Disaster Med. 2017;32(5):501-514.

Clinical time series prediction: towards a hierarchical dynamical system framework

PubMed Central

Liu, Zitao; Hauskrecht, Milos

2014-01-01

Objective Developing machine learning and data mining algorithms for building temporal models of clinical time series is important for understanding of the patient condition, the dynamics of a disease, effect of various patient management interventions and clinical decision making. In this work, we propose and develop a novel hierarchical framework for modeling clinical time series data of varied length and with irregularly sampled observations. Materials and methods Our hierarchical dynamical system framework for modeling clinical time series combines advantages of the two temporal modeling approaches: the linear dynamical system and the Gaussian process. We model the irregularly sampled clinical time series by using multiple Gaussian process sequences in the lower level of our hierarchical framework and capture the transitions between Gaussian processes by utilizing the linear dynamical system. The experiments are conducted on the complete blood count (CBC) panel data of 1000 post-surgical cardiac patients during their hospitalization. Our framework is evaluated and compared to multiple baseline approaches in terms of the mean absolute prediction error and the absolute percentage error. Results We tested our framework by first learning the time series model from data for the patient in the training set, and then applying the model in order to predict future time series values on the patients in the test set. We show that our model outperforms multiple existing models in terms of its predictive accuracy. Our method achieved a 3.13% average prediction accuracy improvement on ten CBC lab time series when it was compared against the best performing baseline. A 5.25% average accuracy improvement was observed when only short-term predictions were considered. Conclusion A new hierarchical dynamical system framework that lets us model irregularly sampled time series data is a promising new direction for modeling clinical time series and for improving their predictive performance. PMID:25534671

A comparative analysis of biomedical research ethics regulation systems in Europe and Latin America with regard to the protection of human subjects.

PubMed

Lamas, Eugenia; Ferrer, Marcela; Molina, Alberto; Salinas, Rodrigo; Hevia, Adriana; Bota, Alexandre; Feinholz, Dafna; Fuchs, Michael; Schramm, Roland; Tealdi, Juan-Carlos; Zorrilla, Sergio

2010-12-01

The European project European and Latin American Systems of Ethics Regulation of Biomedical Research Project (EULABOR) has carried out the first comparative analysis of ethics regulation systems for biomedical research in seven countries in Europe and Latin America, evaluating their roles in the protection of human subjects. We developed a conceptual and methodological framework defining 'ethics regulation system for biomedical research' as a set of actors, institutions, codes and laws involved in overseeing the ethics of biomedical research on humans. This framework allowed us to develop comprehensive national reports by conducting semi-structured interviews to key informants. These reports were summarised and analysed in a comparative analysis. The study showed that the regulatory framework for clinical research in these countries differ in scope. It showed that despite the different political contexts, actors involved and motivations for creating the regulation, in most of the studied countries it was the government who took the lead in setting up the system. The study also showed that Europe and Latin America are similar regarding national bodies and research ethics committees, but the Brazilian system has strong and noteworthy specificities.

A computerized support system to cooperative training in emergency scenarios management and its application to an oil port domain

DOE Office of Scientific and Technical Information (OSTI.GOV)

Balducelli, C.; Bologna, S.; Di Costanzo, G.

1995-12-31

The paper describes part of the results achieved in the framework of the MUSTER project (Multi-Users System for Training and Evaluating Environmental Emergency Response). The aim of this project is to define the detailed specifications of a computer based system supporting collaborative training for emergency management. A system prototype has been implemented to support the refinement and improvement of the system specifications.

Clinical Decision Support-based Quality Measurement (CDS-QM) Framework: Prototype Implementation, Evaluation, and Future Directions

PubMed Central

Kukhareva, Polina V; Kawamoto, Kensaku; Shields, David E; Barfuss, Darryl T; Halley, Anne M; Tippetts, Tyler J; Warner, Phillip B; Bray, Bruce E; Staes, Catherine J

2014-01-01

Electronic quality measurement (QM) and clinical decision support (CDS) are closely related but are typically implemented independently, resulting in significant duplication of effort. While it seems intuitive that technical approaches could be re-used across these two related use cases, such reuse is seldom reported in the literature, especially for standards-based approaches. Therefore, we evaluated the feasibility of using a standards-based CDS framework aligned with anticipated EHR certification criteria to implement electronic QM. The CDS-QM framework was used to automate a complex national quality measure (SCIP-VTE-2) at an academic healthcare system which had previously relied on time-consuming manual chart abstractions. Compared with 305 manually-reviewed reference cases, the recall of automated measurement was 100%. The precision was 96.3% (CI:92.6%-98.5%) for ascertaining the denominator and 96.2% (CI:92.3%-98.4%) for the numerator. We therefore validated that a standards-based CDS-QM framework can successfully enable automated QM, and we identified benefits and challenges with this approach. PMID:25954389

Frameworks for change in healthcare organisations: a formative evaluation of the NHS Change Model.

PubMed

Martin, Graham P; Sutton, Elizabeth; Willars, Janet; Dixon-Woods, Mary

2013-08-01

Organisational change in complex healthcare systems is a multifaceted process. The English National Health Service recently introduced a 'Change Model' that seeks to offer an evidence-based framework for guiding change. We report findings from a formative evaluation of the NHS Change Model and make recommendations for those developing the Model and its users. The evaluation involved 28 interviews with managers and clinicians making use of the Change Model in relation to a variety of projects. Interviews were fully transcribed and were analysed using an approach based on the Framework method. Participants saw the Change Model as valuable and practically useful. Fidelity to core principles of the Model was variable: participants often altered the Model, especially when using it to orchestrate the work of others. In challenging organisational contexts, the Change Model was sometimes used to delegitimise opposition rather than identify shared purpose among different interest groups. Those guiding change may benefit from frameworks, guidance and toolkits to structure and inform their planning and activities. Participants' experiences suggested the Change Model has much potential. Further work on its design and on supporting materials may optimise the approach, but its utility rests in particular on organisational cultures that support faithful application. © The Author(s) 2013 Reprints and permissions:]br]sagepub.co.uk/journalsPermissions.nav.

A Framework for the Study of Complex mHealth Interventions in Diverse Cultural Settings

PubMed Central

Yeates, Karen; Perkins, Nancy; Boesch, Lisa; Hua-Stewart, Diane; Liu, Peter; Sleeth, Jessica; Tobe, Sheldon W

2017-01-01

Background To facilitate decision-making capacity between options of care under real-life service conditions, clinical trials must be pragmatic to evaluate mobile health (mHealth) interventions under the variable conditions of health care settings with a wide range of participants. The mHealth interventions require changes in the behavior of patients and providers, creating considerable complexity and ambiguity related to causal chains. Process evaluations of the implementation are necessary to shed light on the range of unanticipated effects an intervention may have, what the active ingredients in everyday practice are, how they exert their effect, and how these may vary among recipients or between sites. Objective Building on the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth) statement and participatory evaluation theory, we present a framework for the process evaluations for mHealth interventions in multiple cultural settings. We also describe the application of this evaluation framework to the implementation of DREAM-GLOBAL (Diagnosing hypertension—Engaging Action and Management in Getting Lower BP in Indigenous and LMIC [low- and middle-income countries]), a pragmatic randomized controlled trial (RCT), and mHealth intervention designed to improve hypertension management in low-resource environments. We describe the evaluation questions and the data collection processes developed by us. Methods Our literature review revealed that there is a significant knowledge gap related to the development of a process evaluation framework for mHealth interventions. We used community-based participatory research (CBPR) methods and formative research data to develop a process evaluation framework nested within a pragmatic RCT. Results Four human organizational levels of participants impacted by the mHealth intervention were identified that included patients, providers, community and organizations actors, and health systems and settings. These four levels represent evaluation domains and became the core focus of the evaluation. In addition, primary implementation themes to explore in each of the domains were identified as follows: (1) the major active components of the intervention, (2) technology of the intervention, (3) cultural congruence, (4) task shifting, and (5) unintended consequences. Using the four organizational domains and their interaction with primary implementation themes, we developed detailed evaluation research questions and identified the data or information sources to best answer our questions. Conclusions Using DREAM-GLOBAL to illustrate our approach, we succeeded in developing an uncomplicated process evaluation framework for mHealth interventions that provide key information to stakeholders, which can optimize implementation of a pragmatic trial as well as inform scale up. The human organizational level domains used to focus the primary implementation themes in the DREAM-GLOBAL process evaluation framework are sufficiently supported in our research, and the literature and can serve as a valuable tool for other mHealth process evaluations. Trial Registration ClinicalTrials.gov NCT02111226; https://clinicaltrials.gov/ct2/show/NCT02111226 (Archived by WebCite at http://www.webcitation.org/6oxfHXege) PMID:28428165

A new approach to the concept of "relevance" in information retrieval (IR).

PubMed

Kagolovsky, Y; Möhr, J R

2001-01-01

The concept of "relevance" is the fundamental concept of information science in general and information retrieval, in particular. Although "relevance" is extensively used in evaluation of information retrieval, there are considerable problems associated with reaching an agreement on its definition, meaning, evaluation, and application in information retrieval. There are a number of different views on "relevance" and its use for evaluation. Based on a review of the literature the main problems associated with the concept of "relevance" in information retrieval are identified. The authors argue that the proposal for the solution of the problems can be based on the conceptual IR framework built using a systems analytic approach to IR. Using this framework different kinds of "relevance" relationships in the IR process are identified, and a methodology for evaluation of "relevance" based on methods of semantics capturing and comparison is proposed.

Using the Analytic Hierarchy Process (AHP) to understand the most important factors to design and evaluate a telehealth system for Parkinson's disease.

PubMed

Cancela, Jorge; Fico, Giuseppe; Arredondo Waldmeyer, Maria T

2015-01-01

The assessment of a new health technology is a multidisciplinary and multidimensional process, which requires a complex analysis and the convergence of different stakeholders into a common decision. This task is even more delicate when the assessment is carried out in early stage of development processes, when the maturity of the technology prevents conducting a large scale trials to evaluate the cost effectiveness through classic health economics methods. This lack of information may limit the future development and deployment in the clinical practice. This work aims to 1) identify the most relevant user needs of a new medical technology for managing and monitoring Parkinson's Disease (PD) patients and to 2) use these user needs for a preliminary assessment of a specific system called PERFORM, as a case study. Analytic Hierarchy Process (AHP) was used to design a hierarchy of 17 needs, grouped into 5 categories. A total of 16 experts, 6 of them with a clinical background and the remaining 10 with a technical background, were asked to rank these needs and categories. On/Off fluctuations detection, Increase wearability acceptance, and Increase self-management support have been identified as the most relevant user needs. No significant differences were found between the clinician and technical groups. These results have been used to evaluate the PERFORM system and to identify future areas of improvement. First of all, the AHP contributed to the elaboration of a unified hierarchy, integrating the needs of a variety of stakeholders, promoting the discussion and the agreement into a common framework of evaluation. Moreover, the AHP effectively supported the user need elicitation as well as the assignment of different weights and priorities to each need and, consequently, it helped to define a framework for the assessment of telehealth systems for PD management and monitoring. This framework can be used to support the decision-making process for the adoption of new technologies in PD.