Cryptographic framework for document-objects resulting from multiparty collaborative transactions.

PubMed

Goh, A

2000-01-01

Multiparty transactional frameworks--i.e. Electronic Data Interchange (EDI) or Health Level (HL) 7--often result in composite documents which can be accurately modelled using hyperlinked document-objects. The structural complexity arising from multiauthor involvement and transaction-specific sequencing would be poorly handled by conventional digital signature schemes based on a single evaluation of a one-way hash function and asymmetric cryptography. In this paper we outline the generation of structure-specific authentication hash-trees for the the authentication of transactional document-objects, followed by asymmetric signature generation on the hash-tree value. Server-side multi-client signature verification would probably constitute the single most compute-intensive task, hence the motivation for our usage of the Rabin signature protocol which results in significantly reduced verification workloads compared to the more commonly applied Rivest-Shamir-Adleman (RSA) protocol. Data privacy is handled via symmetric encryption of message traffic using session-specific keys obtained through key-negotiation mechanisms based on discrete-logarithm cryptography. Individual client-to-server channels can be secured using a double key-pair variation of Diffie-Hellman (DH) key negotiation, usage of which also enables bidirectional node authentication. The reciprocal server-to-client multicast channel is secured through Burmester-Desmedt (BD) key-negotiation which enjoys significant advantages over the usual multiparty extensions to the DH protocol. The implementation of hash-tree signatures and bi/multidirectional key negotiation results in a comprehensive cryptographic framework for multiparty document-objects satisfying both authentication and data privacy requirements.

A Conceptual Framework for the Evaluation of Emergency Risk Communications

PubMed Central

Lin, Leesa; Gamhewage, Gaya M.

2017-01-01

Objectives. To articulate a conceptual framework in support of evaluation activities in emergency risk communications (ERC). Methods. The framework proposed is based on a systematic review of the scientific literature (2001–2016) combined with data derived from a series of semistructured interviews with experts and practitioners in ERC, and it is designed to support local, national, and international public health organizations in implementing evaluation studies in ERC. Results. We identified a list of ERC outcomes from the full-text review of 152 articles and categorized these into 3 groups, depending upon the level at which the outcome was measured: (1) information environment, (2) population, and (3) public health system. We analyzed interviewees’ data from 18 interviews to identify practices and processes related to the effectiveness of ERC and included these as key structural components and processes in the developed evaluation framework. Conclusions. Researchers and public health practitioners interested in the evaluation of ERC can use the conceptual framework described in this article to guide the development of evaluation studies and methods for assessing communication outcomes related to public health emergencies. PMID:28892436

Evaluation readiness: improved evaluation planning using a data inventory framework.

PubMed

Cohen, A B; Hall, K C; Cohodes, D R

1985-01-01

Factors intrinsic to many programs, such as ambiguously stated objectives, inadequately defined performance measures, and incomplete or unreliable databases, often conspire to limit the evaluability of these programs. Current evaluation planning approaches are somewhat constrained in their ability to overcome these obstacles and to achieve full preparedness for evaluation. In this paper, the concept of evaluation readiness is introduced as a complement to other evaluation planning approaches, most notably that of evaluability assessment. The basic products of evaluation readiness--the formal program definition and the data inventory framework--are described, along with a guide for assuring more timely and appropriate evaluation response capability to support the decision making needs of program managers. The utility of evaluation readiness for program planning, as well as for effective management, is also discussed.

A framework for the evaluation of patient information leaflets

PubMed Central

Garner, Mark; Ning, Zhenye; Francis, Jill

2011-01-01

Abstract Background  The provision of patient information leaflets (PILs) is an important part of health care. PILs require evaluation, but the frameworks that are used for evaluation are largely under‐informed by theory. Most evaluation to date has been based on indices of readability, yet several writers argue that readability is not enough. We propose a framework for evaluating PILs that reflect the central role of the patient perspective in communication and use methods for evaluation based on simple linguistic principles. The proposed framework  The framework has three elements that give rise to three approaches to evaluation. Each element is a necessary but not sufficient condition for effective communication. Readability (focussing on text) may be assessed using existing well‐established procedures. Comprehensibility (focussing on reader and text) may be assessed using multiple‐choice questions based on the lexical and semantic features of the text. Communicative effectiveness (focussing on reader) explores the relationship between the emotional, cognitive and behavioural responses of the reader and the objectives of the PIL. Suggested methods for assessment are described, based on our preliminary empirical investigations. Conclusions  The tripartite model of communicative effectiveness is a patient‐centred framework for evaluating PILs. It may assist the field in moving beyond readability to broader indicators of the quality and appropriateness of printed information provided to patients. PMID:21332620

Development of an evaluation framework for African-European hospital patient safety partnerships.

PubMed

Rutter, Paul; Syed, Shamsuzzoha B; Storr, Julie; Hightower, Joyce D; Bagheri-Nejad, Sepideh; Kelley, Edward; Pittet, Didier

2014-04-01

Patient safety is recognised as a significant healthcare problem worldwide, and healthcare-associated infections are an important aspect. African Partnerships for Patient Safety is a WHO programme that pairs hospitals in Africa with hospitals in Europe with the objective to work together to improve patient safety. To describe the development of an evaluation framework for hospital-to-hospital partnerships participating in the programme. The framework was structured around the programme's three core objectives: facilitate strong interhospital partnerships, improve in-hospital patient safety and spread best practices nationally. Africa-based clinicians, their European partners and experts in patient safety were closely involved in developing the evaluation framework in an iterative process. The process defined six domains of partnership strength, each with measurable subdomains. We developed a questionnaire to measure these subdomains. Participants selected six indicators of hospital patient safety improvement from a short-list of 22 based on their relevance, sensitivity to intervention and measurement feasibility. Participants proposed 20 measures of spread, which were refined into a two-part conceptual framework, and a data capture tool created. Taking a highly participatory approach that closely involved its end users, we developed an evaluation framework and tools to measure partnership strength, patient safety improvements and the spread of best practice.

Reframing Evaluation: Defining an Indigenous Evaluation Framework

ERIC Educational Resources Information Center

LaFrance, Joan; Nichols, Richard

2008-01-01

The American Indian Higher Education Consortium (AIHEC), comprising 34 American Indian tribally controlled colleges and universities, has undertaken a comprehensive effort to develop an "Indigenous Framework for Evaluation" that synthesizes Indigenous ways of knowing and Western evaluation practice. To ground the framework, AIHEC engaged…

Defining ecological and economical hydropoweroperations: a framework for managing dam releasesto meet multiple conflicting objectives

USGS Publications Warehouse

Irwin, Elise R.

2014-01-01

Hydroelectric dams are a flexible source of power, provide flood control, and contribute to the economic growth of local communities through real-estate and recreation. Yet the impoundment of rivers can alter and fragment miles of critical riverine habitat needed for other competing needs such as downstream consumptive water use, fish and wildlife population viability, or other forms of recreation. Multiple conflicting interests can compromise progressive management especially with recognized uncertainties related to whether management actions will fulfill the objectives of policy makers, resource managers and/or facility owners. Decision analytic tools were used in a stakeholder-driven process to develop and implement a template for evaluation and prediction of the effects of water resource management of multiple-use systems under the context provided by R.L. Harris Dam on the Tallapoosa River, Alabama, USA. The approach provided a transparent and structured framework for decision-making and incorporated both existing and new data to meet multiple management objectives. Success of the template has been evaluated by the stakeholder governing body in an adaptive resource management framework since 2005 and is ongoing. Consequences of management of discharge at the dam were evaluated annually relative to stakeholder satisfaction to allow for adjustment of both management scenarios and objectives. This template can be applied to attempt to resolve conflict inherent in many dam-regulated systems where management decisions impact diverse values of stakeholders.

Steps toward improving ethical evaluation in health technology assessment: a proposed framework.

PubMed

Assasi, Nazila; Tarride, Jean-Eric; O'Reilly, Daria; Schwartz, Lisa

2016-06-06

While evaluation of ethical aspects in health technology assessment (HTA) has gained much attention during the past years, the integration of ethics in HTA practice still presents many challenges. In response to the increasing demand for expansion of health technology assessment (HTA) methodology to include ethical issues more systematically, this article reports on a multi-stage study that aimed at construction of a framework for improving the integration of ethics in HTA. The framework was developed through the following phases: 1) a systematic review and content analysis of guidance documents for ethics in HTA; 2) identification of factors influencing the integration of ethical considerations in HTA; 3) preparation of an action-oriented framework based on the key elements of the existing guidance documents and identified barriers to and facilitators of their implementation; and 4) expert consultation and revision of the framework. The proposed framework consists of three main components: an algorithmic flowchart, which exhibits the different steps of an ethical inquiry throughout the HTA process, including: defining the objectives and scope of the evaluation, stakeholder analysis, assessing organizational capacity, framing ethical evaluation questions, ethical analysis, deliberation, and knowledge translation; a stepwise guide, which focuses on the task objectives and potential questions that are required to be addressed at each step; and a list of some commonly recommended or used tools to help facilitate the evaluation process. The proposed framework can be used to support and promote good practice in integration of ethics into HTA. However, further validation of the framework through case studies and expert consultation is required to establish its utility for HTA practice.

Etomica: an object-oriented framework for molecular simulation.

PubMed

Schultz, Andrew J; Kofke, David A

2015-03-30

We describe the design of an object-oriented library of software components that are suitable for constructing simulations of systems of interacting particles. The emphasis of the discussion is on the general design of the components and how they interact, and less on details of the programming interface or its implementation. Example code is provided as an aid to understanding object-oriented programming structures and to demonstrate how the framework is applied. © 2015 Wiley Periodicals, Inc.

Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram.

PubMed

Chang, Hyejung

2015-10-01

Technological advances using telemedicine and telehealth are growing in healthcare fields, but the evaluation framework for them is inconsistent and limited. This paper suggests a comprehensive evaluation framework for telemedicine system implementation and will support related stakeholders' decision-making by promoting general understanding, and resolving arguments and controversies. This study focused on developing a comprehensive evaluation framework by summarizing themes across the range of evaluation techniques and organized foundational evaluation frameworks generally applicable through studies and cases of diverse telemedicine. Evaluation factors related to aspects of information technology; the evaluation of satisfaction of service providers and consumers, cost, quality, and information security are organized using the fishbone diagram. It was not easy to develop a monitoring and evaluation framework for telemedicine since evaluation frameworks for telemedicine are very complex with many potential inputs, activities, outputs, outcomes, and stakeholders. A conceptual framework was developed that incorporates the key dimensions that need to be considered in the evaluation of telehealth implementation for a formal structured approach to the evaluation of a service. The suggested framework consists of six major dimensions and the subsequent branches for each dimension. To implement telemedicine and telehealth services, stakeholders should make decisions based on sufficient evidence in quality and safety measured by the comprehensive evaluation framework. Further work would be valuable in applying more comprehensive evaluations to verify and improve the comprehensive framework across a variety of contexts with more factors and participant group dimensions.

An object-oriented framework for medical image registration, fusion, and visualization.

PubMed

Zhu, Yang-Ming; Cochoff, Steven M

2006-06-01

An object-oriented framework for image registration, fusion, and visualization was developed based on the classic model-view-controller paradigm. The framework employs many design patterns to facilitate legacy code reuse, manage software complexity, and enhance the maintainability and portability of the framework. Three sample applications built a-top of this framework are illustrated to show the effectiveness of this framework: the first one is for volume image grouping and re-sampling, the second one is for 2D registration and fusion, and the last one is for visualization of single images as well as registered volume images.

SERVAL: a new framework for the evaluation of animal health surveillance.

PubMed

Drewe, J A; Hoinville, L J; Cook, A J C; Floyd, T; Gunn, G; Stärk, K D C

2015-02-01

Animal health surveillance programmes may change in response to altering requirements or perceived weaknesses but are seldom subjected to any formal evaluation to ensure that they provide valuable information in an efficient manner. The literature on the evaluation of animal health surveillance systems is sparse, and those that are published may be unstructured and therefore incomplete. To address this gap, we have developed SERVAL, a SuRveillance EVALuation framework, which is novel and aims to be generic and therefore suitable for the evaluation of any animal health surveillance system. The inclusion of socio-economic criteria ensures that economic evaluation is an integral part of this framework. SERVAL was developed with input from a technical workshop of international experts followed by a consultation process involving providers and users of surveillance and evaluation data. It has been applied to a range of case studies encompassing different surveillance and evaluation objectives. Here, we describe the development, structure and application of the SERVAL framework. We discuss users' experiences in applying SERVAL to evaluate animal health surveillance systems in Great Britain. © 2013 Crown Copyright. This article is published with the permission of the Controller of HMSO and the Queen's Printer for Scotland.

Evaluation Framework for Non-Emergency Medical Transportation Services for Patients with End-Stage Renal Disease

DOT National Transportation Integrated Search

2014-12-01

The objective of this project is to design a framework that could be used to evaluate the effectiveness and efficiency of non-emergency transportation services (NEMT) for better livability. In addition to the development of the framework, this projec...

Design of a Model Execution Framework: Repetitive Object-Oriented Simulation Environment (ROSE)

NASA Technical Reports Server (NTRS)

Gray, Justin S.; Briggs, Jeffery L.

2008-01-01

The ROSE framework was designed to facilitate complex system analyses. It completely divorces the model execution process from the model itself. By doing so ROSE frees the modeler to develop a library of standard modeling processes such as Design of Experiments, optimizers, parameter studies, and sensitivity studies which can then be applied to any of their available models. The ROSE framework accomplishes this by means of a well defined API and object structure. Both the API and object structure are presented here with enough detail to implement ROSE in any object-oriented language or modeling tool.

A Patient-Centered Framework for Evaluating Digital Maturity of Health Services: A Systematic Review

PubMed Central

Callahan, Ryan; Darzi, Ara; Mayer, Erik

2016-01-01

Background Digital maturity is the extent to which digital technologies are used as enablers to deliver a high-quality health service. Extensive literature exists about how to assess the components of digital maturity, but it has not been used to design a comprehensive framework for evaluation. Consequently, the measurement systems that do exist are limited to evaluating digital programs within one service or care setting, meaning that digital maturity evaluation is not accounting for the needs of patients across their care pathways. Objective The objective of our study was to identify the best methods and metrics for evaluating digital maturity and to create a novel, evidence-based tool for evaluating digital maturity across patient care pathways. Methods We systematically reviewed the literature to find the best methods and metrics for evaluating digital maturity. We searched the PubMed database for all papers relevant to digital maturity evaluation. Papers were selected if they provided insight into how to appraise digital systems within the health service and if they indicated the factors that constitute or facilitate digital maturity. Papers were analyzed to identify methodology for evaluating digital maturity and indicators of digitally mature systems. We then used the resulting information about methodology to design an evaluation framework. Following that, the indicators of digital maturity were extracted and grouped into increasing levels of maturity and operationalized as metrics within the evaluation framework. Results We identified 28 papers as relevant to evaluating digital maturity, from which we derived 5 themes. The first theme concerned general evaluation methodology for constructing the framework (7 papers). The following 4 themes were the increasing levels of digital maturity: resources and ability (6 papers), usage (7 papers), interoperability (3 papers), and impact (5 papers). The framework includes metrics for each of these levels at each

A Sustainable Evaluation Framework and Its Application

ERIC Educational Resources Information Center

Powell, Robert B.; Stern, Marc J.; Ardoin, Nicole

2006-01-01

This article presents a framework for developing internally sustainable evaluation systems for environmental education organizations, although the framework can be applied to other types of organizations. The authors developed a sustainable evaluation framework (SEF) with the intent of creating an evaluation system that could be self-administered…

A comprehensive health service evaluation and monitoring framework.

PubMed

Reeve, Carole; Humphreys, John; Wakerman, John

2015-12-01

To develop a framework for evaluating and monitoring a primary health care service, integrating hospital and community services. A targeted literature review of primary health service evaluation frameworks was performed to inform the development of the framework specifically for remote communities. Key principles underlying primary health care evaluation were determined and sentinel indicators developed to operationalise the evaluation framework. This framework was then validated with key stakeholders. The framework includes Donabedian's three seminal domains of structure, process and outcomes to determine health service performance. These in turn are dependent on sustainability, quality of patient care and the determinants of health to provide a comprehensive health service evaluation framework. The principles underpinning primary health service evaluation were pertinent to health services in remote contexts. Sentinel indicators were developed to fit the demographic characteristics and health needs of the population. Consultation with key stakeholders confirmed that the evaluation framework was applicable. Data collected routinely by health services can be used to operationalise the proposed health service evaluation framework. Use of an evaluation framework which links policy and health service performance to health outcomes will assist health services to improve performance as part of a continuous quality improvement cycle. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Evaluation of the causal framework used for setting national ambient air quality standards.

PubMed

Goodman, Julie E; Prueitt, Robyn L; Sax, Sonja N; Bailey, Lisa A; Rhomberg, Lorenz R

2013-11-01

Abstract A scientifically sound assessment of the potential hazards associated with a substance requires a systematic, objective and transparent evaluation of the weight of evidence (WoE) for causality of health effects. We critically evaluated the current WoE framework for causal determination used in the United States Environmental Protection Agency's (EPA's) assessments of the scientific data on air pollutants for the National Ambient Air Quality Standards (NAAQS) review process, including its methods for literature searches; study selection, evaluation and integration; and causal judgments. The causal framework used in recent NAAQS evaluations has many valuable features, but it could be more explicit in some cases, and some features are missing that should be included in every WoE evaluation. Because of this, it has not always been applied consistently in evaluations of causality, leading to conclusions that are not always supported by the overall WoE, as we demonstrate using EPA's ozone Integrated Science Assessment as a case study. We propose additions to the NAAQS causal framework based on best practices gleaned from a previously conducted survey of available WoE frameworks. A revision of the NAAQS causal framework so that it more closely aligns with these best practices and the full and consistent application of the framework will improve future assessments of the potential health effects of criteria air pollutants by making the assessments more thorough, transparent, and scientifically sound.

A framework provided an outline toward the proper evaluation of potential screening strategies.

PubMed

Adriaensen, Wim J; Matheï, Cathy; Buntinx, Frank J; Arbyn, Marc

2013-06-01

Screening tests are often introduced into clinical practice without proper evaluation, despite the increasing awareness that screening is a double-edged sword that can lead to either net benefits or harms. Our objective was to develop a comprehensive framework for the evaluation of new screening strategies. Elaborating on the existing concepts proposed by experts, a stepwise framework is proposed to evaluate whether a potential screening test can be introduced as a screening strategy into clinical practice. The principle of screening strategy evaluation is illustrated for cervical cancer, which is a template for screening because of the existence of an easily detectable and treatable precursor lesion. The evaluation procedure consists of six consecutive steps. In steps 1-4, the technical accuracy, place of the test in the screening pathway, diagnostic accuracy, and longitudinal sensitivity and specificity of the screening test are assessed. In steps 5 and 6, the impact of the screening strategy on the patient and population levels, respectively, is evaluated. The framework incorporates a harm and benefit trade-off and cost-effectiveness analysis. Our framework provides an outline toward the proper evaluation of potential screening strategies before considering implementation. Copyright © 2013 Elsevier Inc. All rights reserved.

Implicit kernel sparse shape representation: a sparse-neighbors-based objection segmentation framework.

PubMed

Yao, Jincao; Yu, Huimin; Hu, Roland

2017-01-01

This paper introduces a new implicit-kernel-sparse-shape-representation-based object segmentation framework. Given an input object whose shape is similar to some of the elements in the training set, the proposed model can automatically find a cluster of implicit kernel sparse neighbors to approximately represent the input shape and guide the segmentation. A distance-constrained probabilistic definition together with a dualization energy term is developed to connect high-level shape representation and low-level image information. We theoretically prove that our model not only derives from two projected convex sets but is also equivalent to a sparse-reconstruction-error-based representation in the Hilbert space. Finally, a "wake-sleep"-based segmentation framework is applied to drive the evolutionary curve to recover the original shape of the object. We test our model on two public datasets. Numerical experiments on both synthetic images and real applications show the superior capabilities of the proposed framework.

A framework for telehealth program evaluation.

PubMed

Nepal, Surya; Li, Jane; Jang-Jaccard, Julian; Alem, Leila

2014-04-01

Evaluating telehealth programs is a challenging task, yet it is the most sensible first step when embarking on a telehealth study. How can we frame and report on telehealth studies? What are the health services elements to select based on the application needs? What are the appropriate terms to use to refer to such elements? Various frameworks have been proposed in the literature to answer these questions, and each framework is defined by a set of properties covering different aspects of telehealth systems. The most common properties include application, technology, and functionality. With the proliferation of telehealth, it is important not only to understand these properties, but also to define new properties to account for a wider range of context of use and evaluation outcomes. This article presents a comprehensive framework for delivery design, implementation, and evaluation of telehealth services. We first survey existing frameworks proposed in the literature and then present our proposed comprehensive multidimensional framework for telehealth. Six key dimensions of the proposed framework include health domains, health services, delivery technologies, communication infrastructure, environment setting, and socioeconomic analysis. We define a set of example properties for each dimension. We then demonstrate how we have used our framework to evaluate telehealth programs in rural and remote Australia. A few major international studies have been also mapped to demonstrate the feasibility of the framework. The key characteristics of the framework are as follows: (a) loosely coupled and hence easy to use, (b) provides a basis for describing a wide range of telehealth programs, and (c) extensible to future developments and needs.

A framework to support preceptors' evaluation and development of new nurses' clinical judgment.

PubMed

Nielsen, Ann; Lasater, Kathie; Stock, Mary

2016-07-01

In today's complex, fast-paced world of hospital nursing, new graduate nurses do not have well-developed clinical judgment skills. Nurse preceptors are charged with bridging the gap between new graduates' learning in school and their autonomous practice as RNs. In one large, urban medical center in the U.S., a clinical judgment model and rubric were used as a framework for a new evaluation and orientation process. Preceptors of new graduate nurses who had used the former and new processes described their experiences using the framework. The findings indicated that having a structured framework provided objective ways to evaluate and help develop new graduate nurses' clinical judgment. It is hypothesized that academic clinical supervisors may find such a framework useful to prepare students for transition to practice. Copyright © 2016 Elsevier Ltd. All rights reserved.

A Framework for Including Family Health Spillovers in Economic Evaluation

PubMed Central

Al-Janabi, Hareth; van Exel, Job; Brouwer, Werner; Coast, Joanna

2016-01-01

Health care interventions may affect the health of patients’ family networks. It has been suggested that these “health spillovers” should be included in economic evaluation, but there is not a systematic method for doing this. In this article, we develop a framework for including health spillovers in economic evaluation. We focus on extra-welfarist economic evaluations where the objective is to maximize health benefits from a health care budget (the “health care perspective”). Our framework involves adapting the conventional cost-effectiveness decision rule to include 2 multiplier effects to internalize the spillover effects. These multiplier effects express the ratio of total health effects (for patients and their family networks) to patient health effects. One multiplier effect is specified for health benefit generated from providing a new intervention, one for health benefit displaced by funding this intervention. We show that using multiplier effects to internalize health spillovers could change the optimal funding decisions and generate additional health benefits to society. PMID:26377370

Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol.

PubMed

Kasturi, Rangachar; Goldgof, Dmitry; Soundararajan, Padmanabhan; Manohar, Vasant; Garofolo, John; Bowers, Rachel; Boonstra, Matthew; Korzhova, Valentina; Zhang, Jing

2009-02-01

Common benchmark data sets, standardized performance metrics, and baseline algorithms have demonstrated considerable impact on research and development in a variety of application domains. These resources provide both consumers and developers of technology with a common framework to objectively compare the performance of different algorithms and algorithmic improvements. In this paper, we present such a framework for evaluating object detection and tracking in video: specifically for face, text, and vehicle objects. This framework includes the source video data, ground-truth annotations (along with guidelines for annotation), performance metrics, evaluation protocols, and tools including scoring software and baseline algorithms. For each detection and tracking task and supported domain, we developed a 50-clip training set and a 50-clip test set. Each data clip is approximately 2.5 minutes long and has been completely spatially/temporally annotated at the I-frame level. Each task/domain, therefore, has an associated annotated corpus of approximately 450,000 frames. The scope of such annotation is unprecedented and was designed to begin to support the necessary quantities of data for robust machine learning approaches, as well as a statistically significant comparison of the performance of algorithms. The goal of this work was to systematically address the challenges of object detection and tracking through a common evaluation framework that permits a meaningful objective comparison of techniques, provides the research community with sufficient data for the exploration of automatic modeling techniques, encourages the incorporation of objective evaluation into the development process, and contributes useful lasting resources of a scale and magnitude that will prove to be extremely useful to the computer vision research community for years to come.

A scoping review about conference objectives and evaluative practices: how do we get more out of them?

PubMed

Neves, Justin; Lavis, John N; Ranson, M Kent

2012-08-02

Large multi-day conferences have often been criticized as ineffective ways to improve social outcomes and to influence policy or practice. Unfortunately, many conference evaluations have also been inadequate in determining the impact of a conference on its associated social sector, with little evidence gathered or analyzed to substantiate or refute these criticisms. The aim of this scoping review is to investigate and report stakeholders' objectives for planning or participating in large multi-day conferences and how these objectives are being evaluated. We conducted a scoping review supplemented by a small number of key informant interviews. Eight bibliographic databases were systematically searched to identify papers describing conference objectives and/or evaluations. We developed a conference evaluation framework based on theoretical models and empirical findings, which structured the descriptive synthesis of the data. We identified 3,073 potential papers for review, of which 44 were included in this study. Our evaluation framework connects five key elements in planning a conference and its evaluation (number in brackets refers to number of themes identified): conference objectives (8), purpose of evaluation (7), evaluation methods (5), indicators of success (9) and theories/models (8). Further analysis of indicators of success identified three categories of indicators with differing scopes (i.e. immediate, prospective or follow-up) as well as empirical links between the purpose of evaluations and these indicators. Conference objectives and evaluations were largely correlated with the type of conference (i.e. academic, political/governmental or business) but diverse overall. While much can be done to improve the quality and usefulness of conference evaluations, there are innovative assessments that are currently being utilized by some conferences and warrant further investigation. This review provides conference evaluators and organizers a simple resource to

A scoping review about conference objectives and evaluative practices: how do we get more out of them?

PubMed Central

2012-01-01

Large multi-day conferences have often been criticized as ineffective ways to improve social outcomes and to influence policy or practice. Unfortunately, many conference evaluations have also been inadequate in determining the impact of a conference on its associated social sector, with little evidence gathered or analyzed to substantiate or refute these criticisms. The aim of this scoping review is to investigate and report stakeholders’ objectives for planning or participating in large multi-day conferences and how these objectives are being evaluated. We conducted a scoping review supplemented by a small number of key informant interviews. Eight bibliographic databases were systematically searched to identify papers describing conference objectives and/or evaluations. We developed a conference evaluation framework based on theoretical models and empirical findings, which structured the descriptive synthesis of the data. We identified 3,073 potential papers for review, of which 44 were included in this study. Our evaluation framework connects five key elements in planning a conference and its evaluation (number in brackets refers to number of themes identified): conference objectives (8), purpose of evaluation (7), evaluation methods (5), indicators of success (9) and theories/models (8). Further analysis of indicators of success identified three categories of indicators with differing scopes (i.e. immediate, prospective or follow-up) as well as empirical links between the purpose of evaluations and these indicators. Conference objectives and evaluations were largely correlated with the type of conference (i.e. academic, political/governmental or business) but diverse overall. While much can be done to improve the quality and usefulness of conference evaluations, there are innovative assessments that are currently being utilized by some conferences and warrant further investigation. This review provides conference evaluators and organizers a simple resource to

Measuring research impact in Australia's medical research institutes: a scoping literature review of the objectives for and an assessment of the capabilities of research impact assessment frameworks.

PubMed

Deeming, Simon; Searles, Andrew; Reeves, Penny; Nilsson, Michael

2017-03-21

Realising the economic potential of research institutions, including medical research institutes, represents a policy imperative for many Organisation for Economic Co-operation and Development nations. The assessment of research impact has consequently drawn increasing attention. Research impact assessment frameworks (RIAFs) provide a structure to assess research translation, but minimal research has examined whether alternative RIAFs realise the intended policy outcomes. This paper examines the objectives presented for RIAFs in light of economic imperatives to justify ongoing support for health and medical research investment, leverage productivity via commercialisation and outcome-efficiency gains in health systems, and ensure that translation and impact considerations are embedded into the research process. This paper sought to list the stated objectives for RIAFs, to identify existing frameworks and to evaluate whether the identified frameworks possessed the capabilities necessary to address the specified objectives. A scoping review of the literature to identify objectives specified for RIAFs, inform upon descriptive criteria for each objective and identify existing RIAFs. Criteria were derived for each objective. The capability for the existing RIAFs to realise the alternative objectives was evaluated based upon these criteria. The collated objectives for RIAFs included accountability (top-down), transparency/accountability (bottom-up), advocacy, steering, value for money, management/learning and feedback/allocation, prospective orientation, and speed of translation. Of the 25 RIAFs identified, most satisfied objectives such as accountability and advocacy, which are largely sufficient for the first economic imperative to justify research investment. The frameworks primarily designed to optimise the speed of translation or enable the prospective orientation of research possessed qualities most likely to optimise the productive outcomes from research. However

A visualization framework for design and evaluation

NASA Astrophysics Data System (ADS)

Blundell, Benjamin J.; Ng, Gary; Pettifer, Steve

2006-01-01

The creation of compelling visualisation paradigms is a craft often dominated by intuition and issues of aesthetics, with relatively few models to support good design. The majority of problem cases are approached by simply applying a previously evaluated visualisation technique. A large body of work exists covering the individual aspects of visualisation design such as the human cognition aspects visualisation methods for specific problem areas, psychology studies and so forth, yet most frameworks regarding visualisation are applied after-the-fact as an evaluation measure. We present an extensible framework for visualisation aimed at structuring the design process, increasing decision traceability and delineating the notions of function, aesthetics and usability. The framework can be used to derive a set of requirements for good visualisation design and evaluating existing visualisations, presenting possible improvements. Our framework achieves this by being both broad and general, built on top of existing works, with hooks for extensions and customizations. This paper shows how existing theories of information visualisation fit into the scheme, presents our experience in the application of this framework on several designs, and offers our evaluation of the framework and the designs studied.

A quality assurance framework for the fully automated and objective evaluation of image quality in cone-beam computed tomography.

PubMed

Steiding, Christian; Kolditz, Daniel; Kalender, Willi A

2014-03-01

Thousands of cone-beam computed tomography (CBCT) scanners for vascular, maxillofacial, neurological, and body imaging are in clinical use today, but there is no consensus on uniform acceptance and constancy testing for image quality (IQ) and dose yet. The authors developed a quality assurance (QA) framework for fully automated and time-efficient performance evaluation of these systems. In addition, the dependence of objective Fourier-based IQ metrics on direction and position in 3D volumes was investigated for CBCT. The authors designed a dedicated QA phantom 10 cm in length consisting of five compartments, each with a diameter of 10 cm, and an optional extension ring 16 cm in diameter. A homogeneous section of water-equivalent material allows measuring CT value accuracy, image noise and uniformity, and multidimensional global and local noise power spectra (NPS). For the quantitative determination of 3D high-contrast spatial resolution, the modulation transfer function (MTF) of centrally and peripherally positioned aluminum spheres was computed from edge profiles. Additional in-plane and axial resolution patterns were used to assess resolution qualitatively. The characterization of low-contrast detectability as well as CT value linearity and artifact behavior was tested by utilizing sections with soft-tissue-equivalent and metallic inserts. For an automated QA procedure, a phantom detection algorithm was implemented. All tests used in the dedicated QA program were initially verified in simulation studies and experimentally confirmed on a clinical dental CBCT system. The automated IQ evaluation of volume data sets of the dental CBCT system was achieved with the proposed phantom requiring only one scan for the determination of all desired parameters. Typically, less than 5 min were needed for phantom set-up, scanning, and data analysis. Quantitative evaluation of system performance over time by comparison to previous examinations was also verified. The maximum

Building clinical networks: a developmental evaluation framework.

PubMed

Carswell, Peter; Manning, Benjamin; Long, Janet; Braithwaite, Jeffrey

2014-05-01

Clinical networks have been designed as a cross-organisational mechanism to plan and deliver health services. With recent concerns about the effectiveness of these structures, it is timely to consider an evidence-informed approach for how they can be developed and evaluated. To document an evaluation framework for clinical networks by drawing on the network evaluation literature and a 5-year study of clinical networks. We searched literature in three domains: network evaluation, factors that aid or inhibit network development, and on robust methods to measure network characteristics. This material was used to build a framework required for effective developmental evaluation. The framework's architecture identifies three stages of clinical network development; partner selection, network design and network management. Within each stage is evidence about factors that act as facilitators and barriers to network growth. These factors can be used to measure progress via appropriate methods and tools. The framework can provide for network growth and support informed decisions about progress. For the first time in one place a framework incorporating rigorous methods and tools can identify factors known to affect the development of clinical networks. The target user group is internal stakeholders who need to conduct developmental evaluation to inform key decisions along their network's developmental pathway.

Evaluation of Moving Object Detection Based on Various Input Noise Using Fixed Camera

NASA Astrophysics Data System (ADS)

Kiaee, N.; Hashemizadeh, E.; Zarrinpanjeh, N.

2017-09-01

Detecting and tracking objects in video has been as a research area of interest in the field of image processing and computer vision. This paper evaluates the performance of a novel method for object detection algorithm in video sequences. This process helps us to know the advantage of this method which is being used. The proposed framework compares the correct and wrong detection percentage of this algorithm. This method was evaluated with the collected data in the field of urban transport which include car and pedestrian in fixed camera situation. The results show that the accuracy of the algorithm will decreases because of image resolution reduction.

A framework for evaluating the appropriateness of clinical decision support alerts and responses

PubMed Central

Waitman, Lemuel R; Lewis, Julia B; Wright, Julie A; Choma, David P; Miller, Randolph A; Peterson, Josh F

2011-01-01

Objective Alerting systems, a type of clinical decision support, are increasingly prevalent in healthcare, yet few studies have concurrently measured the appropriateness of alerts with provider responses to alerts. Recent reports of suboptimal alert system design and implementation highlight the need for better evaluation to inform future designs. The authors present a comprehensive framework for evaluating the clinical appropriateness of synchronous, interruptive medication safety alerts. Methods Through literature review and iterative testing, metrics were developed that describe successes, justifiable overrides, provider non-adherence, and unintended adverse consequences of clinical decision support alerts. The framework was validated by applying it to a medication alerting system for patients with acute kidney injury (AKI). Results Through expert review, the framework assesses each alert episode for appropriateness of the alert display and the necessity and urgency of a clinical response. Primary outcomes of the framework include the false positive alert rate, alert override rate, provider non-adherence rate, and rate of provider response appropriateness. Application of the framework to evaluate an existing AKI medication alerting system provided a more complete understanding of the process outcomes measured in the AKI medication alerting system. The authors confirmed that previous alerts and provider responses were most often appropriate. Conclusion The new evaluation model offers a potentially effective method for assessing the clinical appropriateness of synchronous interruptive medication alerts prior to evaluating patient outcomes in a comparative trial. More work can determine the generalizability of the framework for use in other settings and other alert types. PMID:21849334

Developing a monitoring and evaluation framework to integrate and formalize the informal waste and recycling sector: the case of the Philippine National Framework Plan.

PubMed

Serrona, Kevin Roy B; Yu, Jeongsoo; Aguinaldo, Emelita; Florece, Leonardo M

2014-09-01

The Philippines has been making inroads in solid waste management with the enactment and implementation of the Republic Act 9003 or the Ecological Waste Management Act of 2000. Said legislation has had tremendous influence in terms of how the national and local government units confront the challenges of waste management in urban and rural areas using the reduce, reuse, recycle and recovery framework or 4Rs. One of the sectors needing assistance is the informal waste sector whose aspiration is legal recognition of their rank and integration of their waste recovery activities in mainstream waste management. To realize this, the Philippine National Solid Waste Management Commission initiated the formulation of the National Framework Plan for the Informal Waste Sector, which stipulates approaches, strategies and methodologies to concretely involve the said sector in different spheres of local waste management, such as collection, recycling and disposal. What needs to be fleshed out is the monitoring and evaluation component in order to gauge qualitative and quantitative achievements vis-a-vis the Framework Plan. In the process of providing an enabling environment for the informal waste sector, progress has to be monitored and verified qualitatively and quantitatively and measured against activities, outputs, objectives and goals. Using the Framework Plan as the reference, this article developed monitoring and evaluation indicators using the logical framework approach in project management. The primary objective is to institutionalize monitoring and evaluation, not just in informal waste sector plans, but in any waste management initiatives to ensure that envisaged goals are achieved. © The Author(s) 2014.

A framework for assessing Health Economic Evaluation (HEE) quality appraisal instruments.

PubMed

Langer, Astrid

2012-08-16

Health economic evaluations support the health care decision-making process by providing information on costs and consequences of health interventions. The quality of such studies is assessed by health economic evaluation (HEE) quality appraisal instruments. At present, there is no instrument for measuring and improving the quality of such HEE quality appraisal instruments. Therefore, the objectives of this study are to establish a framework for assessing the quality of HEE quality appraisal instruments to support and improve their quality, and to apply this framework to those HEE quality appraisal instruments which have been subject to more scrutiny than others, in order to test the framework and to demonstrate the shortcomings of existing HEE quality appraisal instruments. To develop the quality assessment framework for HEE quality appraisal instruments, the experiences of using appraisal tools for clinical guidelines are used. Based on a deductive iterative process, clinical guideline appraisal instruments identified through literature search are reviewed, consolidated, and adapted to produce the final quality assessment framework for HEE quality appraisal instruments. The final quality assessment framework for HEE quality appraisal instruments consists of 36 items organized within 7 dimensions, each of which captures a specific domain of quality. Applying the quality assessment framework to four existing HEE quality appraisal instruments, it is found that these four quality appraisal instruments are of variable quality. The framework described in this study should be regarded as a starting point for appraising the quality of HEE quality appraisal instruments. This framework can be used by HEE quality appraisal instrument producers to support and improve the quality and acceptance of existing and future HEE quality appraisal instruments. By applying this framework, users of HEE quality appraisal instruments can become aware of methodological deficiencies

Multi-object segmentation framework using deformable models for medical imaging analysis.

PubMed

Namías, Rafael; D'Amato, Juan Pablo; Del Fresno, Mariana; Vénere, Marcelo; Pirró, Nicola; Bellemare, Marc-Emmanuel

2016-08-01

Segmenting structures of interest in medical images is an important step in different tasks such as visualization, quantitative analysis, simulation, and image-guided surgery, among several other clinical applications. Numerous segmentation methods have been developed in the past three decades for extraction of anatomical or functional structures on medical imaging. Deformable models, which include the active contour models or snakes, are among the most popular methods for image segmentation combining several desirable features such as inherent connectivity and smoothness. Even though different approaches have been proposed and significant work has been dedicated to the improvement of such algorithms, there are still challenging research directions as the simultaneous extraction of multiple objects and the integration of individual techniques. This paper presents a novel open-source framework called deformable model array (DMA) for the segmentation of multiple and complex structures of interest in different imaging modalities. While most active contour algorithms can extract one region at a time, DMA allows integrating several deformable models to deal with multiple segmentation scenarios. Moreover, it is possible to consider any existing explicit deformable model formulation and even to incorporate new active contour methods, allowing to select a suitable combination in different conditions. The framework also introduces a control module that coordinates the cooperative evolution of the snakes and is able to solve interaction issues toward the segmentation goal. Thus, DMA can implement complex object and multi-object segmentations in both 2D and 3D using the contextual information derived from the model interaction. These are important features for several medical image analysis tasks in which different but related objects need to be simultaneously extracted. Experimental results on both computed tomography and magnetic resonance imaging show that the proposed

Secure and Efficient Regression Analysis Using a Hybrid Cryptographic Framework: Development and Evaluation

PubMed Central

Jiang, Xiaoqian; Aziz, Md Momin Al; Wang, Shuang; Mohammed, Noman

2018-01-01

Background Machine learning is an effective data-driven tool that is being widely used to extract valuable patterns and insights from data. Specifically, predictive machine learning models are very important in health care for clinical data analysis. The machine learning algorithms that generate predictive models often require pooling data from different sources to discover statistical patterns or correlations among different attributes of the input data. The primary challenge is to fulfill one major objective: preserving the privacy of individuals while discovering knowledge from data. Objective Our objective was to develop a hybrid cryptographic framework for performing regression analysis over distributed data in a secure and efficient way. Methods Existing secure computation schemes are not suitable for processing the large-scale data that are used in cutting-edge machine learning applications. We designed, developed, and evaluated a hybrid cryptographic framework, which can securely perform regression analysis, a fundamental machine learning algorithm using somewhat homomorphic encryption and a newly introduced secure hardware component of Intel Software Guard Extensions (Intel SGX) to ensure both privacy and efficiency at the same time. Results Experimental results demonstrate that our proposed method provides a better trade-off in terms of security and efficiency than solely secure hardware-based methods. Besides, there is no approximation error. Computed model parameters are exactly similar to plaintext results. Conclusions To the best of our knowledge, this kind of secure computation model using a hybrid cryptographic framework, which leverages both somewhat homomorphic encryption and Intel SGX, is not proposed or evaluated to this date. Our proposed framework ensures data security and computational efficiency at the same time. PMID:29506966

An Evaluation Framework and Comparative Analysis of the Widely Used First Programming Languages

PubMed Central

Farooq, Muhammad Shoaib; Khan, Sher Afzal; Ahmad, Farooq; Islam, Saeed; Abid, Adnan

2014-01-01

Computer programming is the core of computer science curriculum. Several programming languages have been used to teach the first course in computer programming, and such languages are referred to as first programming language (FPL). The pool of programming languages has been evolving with the development of new languages, and from this pool different languages have been used as FPL at different times. Though the selection of an appropriate FPL is very important, yet it has been a controversial issue in the presence of many choices. Many efforts have been made for designing a good FPL, however, there is no ample way to evaluate and compare the existing languages so as to find the most suitable FPL. In this article, we have proposed a framework to evaluate the existing imperative, and object oriented languages for their suitability as an appropriate FPL. Furthermore, based on the proposed framework we have devised a customizable scoring function to compute a quantitative suitability score for a language, which reflects its conformance to the proposed framework. Lastly, we have also evaluated the conformance of the widely used FPLs to the proposed framework, and have also computed their suitability scores. PMID:24586449

An evaluation framework and comparative analysis of the widely used first programming languages.

PubMed

Farooq, Muhammad Shoaib; Khan, Sher Afzal; Ahmad, Farooq; Islam, Saeed; Abid, Adnan

2014-01-01

Computer programming is the core of computer science curriculum. Several programming languages have been used to teach the first course in computer programming, and such languages are referred to as first programming language (FPL). The pool of programming languages has been evolving with the development of new languages, and from this pool different languages have been used as FPL at different times. Though the selection of an appropriate FPL is very important, yet it has been a controversial issue in the presence of many choices. Many efforts have been made for designing a good FPL, however, there is no ample way to evaluate and compare the existing languages so as to find the most suitable FPL. In this article, we have proposed a framework to evaluate the existing imperative, and object oriented languages for their suitability as an appropriate FPL. Furthermore, based on the proposed framework we have devised a customizable scoring function to compute a quantitative suitability score for a language, which reflects its conformance to the proposed framework. Lastly, we have also evaluated the conformance of the widely used FPLs to the proposed framework, and have also computed their suitability scores.

Health information systems evaluation frameworks: A systematic review.

PubMed

Eslami Andargoli, Amirhossein; Scheepers, Helana; Rajendran, Diana; Sohal, Amrik

2017-01-01

Evaluation of health information systems (HISs) is complicated because of the complex nature of the health care domain. Various studies have proposed different frameworks to reduce the complexity in the assessment of these systems. The aim of these frameworks is to provide a set of guidelines for the evaluation of the adequacy of health care information systems. This paper aims to analyse studies on the evaluation of HISs by applying a content, context and process (CCP) framework to address the 'who', 'what', 'how', 'when', and 'why' of the evaluation processes used. This will allow for a better understanding of the relative strengths and weaknesses of various HISs evaluation frameworks, and will pave the way for developing a more complete framework for HISs. A systematic literature review on HIS evaluation studies was undertaken to identify the currently available HIS evaluation frameworks. Five academic databases were selected to conduct this systematic literature review. Most of the studies only address some, but not all, of the five main questions, i.e. the who, what, how, when, why, and that there was a lack of consensus in the way these questions were addressed. The critical role of context was also largely neglected in these studies. Evaluation of HISs is complex. The health care domain is highly context sensitive and in order to have a complete assessment of HISs, consideration of contextual factors is necessary. Specifically, to have the right set of criteria to measure the 'what', the answer to the 'who' of the evaluation is necessary. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Disaster Metrics: A Comprehensive Framework for Disaster Evaluation Typologies.

PubMed

Wong, Diana F; Spencer, Caroline; Boyd, Lee; Burkle, Frederick M; Archer, Frank

2017-10-01

Introduction The frequency of disasters is increasing around the world with more people being at risk. There is a moral imperative to improve the way in which disaster evaluations are undertaken and reported with the aim of reducing preventable mortality and morbidity in future events. Disasters are complex events and undertaking disaster evaluations is a specialized area of study at an international level. Hypothesis/Problem While some frameworks have been developed to support consistent disaster research and evaluation, they lack validation, consistent terminology, and standards for reporting across the different phases of a disaster. There is yet to be an agreed, comprehensive framework to structure disaster evaluation typologies. The aim of this paper is to outline an evolving comprehensive framework for disaster evaluation typologies. It is anticipated that this new framework will facilitate an agreement on identifying, structuring, and relating the various evaluations found in the disaster setting with a view to better understand the process, outcomes, and impacts of the effectiveness and efficiency of interventions. Research was undertaken in two phases: (1) a scoping literature review (peer-reviewed and "grey literature") was undertaken to identify current evaluation frameworks and typologies used in the disaster setting; and (2) a structure was developed that included the range of typologies identified in Phase One and suggests possible relationships in the disaster setting. No core, unifying framework to structure disaster evaluation and research was identified in the literature. The authors propose a "Comprehensive Framework for Disaster Evaluation Typologies" that identifies, structures, and suggests relationships for the various typologies detected. The proposed Comprehensive Framework for Disaster Evaluation Typologies outlines the different typologies of disaster evaluations that were identified in this study and brings them together into a single

A quality assurance framework for the fully automated and objective evaluation of image quality in cone-beam computed tomography

DOE Office of Scientific and Technical Information (OSTI.GOV)

Steiding, Christian; Kolditz, Daniel; Kalender, Willi A., E-mail: willi.kalender@imp.uni-erlangen.de

Purpose: Thousands of cone-beam computed tomography (CBCT) scanners for vascular, maxillofacial, neurological, and body imaging are in clinical use today, but there is no consensus on uniform acceptance and constancy testing for image quality (IQ) and dose yet. The authors developed a quality assurance (QA) framework for fully automated and time-efficient performance evaluation of these systems. In addition, the dependence of objective Fourier-based IQ metrics on direction and position in 3D volumes was investigated for CBCT. Methods: The authors designed a dedicated QA phantom 10 cm in length consisting of five compartments, each with a diameter of 10 cm, andmore » an optional extension ring 16 cm in diameter. A homogeneous section of water-equivalent material allows measuring CT value accuracy, image noise and uniformity, and multidimensional global and local noise power spectra (NPS). For the quantitative determination of 3D high-contrast spatial resolution, the modulation transfer function (MTF) of centrally and peripherally positioned aluminum spheres was computed from edge profiles. Additional in-plane and axial resolution patterns were used to assess resolution qualitatively. The characterization of low-contrast detectability as well as CT value linearity and artifact behavior was tested by utilizing sections with soft-tissue-equivalent and metallic inserts. For an automated QA procedure, a phantom detection algorithm was implemented. All tests used in the dedicated QA program were initially verified in simulation studies and experimentally confirmed on a clinical dental CBCT system. Results: The automated IQ evaluation of volume data sets of the dental CBCT system was achieved with the proposed phantom requiring only one scan for the determination of all desired parameters. Typically, less than 5 min were needed for phantom set-up, scanning, and data analysis. Quantitative evaluation of system performance over time by comparison to previous examinations

Evaluation of a performance appraisal framework for radiation therapists in planning and simulation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Becker, Jillian, E-mail: jillian.becker@health.qld.gov.au; Bridge, Pete; Brown, Elizabeth

2015-06-15

Constantly evolving technology and techniques within radiation therapy require practitioners to maintain a continuous approach to professional development and training. Systems of performance appraisal and adoption of regular feedback mechanisms are vital to support this development yet frequently lack structure and rely on informal peer support. A Radiation Therapy Performance Appraisal Framework (RT-PAF) for radiation therapists in planning and simulation was developed to define expectations of practice and promote a supportive and objective culture of performance and skills appraisal. Evaluation of the framework was conducted via an anonymous online survey tool. Nine peer reviewers and fourteen recipients provided feedback onmore » its effectiveness and the challenges and limitations of the approach. Findings from the evaluation were positive and suggested that both groups gained benefit from and expressed a strong interest in embedding the approach more routinely. Respondents identified common challenges related to the limited ability to implement suggested development strategies; this was strongly associated with time and rostering issues. This framework successfully defined expectations for practice and provided a fair and objective feedback process that focussed on skills development. It empowered staff to maintain their skills and reach their professional potential. Management support, particularly in regard to provision of protected time was highlighted as critical to the framework's ongoing success. The demonstrated benefits arising in terms of staff satisfaction and development highlight the importance of this commitment to the modern radiation therapy workforce.« less

Evaluation of Frameworks for HSCT Design Optimization

NASA Technical Reports Server (NTRS)

Krishnan, Ramki

1998-01-01

This report is an evaluation of engineering frameworks that could be used to augment, supplement, or replace the existing FIDO 3.5 (Framework for Interdisciplinary Design and Optimization Version 3.5) framework. The report begins with the motivation for this effort, followed by a description of an "ideal" multidisciplinary design and optimization (MDO) framework. The discussion then turns to how each candidate framework stacks up against this ideal. This report ends with recommendations as to the "best" frameworks that should be down-selected for detailed review.

Health services research evaluation principles. Broadening a general framework for evaluating health information technology.

PubMed

Sockolow, P S; Crawford, P R; Lehmann, H P

2012-01-01

Our forthcoming national experiment in increased health information technology (HIT) adoption funded by the American Recovery and Reinvestment Act of 2009 will require a comprehensive approach to evaluating HIT. The quality of evaluation studies of HIT to date reveals a need for broader evaluation frameworks that limits the generalizability of findings and the depth of lessons learned. Develop an informatics evaluation framework for health information technology (HIT) integrating components of health services research (HSR) evaluation and informatics evaluation to address identified shortcomings in available HIT evaluation frameworks. A systematic literature review updated and expanded the exhaustive review by Ammenwerth and deKeizer (AdK). From retained studies, criteria were elicited and organized into classes within a framework. The resulting Health Information Technology Research-based Evaluation Framework (HITREF) was used to guide clinician satisfaction survey construction, multi-dimensional analysis of data, and interpretation of findings in an evaluation of a vanguard community health care EHR. The updated review identified 128 electronic health record (EHR) evaluation studies and seven evaluation criteria not in AdK: EHR Selection/Development/Training; Patient Privacy Concerns; Unintended Consequences/ Benefits; Functionality; Patient Satisfaction with EHR; Barriers/Facilitators to Adoption; and Patient Satisfaction with Care. HITREF was used productively and was a complete evaluation framework which included all themes that emerged. We can recommend to future EHR evaluators that they consider adding a complete, research-based HIT evaluation framework, such as HITREF, to their evaluation tools suite to monitor HIT challenges as the federal government strives to increase HIT adoption.

A Framework for the Flexible Content Packaging of Learning Objects and Learning Designs

ERIC Educational Resources Information Center

Lukasiak, Jason; Agostinho, Shirley; Burnett, Ian; Drury, Gerrard; Goodes, Jason; Bennett, Sue; Lockyer, Lori; Harper, Barry

2004-01-01

This paper presents a platform-independent method for packaging learning objects and learning designs. The method, entitled a Smart Learning Design Framework, is based on the MPEG-21 standard, and uses IEEE Learning Object Metadata (LOM) to provide bibliographic, technical, and pedagogical descriptors for the retrieval and description of learning…

Sediment Evaluation Framework for the Pacific Northwest

EPA Pesticide Factsheets

The Sediment Evaluation Framework provides a regional framework for assessment, characterization and management of sediments in the Pacific Northwest to determine suitability for unconfined in-water disposal.

Judgment under uncertainty; a probabilistic evaluation framework for decision-making about sanitation systems in low-income countries.

PubMed

Malekpour, Shirin; Langeveld, Jeroen; Letema, Sammy; Clemens, François; van Lier, Jules B

2013-03-30

This paper introduces the probabilistic evaluation framework, to enable transparent and objective decision-making in technology selection for sanitation solutions in low-income countries. The probabilistic framework recognizes the often poor quality of the available data for evaluations. Within this framework, the evaluations will be done based on the probabilities that the expected outcomes occur in practice, considering the uncertainties in evaluation parameters. Consequently, the outcome of evaluations will not be single point estimates; but there exists a range of possible outcomes. A first trial application of this framework for evaluation of sanitation options in the Nyalenda settlement in Kisumu, Kenya, showed how the range of values that an evaluation parameter may obtain in practice would influence the evaluation outcomes. In addition, as the probabilistic evaluation requires various site-specific data, sensitivity analysis was performed to determine the influence of each data set quality on the evaluation outcomes. Based on that, data collection activities could be (re)directed, in a trade-off between the required investments in those activities and the resolution of the decisions that are to be made. Copyright © 2013 Elsevier Ltd. All rights reserved.

A Conceptual Framework for Error Remediation with Multiple External Representations Applied to Learning Objects

ERIC Educational Resources Information Center

Leite, Maici Duarte; Marczal, Diego; Pimentel, Andrey Ricardo; Direne, Alexandre Ibrahim

2014-01-01

This paper presents the application of some concepts of Intelligent Tutoring Systems (ITS) to elaborate a conceptual framework that uses the remediation of errors with Multiple External Representations (MERs) in Learning Objects (LO). To this is demonstrated a development of LO for teaching the Pythagorean Theorem through this framework. This…

BioInt: an integrative biological object-oriented application framework and interpreter.

PubMed

Desai, Sanket; Burra, Prasad

2015-01-01

BioInt, a biological programming application framework and interpreter, is an attempt to equip the researchers with seamless integration, efficient extraction and effortless analysis of the data from various biological databases and algorithms. Based on the type of biological data, algorithms and related functionalities, a biology-specific framework was developed which has nine modules. The modules are a compilation of numerous reusable BioADTs. This software ecosystem containing more than 450 biological objects underneath the interpreter makes it flexible, integrative and comprehensive. Similar to Python, BioInt eliminates the compilation and linking steps cutting the time significantly. The researcher can write the scripts using available BioADTs (following C++ syntax) and execute them interactively or use as a command line application. It has features that enable automation, extension of the framework with new/external BioADTs/libraries and deployment of complex work flows.

A data set for evaluating the performance of multi-class multi-object video tracking

NASA Astrophysics Data System (ADS)

Chakraborty, Avishek; Stamatescu, Victor; Wong, Sebastien C.; Wigley, Grant; Kearney, David

2017-05-01

One of the challenges in evaluating multi-object video detection, tracking and classification systems is having publically available data sets with which to compare different systems. However, the measures of performance for tracking and classification are different. Data sets that are suitable for evaluating tracking systems may not be appropriate for classification. Tracking video data sets typically only have ground truth track IDs, while classification video data sets only have ground truth class-label IDs. The former identifies the same object over multiple frames, while the latter identifies the type of object in individual frames. This paper describes an advancement of the ground truth meta-data for the DARPA Neovision2 Tower data set to allow both the evaluation of tracking and classification. The ground truth data sets presented in this paper contain unique object IDs across 5 different classes of object (Car, Bus, Truck, Person, Cyclist) for 24 videos of 871 image frames each. In addition to the object IDs and class labels, the ground truth data also contains the original bounding box coordinates together with new bounding boxes in instances where un-annotated objects were present. The unique IDs are maintained during occlusions between multiple objects or when objects re-enter the field of view. This will provide: a solid foundation for evaluating the performance of multi-object tracking of different types of objects, a straightforward comparison of tracking system performance using the standard Multi Object Tracking (MOT) framework, and classification performance using the Neovision2 metrics. These data have been hosted publically.

A framework for assessing Health Economic Evaluation (HEE) quality appraisal instruments

PubMed Central

2012-01-01

Background Health economic evaluations support the health care decision-making process by providing information on costs and consequences of health interventions. The quality of such studies is assessed by health economic evaluation (HEE) quality appraisal instruments. At present, there is no instrument for measuring and improving the quality of such HEE quality appraisal instruments. Therefore, the objectives of this study are to establish a framework for assessing the quality of HEE quality appraisal instruments to support and improve their quality, and to apply this framework to those HEE quality appraisal instruments which have been subject to more scrutiny than others, in order to test the framework and to demonstrate the shortcomings of existing HEE quality appraisal instruments. Methods To develop the quality assessment framework for HEE quality appraisal instruments, the experiences of using appraisal tools for clinical guidelines are used. Based on a deductive iterative process, clinical guideline appraisal instruments identified through literature search are reviewed, consolidated, and adapted to produce the final quality assessment framework for HEE quality appraisal instruments. Results The final quality assessment framework for HEE quality appraisal instruments consists of 36 items organized within 7 dimensions, each of which captures a specific domain of quality. Applying the quality assessment framework to four existing HEE quality appraisal instruments, it is found that these four quality appraisal instruments are of variable quality. Conclusions The framework described in this study should be regarded as a starting point for appraising the quality of HEE quality appraisal instruments. This framework can be used by HEE quality appraisal instrument producers to support and improve the quality and acceptance of existing and future HEE quality appraisal instruments. By applying this framework, users of HEE quality appraisal instruments can become aware

[A framework for evaluating primary health care in Latin America].

PubMed

Haggerty, Jeannie L; Yavich, Natalia; Báscolo, Ernesto Pablo

2009-11-01

To determine the relevancy of applying the Canadian primary health care (PHC) assessment strategy to Latin America and to propose any modifications that might be needed for reaching a consensus in Latin America. The Delphi method was used to reach a consensus among 29 experts engaged in PHC development or evaluation in Latin America. Four virtual sessions and a face-to-face meeting were held to discuss the PHC evaluation logic model and the seven goals and six conditioning factors that make up the Canadian strategy, as well as any questions regarding the evaluation and indicators. The relevance of each concept was ranked according to the perspective of the Latin American countries. The experts considered the Canadian strategy's objectives and conditioning factors to be highly relevant to assessing PHC in Latin America, though they acknowledged that additional modification would increase relevance. The chief suggestions were to create a PHC vision and mission, to include additional objectives and conditioning factors, and to rework the original set. The objectives that concerned coordination and integrated comprehensive care did not achieve a high degree of consensus because of ambiguities in the original text and multiple interpretations of statements regarding certain aspects of the evaluation. Considerable progress was made on the road to building a PHC evaluation framework for the Region of the Americas. Indicators and information-gathering tools, which can be appropriately and practically applied in diverse contexts, need to be developed.

Assessment of Intralaminar Progressive Damage and Failure Analysis Using an Efficient Evaluation Framework

NASA Technical Reports Server (NTRS)

Hyder, Imran; Schaefer, Joseph; Justusson, Brian; Wanthal, Steve; Leone, Frank; Rose, Cheryl

2017-01-01

Reducing the timeline for development and certification for composite structures has been a long standing objective of the aerospace industry. This timeline can be further exacerbated when attempting to integrate new fiber-reinforced composite materials due to the large number of testing required at every level of design. computational progressive damage and failure analysis (PDFA) attempts to mitigate this effect; however, new PDFA methods have been slow to be adopted in industry since material model evaluation techniques have not been fully defined. This study presents an efficient evaluation framework which uses a piecewise verification and validation (V&V) approach for PDFA methods. Specifically, the framework is applied to evaluate PDFA research codes within the context of intralaminar damage. Methods are incrementally taken through various V&V exercises specifically tailored to study PDFA intralaminar damage modeling capability. Finally, methods are evaluated against a defined set of success criteria to highlight successes and limitations.

An Object-Oriented Finite Element Framework for Multiphysics Phase Field Simulations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Michael R Tonks; Derek R Gaston; Paul C Millett

2012-01-01

The phase field approach is a powerful and popular method for modeling microstructure evolution. In this work, advanced numerical tools are used to create a phase field framework that facilitates rapid model development. This framework, called MARMOT, is based on Idaho National Laboratory's finite element Multiphysics Object-Oriented Simulation Environment. In MARMOT, the system of phase field partial differential equations (PDEs) are solved simultaneously with PDEs describing additional physics, such as solid mechanics and heat conduction, using the Jacobian-Free Newton Krylov Method. An object-oriented architecture is created by taking advantage of commonalities in phase fields models to facilitate development of newmore » models with very little written code. In addition, MARMOT provides access to mesh and time step adaptivity, reducing the cost for performing simulations with large disparities in both spatial and temporal scales. In this work, phase separation simulations are used to show the numerical performance of MARMOT. Deformation-induced grain growth and void growth simulations are included to demonstrate the muliphysics capability.« less

Evaluating impact of clinical guidelines using a realist evaluation framework.

PubMed

Reddy, Sandeep; Wakerman, John; Westhorp, Gill; Herring, Sally

2015-12-01

The Remote Primary Health Care Manuals (RPHCM) project team manages the development and publication of clinical protocols and procedures for primary care clinicians practicing in remote Australia. The Central Australian Rural Practitioners Association Standard Treatment Manual, the flagship manual of the RPHCM suite, has been evaluated for accessibility and acceptability in remote clinics three times in its 20-year history. These evaluations did not consider a theory-based framework or a programme theory, resulting in some limitations with the evaluation findings. With the RPHCM having an aim of enabling evidence-based practice in remote clinics and anecdotally reported to do so, testing this empirically for the full suite is vital for both stakeholders and future editions of the RPHCM. The project team utilized a realist evaluation framework to assess how, why and for what the RPHCM were being used by remote practitioners. A theory regarding the circumstances in which the manuals have and have not enabled evidence-based practice in the remote clinical context was tested. The project assessed this theory for all the manuals in the RPHCM suite, across government and aboriginal community-controlled clinics, in three regions of Australia. Implementing a realist evaluation framework to generate robust findings in this context has required innovation in the evaluation design and adaptation by researchers. This article captures the RPHCM team's experience in designing this evaluation. © 2015 John Wiley & Sons, Ltd.

Objective Fidelity Evaluation in Multisensory Virtual Environments: Auditory Cue Fidelity in Flight Simulation

PubMed Central

Meyer, Georg F.; Wong, Li Ting; Timson, Emma; Perfect, Philip; White, Mark D.

2012-01-01

We argue that objective fidelity evaluation of virtual environments, such as flight simulation, should be human-performance-centred and task-specific rather than measure the match between simulation and physical reality. We show how principled experimental paradigms and behavioural models to quantify human performance in simulated environments that have emerged from research in multisensory perception provide a framework for the objective evaluation of the contribution of individual cues to human performance measures of fidelity. We present three examples in a flight simulation environment as a case study: Experiment 1: Detection and categorisation of auditory and kinematic motion cues; Experiment 2: Performance evaluation in a target-tracking task; Experiment 3: Transferrable learning of auditory motion cues. We show how the contribution of individual cues to human performance can be robustly evaluated for each task and that the contribution is highly task dependent. The same auditory cues that can be discriminated and are optimally integrated in experiment 1, do not contribute to target-tracking performance in an in-flight refuelling simulation without training, experiment 2. In experiment 3, however, we demonstrate that the auditory cue leads to significant, transferrable, performance improvements with training. We conclude that objective fidelity evaluation requires a task-specific analysis of the contribution of individual cues. PMID:22957068

Quantitative framework for prospective motion correction evaluation.

PubMed

Pannetier, Nicolas A; Stavrinos, Theano; Ng, Peter; Herbst, Michael; Zaitsev, Maxim; Young, Karl; Matson, Gerald; Schuff, Norbert

2016-02-01

Establishing a framework to evaluate performances of prospective motion correction (PMC) MRI considering motion variability between MRI scans. A framework was developed to obtain quantitative comparisons between different motion correction setups, considering that varying intrinsic motion patterns between acquisitions can induce bias. Intrinsic motion was considered by replaying in a phantom experiment the recorded motion trajectories from subjects. T1-weighted MRI on five volunteers and two different marker fixations (mouth guard and nose bridge fixations) were used to test the framework. Two metrics were investigated to quantify the improvement of the image quality with PMC. Motion patterns vary between subjects as well as between repeated scans within a subject. This variability can be approximated by replaying the motion in a distinct phantom experiment and used as a covariate in models comparing motion corrections. We show that considering the intrinsic motion alters the statistical significance in comparing marker fixations. As an example, two marker fixations, a mouth guard and a nose bridge, were evaluated in terms of their effectiveness for PMC. A mouth guard achieved better PMC performance. Intrinsic motion patterns can bias comparisons between PMC configurations and must be considered for robust evaluations. A framework for evaluating intrinsic motion patterns in PMC is presented. © 2015 Wiley Periodicals, Inc.

MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans.

PubMed

Mendrik, Adriënne M; Vincken, Koen L; Kuijf, Hugo J; Breeuwer, Marcel; Bouvy, Willem H; de Bresser, Jeroen; Alansary, Amir; de Bruijne, Marleen; Carass, Aaron; El-Baz, Ayman; Jog, Amod; Katyal, Ranveer; Khan, Ali R; van der Lijn, Fedde; Mahmood, Qaiser; Mukherjee, Ryan; van Opbroek, Annegreet; Paneri, Sahil; Pereira, Sérgio; Persson, Mikael; Rajchl, Martin; Sarikaya, Duygu; Smedby, Örjan; Silva, Carlos A; Vrooman, Henri A; Vyas, Saurabh; Wang, Chunliang; Zhao, Liang; Biessels, Geert Jan; Viergever, Max A

2015-01-01

Many methods have been proposed for tissue segmentation in brain MRI scans. The multitude of methods proposed complicates the choice of one method above others. We have therefore established the MRBrainS online evaluation framework for evaluating (semi)automatic algorithms that segment gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) on 3T brain MRI scans of elderly subjects (65-80 y). Participants apply their algorithms to the provided data, after which their results are evaluated and ranked. Full manual segmentations of GM, WM, and CSF are available for all scans and used as the reference standard. Five datasets are provided for training and fifteen for testing. The evaluated methods are ranked based on their overall performance to segment GM, WM, and CSF and evaluated using three evaluation metrics (Dice, H95, and AVD) and the results are published on the MRBrainS13 website. We present the results of eleven segmentation algorithms that participated in the MRBrainS13 challenge workshop at MICCAI, where the framework was launched, and three commonly used freeware packages: FreeSurfer, FSL, and SPM. The MRBrainS evaluation framework provides an objective and direct comparison of all evaluated algorithms and can aid in selecting the best performing method for the segmentation goal at hand.

MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans

PubMed Central

Mendrik, Adriënne M.; Vincken, Koen L.; Kuijf, Hugo J.; Breeuwer, Marcel; Bouvy, Willem H.; de Bresser, Jeroen; Alansary, Amir; de Bruijne, Marleen; Carass, Aaron; El-Baz, Ayman; Jog, Amod; Katyal, Ranveer; Khan, Ali R.; van der Lijn, Fedde; Mahmood, Qaiser; Mukherjee, Ryan; van Opbroek, Annegreet; Paneri, Sahil; Pereira, Sérgio; Rajchl, Martin; Sarikaya, Duygu; Smedby, Örjan; Silva, Carlos A.; Vrooman, Henri A.; Vyas, Saurabh; Wang, Chunliang; Zhao, Liang; Biessels, Geert Jan; Viergever, Max A.

2015-01-01

Many methods have been proposed for tissue segmentation in brain MRI scans. The multitude of methods proposed complicates the choice of one method above others. We have therefore established the MRBrainS online evaluation framework for evaluating (semi)automatic algorithms that segment gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) on 3T brain MRI scans of elderly subjects (65–80 y). Participants apply their algorithms to the provided data, after which their results are evaluated and ranked. Full manual segmentations of GM, WM, and CSF are available for all scans and used as the reference standard. Five datasets are provided for training and fifteen for testing. The evaluated methods are ranked based on their overall performance to segment GM, WM, and CSF and evaluated using three evaluation metrics (Dice, H95, and AVD) and the results are published on the MRBrainS13 website. We present the results of eleven segmentation algorithms that participated in the MRBrainS13 challenge workshop at MICCAI, where the framework was launched, and three commonly used freeware packages: FreeSurfer, FSL, and SPM. The MRBrainS evaluation framework provides an objective and direct comparison of all evaluated algorithms and can aid in selecting the best performing method for the segmentation goal at hand. PMID:26759553

Sequentially Executed Model Evaluation Framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

2015-10-20

Provides a message passing framework between generic input, model and output drivers, and specifies an API for developing such drivers. Also provides batch and real-time controllers which step the model and I/O through the time domain (or other discrete domain), and sample I/O drivers. This is a library framework, and does not, itself, solve any problems or execute any modeling. The SeMe framework aids in development of models which operate on sequential information, such as time-series, where evaluation is based on prior results combined with new data for this iteration. Has applications in quality monitoring, and was developed as partmore » of the CANARY-EDS software, where real-time water quality data is being analyzed for anomalies.« less

Quantile equivalence to evaluate compliance with habitat management objectives

USGS Publications Warehouse

Cade, Brian S.; Johnson, Pamela R.

2011-01-01

Equivalence estimated with linear quantile regression was used to evaluate compliance with habitat management objectives at Arapaho National Wildlife Refuge based on monitoring data collected in upland (5,781 ha; n = 511 transects) and riparian and meadow (2,856 ha, n = 389 transects) habitats from 2005 to 2008. Quantiles were used because the management objectives specified proportions of the habitat area that needed to comply with vegetation criteria. The linear model was used to obtain estimates that were averaged across 4 y. The equivalence testing framework allowed us to interpret confidence intervals for estimated proportions with respect to intervals of vegetative criteria (equivalence regions) in either a liberal, benefit-of-doubt or conservative, fail-safe approach associated with minimizing alternative risks. Simple Boolean conditional arguments were used to combine the quantile equivalence results for individual vegetation components into a joint statement for the multivariable management objectives. For example, management objective 2A required at least 809 ha of upland habitat with a shrub composition ≥0.70 sagebrush (Artemisia spp.), 20–30% canopy cover of sagebrush ≥25 cm in height, ≥20% canopy cover of grasses, and ≥10% canopy cover of forbs on average over 4 y. Shrub composition and canopy cover of grass each were readily met on >3,000 ha under either conservative or liberal interpretations of sampling variability. However, there were only 809–1,214 ha (conservative to liberal) with ≥10% forb canopy cover and 405–1,098 ha with 20–30%canopy cover of sagebrush ≥25 cm in height. Only 91–180 ha of uplands simultaneously met criteria for all four components, primarily because canopy cover of sagebrush and forbs was inversely related when considered at the spatial scale (30 m) of a sample transect. We demonstrate how the quantile equivalence analyses also can help refine the numerical specification of habitat objectives and explore

Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery

NASA Astrophysics Data System (ADS)

Zhong, Yanfei; Han, Xiaobing; Zhang, Liangpei

2018-04-01

Multi-class geospatial object detection from high spatial resolution (HSR) remote sensing imagery is attracting increasing attention in a wide range of object-related civil and engineering applications. However, the distribution of objects in HSR remote sensing imagery is location-variable and complicated, and how to accurately detect the objects in HSR remote sensing imagery is a critical problem. Due to the powerful feature extraction and representation capability of deep learning, the deep learning based region proposal generation and object detection integrated framework has greatly promoted the performance of multi-class geospatial object detection for HSR remote sensing imagery. However, due to the translation caused by the convolution operation in the convolutional neural network (CNN), although the performance of the classification stage is seldom influenced, the localization accuracies of the predicted bounding boxes in the detection stage are easily influenced. The dilemma between translation-invariance in the classification stage and translation-variance in the object detection stage has not been addressed for HSR remote sensing imagery, and causes position accuracy problems for multi-class geospatial object detection with region proposal generation and object detection. In order to further improve the performance of the region proposal generation and object detection integrated framework for HSR remote sensing imagery object detection, a position-sensitive balancing (PSB) framework is proposed in this paper for multi-class geospatial object detection from HSR remote sensing imagery. The proposed PSB framework takes full advantage of the fully convolutional network (FCN), on the basis of a residual network, and adopts the PSB framework to solve the dilemma between translation-invariance in the classification stage and translation-variance in the object detection stage. In addition, a pre-training mechanism is utilized to accelerate the training procedure

The Macro- and Micropolitics of Personnel Evaluation: A Framework.

ERIC Educational Resources Information Center

Bridges, Edwin M.; Groves, Barry R.

1999-01-01

Explicates a conceptual framework for analyzing the politics of personnel evaluation in an educational context. Using several elements of the framework, discusses the politics of teacher evaluation in California in relation to the types of personnel evaluation decisions, the actors, their access to these decisions, sources and levels of power, and…

A framework for the evaluation of new interventional procedures.

PubMed

Lourenco, Tania; Grant, Adrian M; Burr, Jennifer M; Vale, Luke

2012-03-01

The introduction of new interventional procedures is less regulated than for other health technologies such as pharmaceuticals. Decisions are often taken on evidence of efficacy and short-term safety from small-scale usually observational studies. This reflects the particular challenges of evaluating interventional procedures - the extra facets of skill and training and the difficulty defining a 'new' technology. Currently, there is no framework to evaluate new interventional procedures before they become available in clinical practice as opposed to new pharmaceuticals. This paper proposes a framework to guide the evaluation of a new interventional procedure. A framework was developed consisting of a four-stage progressive evaluation for a new interventional procedure: Stage 1: Development; Stage 2: Efficacy and short-term safety; Stage 3: Effectiveness and cost-effectiveness; and Stage 4: Implementation. The framework also suggests the types of studies or data collection methods that can be used to satisfy each stage. This paper makes a first step on a framework for generating evidence on new interventional procedures. The difficulties and limitations of applying such a framework are discussed. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

A cortical framework for invariant object categorization and recognition.

PubMed

Rodrigues, João; Hans du Buf, J M

2009-08-01

In this paper we present a new model for invariant object categorization and recognition. It is based on explicit multi-scale features: lines, edges and keypoints are extracted from responses of simple, complex and end-stopped cells in cortical area V1, and keypoints are used to construct saliency maps for Focus-of-Attention. The model is a functional but dichotomous one, because keypoints are employed to model the "where" data stream, with dynamic routing of features from V1 to higher areas to obtain translation, rotation and size invariance, whereas lines and edges are employed in the "what" stream for object categorization and recognition. Furthermore, both the "where" and "what" pathways are dynamic in that information at coarse scales is employed first, after which information at progressively finer scales is added in order to refine the processes, i.e., both the dynamic feature routing and the categorization level. The construction of group and object templates, which are thought to be available in the prefrontal cortex with "what" and "where" components in PF46d and PF46v, is also illustrated. The model was tested in the framework of an integrated and biologically plausible architecture.

Objective criteria ranking framework for renewable energy policy decisions in Nigeria

NASA Astrophysics Data System (ADS)

K, Nwofor O.; N, Dike V.

2016-08-01

We present a framework that seeks to improve the objectivity of renewable energy policy decisions in Nigeria. It consists of expert ranking of resource abundance, resource efficiency and resource environmental comfort in the choice of renewable energy options for large scale power generation. The rankings are converted to a more objective function called Resource Appraisal Function (RAF) using dependence operators derived from logical relationships amongst the various criteria. The preferred option is that with the highest average RAF coupled with the least RAF variance. The method can be extended to more options, more criteria, and more opinions and can be adapted for similar decisions in education, environment and health sectors.

Using Recommendations in Evaluation: A Decision-Making Framework for Evaluators

ERIC Educational Resources Information Center

Iriti, Jennifer E.; Bickel, William E.; Nelson, Catherine Awsumb

2005-01-01

Is it appropriate and useful for evaluators to use findings to make recommendations? If so, under what circumstances? How specific should they be? This article presents a decision-making framework for the appropriateness of recommendations in varying contexts. On the basis of reviews of evaluation theory, selected evaluation reports, and feedback…

User's Manual for the Object User Interface (OUI): An Environmental Resource Modeling Framework

USGS Publications Warehouse

Markstrom, Steven L.; Koczot, Kathryn M.

2008-01-01

The Object User Interface is a computer application that provides a framework for coupling environmental-resource models and for managing associated temporal and spatial data. The Object User Interface is designed to be easily extensible to incorporate models and data interfaces defined by the user. Additionally, the Object User Interface is highly configurable through the use of a user-modifiable, text-based control file that is written in the eXtensible Markup Language. The Object User Interface user's manual provides (1) installation instructions, (2) an overview of the graphical user interface, (3) a description of the software tools, (4) a project example, and (5) specifications for user configuration and extension.

The 'global health' education framework: a conceptual guide for monitoring, evaluation and practice

PubMed Central

2011-01-01

Background In the past decades, the increasing importance of and rapid changes in the global health arena have provoked discussions on the implications for the education of health professionals. In the case of Germany, it remains yet unclear whether international or global aspects are sufficiently addressed within medical education. Evaluation challenges exist in Germany and elsewhere due to a lack of conceptual guides to develop, evaluate or assess education in this field. Objective To propose a framework conceptualising 'global health' education (GHE) in practice, to guide the evaluation and monitoring of educational interventions and reforms through a set of key indicators that characterise GHE. Methods Literature review; deduction. Results and Conclusion Currently, 'new' health challenges and educational needs as a result of the globalisation process are discussed and linked to the evolving term 'global health'. The lack of a common definition of this term complicates attempts to analyse global health in the field of education. The proposed GHE framework addresses these problems and presents a set of key characteristics of education in this field. The framework builds on the models of 'social determinants of health' and 'globalisation and health' and is oriented towards 'health for all' and 'health equity'. It provides an action-oriented construct for a bottom-up engagement with global health by the health workforce. Ten indicators are deduced for use in monitoring and evaluation. PMID:21501519

Developing an evaluation framework for clinical redesign programs: lessons learnt.

PubMed

Samaranayake, Premaratne; Dadich, Ann; Fitzgerald, Anneke; Zeitz, Kathryn

2016-09-19

Purpose The purpose of this paper is to present lessons learnt through the development of an evaluation framework for a clinical redesign programme - the aim of which was to improve the patient journey through improved discharge practices within an Australian public hospital. Design/methodology/approach The development of the evaluation framework involved three stages - namely, the analysis of secondary data relating to the discharge planning pathway; the analysis of primary data including field-notes and interview transcripts on hospital processes; and the triangulation of these data sets to devise the framework. The evaluation framework ensured that resource use, process management, patient satisfaction, and staff well-being and productivity were each connected with measures, targets, and the aim of clinical redesign programme. Findings The application of business process management and a balanced scorecard enabled a different way of framing the evaluation, ensuring measurable outcomes were connected to inputs and outputs. Lessons learnt include: first, the importance of mixed-methods research to devise the framework and evaluate the redesigned processes; second, the need for appropriate tools and resources to adequately capture change across the different domains of the redesign programme; and third, the value of developing and applying an evaluative framework progressively. Research limitations/implications The evaluation framework is limited by its retrospective application to a clinical process redesign programme. Originality/value This research supports benchmarking with national and international practices in relation to best practice healthcare redesign processes. Additionally, it provides a theoretical contribution on evaluating health services improvement and redesign initiatives.

Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography.

PubMed

Kirişli, H A; Schaap, M; Metz, C T; Dharampal, A S; Meijboom, W B; Papadopoulou, S L; Dedic, A; Nieman, K; de Graaf, M A; Meijs, M F L; Cramer, M J; Broersen, A; Cetin, S; Eslami, A; Flórez-Valencia, L; Lor, K L; Matuszewski, B; Melki, I; Mohr, B; Oksüz, I; Shahzad, R; Wang, C; Kitslaar, P H; Unal, G; Katouzian, A; Örkisz, M; Chen, C M; Precioso, F; Najman, L; Masood, S; Ünay, D; van Vliet, L; Moreno, R; Goldenberg, R; Vuçini, E; Krestin, G P; Niessen, W J; van Walsum, T

2013-12-01

Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CTA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CTA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (1) (semi-)automatically detect and quantify stenosis on CTA, in comparison with quantitative coronary angiography (QCA) and CTA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CTA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CTA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at http://coronary.bigr.nl/stenoses/. Copyright © 2013 Elsevier B.V. All rights reserved.

Development of a Dynamically Configurable, Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation

NASA Technical Reports Server (NTRS)

Afjeh, Abdollah A.; Reed, John A.

2003-01-01

The following reports are presented on this project:A first year progress report on: Development of a Dynamically Configurable,Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation; A second year progress report on: Development of a Dynamically Configurable, Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation; An Extensible, Interchangeable and Sharable Database Model for Improving Multidisciplinary Aircraft Design; Interactive, Secure Web-enabled Aircraft Engine Simulation Using XML Databinding Integration; and Improving the Aircraft Design Process Using Web-based Modeling and Simulation.

A hydromorphological framework for the evaluation of e-flows

NASA Astrophysics Data System (ADS)

Bussettini, Martina; Rinaldi, Massimo; Grant, Gordon

2017-04-01

Anthropogenic alteration of hydromorphological processes in rivers is a major factor that diminishes river health and undermines environmental objectives envisaged by river protection policies. Specifying environmental flows to address those impacts can be a key strategy for the maintenance of functional river processes and the achievement of those objectives. Environmental flows are determined by various methods and approaches, based primarily on hydrological and/or hydraulic evaluations, although holistic methodologies, considering the many interacting factors that structure aquatic ecosystems, including sediments, are increasingly used. Hydrological and geomorphological processes are highly coupled and any change in one typically affects the other. The coupling varies over different spatial and temporal scales, and changing either hydrological or geomorphological processes can result in alteration of river habitats, ultimately impacting ecological processes. In spite of these linkages, current restoration approaches typically focus only on changes on hydrological regime as a means promoting ecological enhancements. Neglecting sediment transport and its interaction with flow in shaping riverine habitats is likely to results not only in minor or no enhancements in the ecology, but may also increase the costs of water use. A more integrated view of how human activities jointly affect sediment regime, river morphology and river flows is therefore needed in order to determine the most effective actions to rehabilitate river processes to desired states. These states involve considerations of the combination of intrinsic ("natural") conditions (e.g. river sensitivity and morphological potential, off-site conditions) and socio-economic constraints. The evaluation of such factors, the analysis of different scenarios, and the selection of appropriate actions require the contextualization of river reaches within a wider spatial-temporal hydromorphological framework. Here

Secure and Efficient Regression Analysis Using a Hybrid Cryptographic Framework: Development and Evaluation.

PubMed

Sadat, Md Nazmus; Jiang, Xiaoqian; Aziz, Md Momin Al; Wang, Shuang; Mohammed, Noman

2018-03-05

Machine learning is an effective data-driven tool that is being widely used to extract valuable patterns and insights from data. Specifically, predictive machine learning models are very important in health care for clinical data analysis. The machine learning algorithms that generate predictive models often require pooling data from different sources to discover statistical patterns or correlations among different attributes of the input data. The primary challenge is to fulfill one major objective: preserving the privacy of individuals while discovering knowledge from data. Our objective was to develop a hybrid cryptographic framework for performing regression analysis over distributed data in a secure and efficient way. Existing secure computation schemes are not suitable for processing the large-scale data that are used in cutting-edge machine learning applications. We designed, developed, and evaluated a hybrid cryptographic framework, which can securely perform regression analysis, a fundamental machine learning algorithm using somewhat homomorphic encryption and a newly introduced secure hardware component of Intel Software Guard Extensions (Intel SGX) to ensure both privacy and efficiency at the same time. Experimental results demonstrate that our proposed method provides a better trade-off in terms of security and efficiency than solely secure hardware-based methods. Besides, there is no approximation error. Computed model parameters are exactly similar to plaintext results. To the best of our knowledge, this kind of secure computation model using a hybrid cryptographic framework, which leverages both somewhat homomorphic encryption and Intel SGX, is not proposed or evaluated to this date. Our proposed framework ensures data security and computational efficiency at the same time. ©Md Nazmus Sadat, Xiaoqian Jiang, Md Momin Al Aziz, Shuang Wang, Noman Mohammed. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 05.03.2018.

Probabilistic Evaluation of Ecological and Economic Objectives of River Basin Management Reveals a Potential Flaw in the Goal Setting of the EU Water Framework Directive

NASA Astrophysics Data System (ADS)

Hjerppe, Turo; Taskinen, Antti; Kotamäki, Niina; Malve, Olli; Kettunen, Juhani

2017-04-01

The biological status of European lakes has not improved as expected despite up-to-date legislation and ecological standards. As a result, the realism of objectives and the attainment of related ecological standards are under doubt. This paper gets to the bottom of a river basin management plan of a eutrophic lake in Finland and presents the ecological and economic impacts of environmental and societal drivers and planned management measures. For these purposes, we performed a Monte Carlo simulation of a diffuse nutrient load, lake water quality and cost-benefit models. Simulations were integrated into a Bayesian influence diagram that revealed the basic uncertainties. It turned out that the attainment of good ecological status as qualified in the Water Framework Directive of the European Union is unlikely within given socio-economic constraints. Therefore, management objectives and ecological and economic standards need to be reassessed and reset to provide a realistic goal setting for management. More effort should be put into the evaluation of the total monetary benefits and on the monitoring of lake phosphorus balances to reduce the uncertainties, and the resulting margin of safety and costs and risks of planned management measures.

Probabilistic Evaluation of Ecological and Economic Objectives of River Basin Management Reveals a Potential Flaw in the Goal Setting of the EU Water Framework Directive.

PubMed

Hjerppe, Turo; Taskinen, Antti; Kotamäki, Niina; Malve, Olli; Kettunen, Juhani

2017-04-01

The biological status of European lakes has not improved as expected despite up-to-date legislation and ecological standards. As a result, the realism of objectives and the attainment of related ecological standards are under doubt. This paper gets to the bottom of a river basin management plan of a eutrophic lake in Finland and presents the ecological and economic impacts of environmental and societal drivers and planned management measures. For these purposes, we performed a Monte Carlo simulation of a diffuse nutrient load, lake water quality and cost-benefit models. Simulations were integrated into a Bayesian influence diagram that revealed the basic uncertainties. It turned out that the attainment of good ecological status as qualified in the Water Framework Directive of the European Union is unlikely within given socio-economic constraints. Therefore, management objectives and ecological and economic standards need to be reassessed and reset to provide a realistic goal setting for management. More effort should be put into the evaluation of the total monetary benefits and on the monitoring of lake phosphorus balances to reduce the uncertainties, and the resulting margin of safety and costs and risks of planned management measures.

A user-centred evaluation framework for the Sealife semantic web browsers.

PubMed

Oliver, Helen; Diallo, Gayo; de Quincey, Ed; Alexopoulou, Dimitra; Habermann, Bianca; Kostkova, Patty; Schroeder, Michael; Jupp, Simon; Khelif, Khaled; Stevens, Robert; Jawaheer, Gawesh; Madle, Gemma

2009-10-01

Semantically-enriched browsing has enhanced the browsing experience by providing contextualized dynamically generated Web content, and quicker access to searched-for information. However, adoption of Semantic Web technologies is limited and user perception from the non-IT domain sceptical. Furthermore, little attention has been given to evaluating semantic browsers with real users to demonstrate the enhancements and obtain valuable feedback. The Sealife project investigates semantic browsing and its application to the life science domain. Sealife's main objective is to develop the notion of context-based information integration by extending three existing Semantic Web browsers (SWBs) to link the existing Web to the eScience infrastructure. This paper describes a user-centred evaluation framework that was developed to evaluate the Sealife SWBs that elicited feedback on users' perceptions on ease of use and information findability. Three sources of data: i) web server logs; ii) user questionnaires; and iii) semi-structured interviews were analysed and comparisons made between each browser and a control system. It was found that the evaluation framework used successfully elicited users' perceptions of the three distinct SWBs. The results indicate that the browser with the most mature and polished interface was rated higher for usability, and semantic links were used by the users of all three browsers. Confirmation or contradiction of our original hypotheses with relation to SWBs is detailed along with observations of implementation issues.

Towards an evaluation framework for Laboratory Information Systems.

PubMed

Yusof, Maryati M; Arifin, Azila

Laboratory testing and reporting are error-prone and redundant due to repeated, unnecessary requests and delayed or missed reactions to laboratory reports. Occurring errors may negatively affect the patient treatment process and clinical decision making. Evaluation on laboratory testing and Laboratory Information System (LIS) may explain the root cause to improve the testing process and enhance LIS in supporting the process. This paper discusses a new evaluation framework for LIS that encompasses the laboratory testing cycle and the socio-technical part of LIS. Literature review on discourses, dimensions and evaluation methods of laboratory testing and LIS. A critical appraisal of the Total Testing Process (TTP) and the human, organization, technology-fit factors (HOT-fit) evaluation frameworks was undertaken in order to identify error incident, its contributing factors and preventive action pertinent to laboratory testing process and LIS. A new evaluation framework for LIS using a comprehensive and socio-technical approach is outlined. Positive relationship between laboratory and clinical staff resulted in a smooth laboratory testing process, reduced errors and increased process efficiency whilst effective use of LIS streamlined the testing processes. The TTP-LIS framework could serve as an assessment as well as a problem-solving tool for the laboratory testing process and system. Copyright © 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Ltd. All rights reserved.

Framework for evaluating disease severity measures in older adults with comorbidity.

PubMed

Boyd, Cynthia M; Weiss, Carlos O; Halter, Jeff; Han, K Carol; Ershler, William B; Fried, Linda P

2007-03-01

Accounting for the influence of concurrent conditions on health and functional status for both research and clinical decision-making purposes is especially important in older adults. Although approaches to classifying severity of individual diseases and conditions have been developed, the utility of these classification systems has not been evaluated in the presence of multiple conditions. We present a framework for evaluating severity classification systems for common chronic diseases. The framework evaluates the: (a) goal or purpose of the classification system; (b) physiological and/or functional criteria for severity graduation; and (c) potential reliability and validity of the system balanced against burden and costs associated with classification. Approaches to severity classification of individual diseases were not originally conceived for the study of comorbidity. Therefore, they vary greatly in terms of objectives, physiological systems covered, level of severity characterization, reliability and validity, and costs and burdens. Using different severity classification systems to account for differing levels of disease severity in a patient with multiple diseases, or, assessing global disease burden may be challenging. Most approaches to severity classification are not adequate to address comorbidity. Nevertheless, thoughtful use of some existing approaches and refinement of others may advance the study of comorbidity and diagnostic and therapeutic approaches to patients with multimorbidity.

A proposed ITS evaluation framework for Texas

DOT National Transportation Integrated Search

1999-03-01

This report presents a proposed intelligent transportation system (ITS) evaluation framework that can be used by the Texas Department of Transportation (TxDOT) in developing evaluation plans for specific ITS applications and deployments in Texas. The...

Framework for Evaluation of Equity Initiatives

ERIC Educational Resources Information Center

Bexley, Emmaline; Harris, Kerri-Lee; James, Richard

2010-01-01

The Framework for Evaluation of Equity Initiatives has been prepared to support the Go8 Equity Strategy. Its purpose is to assist Group of Eight (Go8) universities to evaluate the effectiveness of their equity initiatives and interventions in the context of federal policies and the distinctive missions and responsibilities of the individual Go8…

Creating Critical Objectives and Assessments Using a Critical Communication Pedagogical Framework

ERIC Educational Resources Information Center

Kahl, David H., Jr.

2018-01-01

Courses: Instructional Communication, Graduate Teaching Assistant Training Programs. Objectives: Students will (1) understand critical communication pedagogy (CCP); (2) evaluate traditional and critical objectives and assessment procedures; and (3) create critical objectives and assessment procedures.

Designing a Workable Framework for Evaluating Distance Language Instruction

ERIC Educational Resources Information Center

Madyarov, Irshat

2009-01-01

Teaching foreign languages at distance is now becoming widespread; so is the need for evaluating online language courses. This article discusses an example of a framework that was applied to evaluate an online English as a foreign language (EFL) course at a Middle Eastern university. The development of the framework investigated areas of interest…

An Evaluation Framework and Instrument for Evaluating e-Assessment Tools

ERIC Educational Resources Information Center

Singh, Upasana Gitanjali; de Villiers, Mary Ruth

2017-01-01

e-Assessment, in the form of tools and systems that deliver and administer multiple choice questions (MCQs), is used increasingly, raising the need for evaluation and validation of such systems. This research uses literature and a series of six empirical action research studies to develop an evaluation framework of categories and criteria called…

Faunus: An object oriented framework for molecular simulation

PubMed Central

Lund, Mikael; Trulsson, Martin; Persson, Björn

2008-01-01

Background We present a C++ class library for Monte Carlo simulation of molecular systems, including proteins in solution. The design is generic and highly modular, enabling multiple developers to easily implement additional features. The statistical mechanical methods are documented by extensive use of code comments that – subsequently – are collected to automatically build a web-based manual. Results We show how an object oriented design can be used to create an intuitively appealing coding framework for molecular simulation. This is exemplified in a minimalistic C++ program that can calculate protein protonation states. We further discuss performance issues related to high level coding abstraction. Conclusion C++ and the Standard Template Library (STL) provide a high-performance platform for generic molecular modeling. Automatic generation of code documentation from inline comments has proven particularly useful in that no separate manual needs to be maintained. PMID:18241331

A user-centred evaluation framework for the Sealife semantic web browsers

PubMed Central

Oliver, Helen; Diallo, Gayo; de Quincey, Ed; Alexopoulou, Dimitra; Habermann, Bianca; Kostkova, Patty; Schroeder, Michael; Jupp, Simon; Khelif, Khaled; Stevens, Robert; Jawaheer, Gawesh; Madle, Gemma

2009-01-01

Background Semantically-enriched browsing has enhanced the browsing experience by providing contextualised dynamically generated Web content, and quicker access to searched-for information. However, adoption of Semantic Web technologies is limited and user perception from the non-IT domain sceptical. Furthermore, little attention has been given to evaluating semantic browsers with real users to demonstrate the enhancements and obtain valuable feedback. The Sealife project investigates semantic browsing and its application to the life science domain. Sealife's main objective is to develop the notion of context-based information integration by extending three existing Semantic Web browsers (SWBs) to link the existing Web to the eScience infrastructure. Methods This paper describes a user-centred evaluation framework that was developed to evaluate the Sealife SWBs that elicited feedback on users' perceptions on ease of use and information findability. Three sources of data: i) web server logs; ii) user questionnaires; and iii) semi-structured interviews were analysed and comparisons made between each browser and a control system. Results It was found that the evaluation framework used successfully elicited users' perceptions of the three distinct SWBs. The results indicate that the browser with the most mature and polished interface was rated higher for usability, and semantic links were used by the users of all three browsers. Conclusion Confirmation or contradiction of our original hypotheses with relation to SWBs is detailed along with observations of implementation issues. PMID:19796398

Evaluation of Learning Materials: A Holistic Framework

ERIC Educational Resources Information Center

Bundsgaard, Jeppe; Hansen, Thomas Illum

2011-01-01

This paper presents a holistic framework for evaluating learning materials and designs for learning. A holistic evaluation comprises investigations of the potential learning potential, the actualised learning potential, and the actual learning. Each aspect is explained and exemplified through theoretical models and definitions. (Contains 3 figures…

A Computational Framework for Quantitative Evaluation of Movement during Rehabilitation

NASA Astrophysics Data System (ADS)

Chen, Yinpeng; Duff, Margaret; Lehrer, Nicole; Sundaram, Hari; He, Jiping; Wolf, Steven L.; Rikakis, Thanassis

2011-06-01

This paper presents a novel generalized computational framework for quantitative kinematic evaluation of movement in a rehabilitation clinic setting. The framework integrates clinical knowledge and computational data-driven analysis together in a systematic manner. The framework provides three key benefits to rehabilitation: (a) the resulting continuous normalized measure allows the clinician to monitor movement quality on a fine scale and easily compare impairments across participants, (b) the framework reveals the effect of individual movement components on the composite movement performance helping the clinician decide the training foci, and (c) the evaluation runs in real-time, which allows the clinician to constantly track a patient's progress and make appropriate adaptations to the therapy protocol. The creation of such an evaluation is difficult because of the sparse amount of recorded clinical observations, the high dimensionality of movement and high variations in subject's performance. We address these issues by modeling the evaluation function as linear combination of multiple normalized kinematic attributes y = Σwiφi(xi) and estimating the attribute normalization function φi(ṡ) by integrating distributions of idealized movement and deviated movement. The weights wi are derived from a therapist's pair-wise comparison using a modified RankSVM algorithm. We have applied this framework to evaluate upper limb movement for stroke survivors with excellent results—the evaluation results are highly correlated to the therapist's observations.

'CHEATS': a generic information communication technology (ICT) evaluation framework.

PubMed

Shaw, Nicola T

2002-05-01

This paper describes a generic framework for the evaluation of information communication technologies. This framework, CHEATS, utilises both qualitative and quantitative research methods and has proved appropriate in multiple clinical settings including telepsychiatry, teledermatology and teleeducation. The paper demonstrates how a multidisciplinary approach is essential when evaluating new and emerging technologies, particularly when such systems are implemented in real service as opposed to a research setting.

A Conceptual Framework for the Evaluation of Emergency Risk Communications.

PubMed

Savoia, Elena; Lin, Leesa; Gamhewage, Gaya M

2017-09-01

To articulate a conceptual framework in support of evaluation activities in emergency risk communications (ERC). The framework proposed is based on a systematic review of the scientific literature (2001-2016) combined with data derived from a series of semistructured interviews with experts and practitioners in ERC, and it is designed to support local, national, and international public health organizations in implementing evaluation studies in ERC. We identified a list of ERC outcomes from the full-text review of 152 articles and categorized these into 3 groups, depending upon the level at which the outcome was measured: (1) information environment, (2) population, and (3) public health system. We analyzed interviewees' data from 18 interviews to identify practices and processes related to the effectiveness of ERC and included these as key structural components and processes in the developed evaluation framework. Researchers and public health practitioners interested in the evaluation of ERC can use the conceptual framework described in this article to guide the development of evaluation studies and methods for assessing communication outcomes related to public health emergencies.

The MIXED framework: A novel approach to evaluating mixed-methods rigor.

PubMed

Eckhardt, Ann L; DeVon, Holli A

2017-10-01

Evaluation of rigor in mixed-methods (MM) research is a persistent challenge due to the combination of inconsistent philosophical paradigms, the use of multiple research methods which require different skill sets, and the need to combine research at different points in the research process. Researchers have proposed a variety of ways to thoroughly evaluate MM research, but each method fails to provide a framework that is useful for the consumer of research. In contrast, the MIXED framework is meant to bridge the gap between an academic exercise and practical assessment of a published work. The MIXED framework (methods, inference, expertise, evaluation, and design) borrows from previously published frameworks to create a useful tool for the evaluation of a published study. The MIXED framework uses an experimental eight-item scale that allows for comprehensive integrated assessment of MM rigor in published manuscripts. Mixed methods are becoming increasingly prevalent in nursing and healthcare research requiring researchers and consumers to address issues unique to MM such as evaluation of rigor. © 2017 John Wiley & Sons Ltd.

A multi-objective framework to predict flows of ungauged rivers within regions of sparse hydrometeorologic observation

NASA Astrophysics Data System (ADS)

Alipour, M.; Kibler, K. M.

2017-12-01

Despite advances in flow prediction, managers of ungauged rivers located within broad regions of sparse hydrometeorologic observation still lack prescriptive methods robust to the data challenges of such regions. We propose a multi-objective streamflow prediction framework for regions of minimum observation to select models that balance runoff efficiency with choice of accurate parameter values. We supplement sparse observed data with uncertain or low-resolution information incorporated as `soft' a priori parameter estimates. The performance of the proposed framework is tested against traditional single-objective and constrained single-objective calibrations in two catchments in a remote area of southwestern China. We find that the multi-objective approach performs well with respect to runoff efficiency in both catchments (NSE = 0.74 and 0.72), within the range of efficiencies returned by other models (NSE = 0.67 - 0.78). However, soil moisture capacity estimated by the multi-objective model resonates with a priori estimates (parameter residuals of 61 cm versus 289 and 518 cm for maximum soil moisture capacity in one catchment, and 20 cm versus 246 and 475 cm in the other; parameter residuals of 0.48 versus 0.65 and 0.7 for soil moisture distribution shape factor in one catchment, and 0.91 versus 0.79 and 1.24 in the other). Thus, optimization to a multi-criteria objective function led to very different representations of soil moisture capacity as compared to models selected by single-objective calibration, without compromising runoff efficiency. These different soil moisture representations may translate into considerably different hydrological behaviors. The proposed approach thus offers a preliminary step towards greater process understanding in regions of severe data limitations. For instance, the multi-objective framework may be an adept tool to discern between models of similar efficiency to select models that provide the "right answers for the right reasons

A Conceptual Framework Curriculum Evaluation Electrical Engineering Education

ERIC Educational Resources Information Center

Imansari, Nurulita; Sutadji, Eddy

2017-01-01

This evaluation is a conceptual framework that has been analyzed in the hope that can help research related an evaluation of the curriculum. The Model of evaluation used was CIPPO model. CIPPO Model consists of "context," "input," "process," "product," and "outcomes." On the dimension of the…

A framework for m-health service development and success evaluation.

PubMed

Sadegh, S Saeedeh; Khakshour Saadat, Parisa; Sepehri, Mohammad Mehdi; Assadi, Vahid

2018-04-01

The emergence of mobile technology has influenced many service industries including health care. Mobile health (m-Health) applications have been used widely, and many services have been developed that have changed delivery systems and have improved effectiveness of health care services. Stakeholders of m-Health services have various resources and rights that lends to a complexity in service delivery. In addition, abundance of different m-Health services makes it difficult to choose an appropriate service for these stakeholders that include customers, patients, users or even providers. Moreover, a comprehensive framework is not yet provided in the literature that would help manage and evaluate m-health services, considering various stakeholder's benefits. In this paper, a comprehensive literature review has been done on famous frameworks and models in the field of Information Technology and electronic health with the aim of finding different aspects of developing and managing m-health services. Using the results of literature review and conducting a stakeholder analysis, we have proposed an m-health evaluation framework which evaluates the success of a given m-health service through a three-stage life cycle: (1) Service Requirement Analysis, (2) Service Development, and (3) Service Delivery. Key factors of m-health evaluation in each step are introduced in the proposed framework considering m-health key stakeholder's benefits. The proposed framework is validated via expert interviews, and key factors in each evaluation step is validated using PLS model. Results show that path coefficients are higher than their threshold which supports the validity of proposed framework. Copyright © 2018 Elsevier B.V. All rights reserved.

An object-oriented framework for distributed hydrologic and geomorphic modeling using triangulated irregular networks

NASA Astrophysics Data System (ADS)

Tucker, Gregory E.; Lancaster, Stephen T.; Gasparini, Nicole M.; Bras, Rafael L.; Rybarczyk, Scott M.

2001-10-01

We describe a new set of data structures and algorithms for dynamic terrain modeling using a triangulated irregular network (TINs). The framework provides an efficient method for storing, accessing, and updating a Delaunay triangulation and its associated Voronoi diagram. The basic data structure consists of three interconnected data objects: triangles, nodes, and directed edges. Encapsulating each of these geometric elements within a data object makes it possible to essentially decouple the TIN representation from the modeling applications that make use of it. Both the triangulation and its corresponding Voronoi diagram can be rapidly retrieved or updated, making these methods well suited to adaptive remeshing schemes. We develop a set of algorithms for defining drainage networks and identifying closed depressions (e.g., lakes) for hydrologic and geomorphic modeling applications. We also outline simple numerical algorithms for solving network routing and 2D transport equations within the TIN framework. The methods are illustrated with two example applications, a landscape evolution model and a distributed rainfall-runoff model.

Evaluating health-promoting schools in Hong Kong: development of a framework.

PubMed

Lee, Albert; Cheng, Frances F K; St Leger, Lawry

2005-06-01

Health-promoting schools (HPS)/healthy schools have existed internationally for about 15 years. Yet there are few comprehensive evaluation frameworks available which enable the outcomes of HPS initiatives to be assessed. This paper identifies an evaluation framework developed in Hong Kong. The framework uses a range of approaches to explore what schools actually do in their health promotion and health education initiatives. The framework, which is based on the WHO (Western Pacific Regional Office) Guidelines for HPS, is described in detail. The appropriate instruments for data collection are described and their origins identified. The evaluation plan and protocol, which underpinned the very comprehensive evaluation in Hong Kong, are explained. Finally, a case is argued for evaluation of HPS to be more in line with the educational dynamics of schools and the research literature on effective schooling, rather than focusing primarily on health-related measures.

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework

PubMed Central

Zhou, Ronggang; Chan, Alan H. S.

2016-01-01

BACKGROUND: In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. OBJECTIVE: This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. METHODS: With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. RESULTS AND CONCLUSIONS: Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process. PMID:28035943

Towards a hierarchical optimization framework for spatially targeting incentive policies to promote green infrastructure amidst multiple objectives and uncertainty

EPA Science Inventory

We introduce a hierarchical optimization framework for spatially targeting green infrastructure (GI) incentive policies in order to meet objectives related to cost and environmental effectiveness. The framework explicitly simulates the interaction between multiple levels of polic...

Rubrics for Evaluating Open Education Resource (OER) Objects

ERIC Educational Resources Information Center

Achieve, Inc., 2011

2011-01-01

The rubrics presented in this report represent an evaluation system for objects found within Open Education Resources. An object could include images, applets, lessons, units, assessments and more. For the purpose of this evaluation, any component that can exist as a stand-alone qualifies as an object. The rubrics in this packet can be applied…

Web of Objects Based Ambient Assisted Living Framework for Emergency Psychiatric State Prediction

PubMed Central

Alam, Md Golam Rabiul; Abedin, Sarder Fakhrul; Al Ameen, Moshaddique; Hong, Choong Seon

2016-01-01

Ambient assisted living can facilitate optimum health and wellness by aiding physical, mental and social well-being. In this paper, patients’ psychiatric symptoms are collected through lightweight biosensors and web-based psychiatric screening scales in a smart home environment and then analyzed through machine learning algorithms to provide ambient intelligence in a psychiatric emergency. The psychiatric states are modeled through a Hidden Markov Model (HMM), and the model parameters are estimated using a Viterbi path counting and scalable Stochastic Variational Inference (SVI)-based training algorithm. The most likely psychiatric state sequence of the corresponding observation sequence is determined, and an emergency psychiatric state is predicted through the proposed algorithm. Moreover, to enable personalized psychiatric emergency care, a service a web of objects-based framework is proposed for a smart-home environment. In this framework, the biosensor observations and the psychiatric rating scales are objectified and virtualized in the web space. Then, the web of objects of sensor observations and psychiatric rating scores are used to assess the dweller’s mental health status and to predict an emergency psychiatric state. The proposed psychiatric state prediction algorithm reported 83.03 percent prediction accuracy in an empirical performance study. PMID:27608023

ITEM SELECTION TECHNIQUES AND EVALUATION OF INSTRUCTIONAL OBJECTIVES.

ERIC Educational Resources Information Center

COX, RICHARD C.

THE VALIDITY OF AN EDUCATIONAL ACHIEVEMENT TEST DEPENDS UPON THE CORRESPONDENCE BETWEEN SPECIFIED EDUCATIONAL OBJECTIVES AND THE EXTENT TO WHICH THESE OBJECTIVES ARE MEASURED BY THE EVALUATION INSTRUMENT. THIS STUDY IS DESIGNED TO EVALUATE THE EFFECT OF STATISTICAL ITEM SELECTION ON THE STRUCTURE OF THE FINAL EVALUATION INSTRUMENT AS COMPARED WITH…

A conceptual framework for evaluation of public health and primary care system performance in iran.

PubMed

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Akbari Sari, Ali; Mesdaghinia, Alireza

2015-01-26

The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report.

Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation

PubMed Central

2017-01-01

People are often aware of their mistakes, and report levels of confidence in their choices that correlate with objective performance. These metacognitive assessments of decision quality are important for the guidance of behavior, particularly when external feedback is absent or sporadic. However, a computational framework that accounts for both confidence and error detection is lacking. In addition, accounts of dissociations between performance and metacognition have often relied on ad hoc assumptions, precluding a unified account of intact and impaired self-evaluation. Here we present a general Bayesian framework in which self-evaluation is cast as a “second-order” inference on a coupled but distinct decision system, computationally equivalent to inferring the performance of another actor. Second-order computation may ensue whenever there is a separation between internal states supporting decisions and confidence estimates over space and/or time. We contrast second-order computation against simpler first-order models in which the same internal state supports both decisions and confidence estimates. Through simulations we show that second-order computation provides a unified account of different types of self-evaluation often considered in separate literatures, such as confidence and error detection, and generates novel predictions about the contribution of one’s own actions to metacognitive judgments. In addition, the model provides insight into why subjects’ metacognition may sometimes be better or worse than task performance. We suggest that second-order computation may underpin self-evaluative judgments across a range of domains. PMID:28004960

A Framework for Examining How Mathematics Teachers Evaluate Technology

ERIC Educational Resources Information Center

Smith, Ryan C.; Shin, Dongjo; Kim, Somin

2016-01-01

Our mathematics cognitive technology noticing framework is based on professional noticing and curricular noticing frameworks and data collected in a study that explored how secondary mathematics teachers evaluate technology. Our participants displayed three categories of noticing: attention to features of technology, interpretation of the…

A multi-resolution strategy for a multi-objective deformable image registration framework that accommodates large anatomical differences

NASA Astrophysics Data System (ADS)

Alderliesten, Tanja; Bosman, Peter A. N.; Sonke, Jan-Jakob; Bel, Arjan

2014-03-01

Currently, two major challenges dominate the field of deformable image registration. The first challenge is related to the tuning of the developed methods to specific problems (i.e. how to best combine different objectives such as similarity measure and transformation effort). This is one of the reasons why, despite significant progress, clinical implementation of such techniques has proven to be difficult. The second challenge is to account for large anatomical differences (e.g. large deformations, (dis)appearing structures) that occurred between image acquisitions. In this paper, we study a framework based on multi-objective optimization to improve registration robustness and to simplify tuning for specific applications. Within this framework we specifically consider the use of an advanced model-based evolutionary algorithm for optimization and a dual-dynamic transformation model (i.e. two "non-fixed" grids: one for the source- and one for the target image) to accommodate for large anatomical differences. The framework computes and presents multiple outcomes that represent efficient trade-offs between the different objectives (a so-called Pareto front). In image processing it is common practice, for reasons of robustness and accuracy, to use a multi-resolution strategy. This is, however, only well-established for single-objective registration methods. Here we describe how such a strategy can be realized for our multi-objective approach and compare its results with a single-resolution strategy. For this study we selected the case of prone-supine breast MRI registration. Results show that the well-known advantages of a multi-resolution strategy are successfully transferred to our multi-objective approach, resulting in superior (i.e. Pareto-dominating) outcomes.

A Technology Immune Technology Enabled Problem within an Action on Objects Framework: Stamping Functions

ERIC Educational Resources Information Center

Connell, Michael; Abramovich, Sergei

2017-01-01

This paper illustrates how the notion of Technology Immune Technology Enabled (TITE) problems (Abramovich, 2014), in this case an exploration of variations in surface area we refer to as Stamping Functions, might be incorporated into a K-6 mathematics methods class operating within an Action on Objects framework (Connell, 2001). TITE problems have…

Health Information Technology Evaluation Framework (HITREF) Comprehensiveness as Assessed in Electronic Point-of-Care Documentation Systems Evaluations.

PubMed

Sockolow, Paulina S; Bowles, Kathryn H; Rogers, Michelle

2015-01-01

We assessed the Health Information Technology (HIT) Reference-based Evaluation Framework (HITREF) comprehensiveness in two HIT evaluations in settings different from that in which the HITREF was developed. Clinician satisfaction themes that emerged from clinician interviews in the home care and the hospital studies were compared to the framework components. Across both studies, respondents commented on 12 of the 20 HITREF components within 5 of the 6 HITREF concepts. No new components emerged that were missing from the HITREF providing evidence that the HITREF is a comprehensive framework. HITREF use in a range of HIT evaluations by researchers new to the HITREF demonstrates that it can be used as intended. Therefore, we continue to recommend the HITREF as a comprehensive, research-based HIT evaluation framework to increase the capacity of informatics evaluators' use of best practice and evidence-based practice to support the credibility of their findings for fulfilling the purpose of program evaluation.

A conceptual framework for evaluating tsunami resilience

NASA Astrophysics Data System (ADS)

Pushpalal, Dinil

2017-02-01

As many coastal towns in the northeast coast of Japan were destroyed by tsunami accompanied with the Great East Japan Earthquake, a few of them were survived or little damaged with no or less casualties due to some reasons. Yoshihama in Iwate prefecture is one of such little damaged communities and is known as “Lucky Beach.” There were such “lucky” and “unlucky” regions in Indonesia and Sri Lanka too, which were affected by Indian Ocean Tsunami. Identification of reasons for vulnerability or resilience is the primary consideration of this article. It presents pragmatic conceptual framework for evaluating resilience, based on author’s firsthand experience on above both tsunamis. Integral resilience of a given area has been considered after dividing into three phases namely, onsite resilience, instantaneous survivability, and recovery potentiality of the area. The author assumes that capacity of each phase depends on socioeconomic, infrastructural and geographical factors of the area considered. The paper moves forward, arguing appropriateness of the framework by giving examples collected from Japan, Indonesia and Sri Lanka. The framework will be useful for evaluating resilience of coastal townships and also planning resilient townships, specifically focusing on tsunami.

Individual performance review in hospital practice: the development of a framework and evaluation of doctors' attitudes to its value and implementation.

PubMed

Trebble, T M; Cruickshank, L; Hockey, P M; Heyworth, N; Powell, T; Clarke, N

2013-11-01

Appraisal, or independent performance review (IPR) is used in human resources management in the commercial and public sectors to evaluate the performance of an employee against agreed local organisational expectations and objectives, and to identify their requirements for development and effective management. IPR for NHS consultants may provide essential information for job planning, contribute towards medical appraisal for revalidation, and facilitate productivity and quality improvement. To develop a framework for IPR for consultants, and to determine attitudes on its value, process and content. Information from commercial, public and voluntary sector models and published and other literature sources were used to develop an IPR framework. This was assessed through a three-cycle action research methodology involving qualitative interviews with 22 consultants (predominantly with medical management roles). The domains of the IPR framework included: (1) performance against objectives; (2) behaviour and leadership; (3) talent management; (4) agreed future objectives. A number of themes were identified from the consultant interviews including: ineffective current appraisal systems reflecting a lack of valid performance data and allotted time; a lack of empowerment of medical managers to address performance issues; IPR as a more explicit system, offering value in evaluating doctors performance; and the dependence of successful implementation on the engagement of the Trust executive. IPR may have value for performance evaluation of consultants, contributing toward job planning and complementing medical appraisal. Support by their employing organisation and engagement with medical managers in design and implementation is likely to be essential.

Proposal of a framework for evaluating military surveillance systems for early detection of outbreaks on duty areas

PubMed Central

Meynard, Jean-Baptiste; Chaudet, Herve; Green, Andrew D; Jefferson, Henry L; Texier, Gaetan; Webber, Daniel; Dupuy, Bruce; Boutin, Jean-Paul

2008-01-01

Background In recent years a wide variety of epidemiological surveillance systems have been developed to provide early identification of outbreaks of infectious disease. Each system has had its own strengths and weaknesses. In 2002 a Working Group of the Centers for Disease Control and Prevention (CDC) produced a framework for evaluation, which proved suitable for many public health surveillance systems. However this did not easily adapt to the military setting, where by necessity a variety of different parameters are assessed, different constraints placed on the systems, and different objectives required. This paper describes a proposed framework for evaluation of military syndromic surveillance systems designed to detect outbreaks of disease on operational deployments. Methods The new framework described in this paper was developed from the cumulative experience of British and French military syndromic surveillance systems. The methods included a general assessment framework (CDC), followed by more specific methods of conducting evaluation. These included Knowledge/Attitude/Practice surveys (KAP surveys), technical audits, ergonomic studies, simulations and multi-national exercises. A variety of military constraints required integration into the evaluation. Examples of these include the variability of geographical conditions in the field, deployment to areas without prior knowledge of naturally-occurring disease patterns, the differences in field sanitation between locations and over the length of deployment, the mobility of military forces, turnover of personnel, continuity of surveillance across different locations, integration with surveillance systems from other nations working alongside each other, compatibility with non-medical information systems, and security. Results A framework for evaluation has been developed that can be used for military surveillance systems in a staged manner consisting of initial, intermediate and final evaluations. For each stage

The Utility of the Memorable Messages Framework as an Intermediary Evaluation Tool for Fruit and Vegetable Consumption in a Nutrition Education Program

ERIC Educational Resources Information Center

Davis, LaShara A.; Morgan, Susan E.; Mobley, Amy R.

2016-01-01

Additional strategies to evaluate the impact of community nutrition education programs on low-income individuals are needed. The objective of this qualitative study was to examine the use of the Memorable Messages Framework as an intermediary nutrition education program evaluation tool to determine what fruit and vegetable messages were reported…

Beyond Effectiveness: A Pragmatic Evaluation Framework for Learning and Continuous Quality Improvement of e-Learning Interventions in Healthcare.

PubMed

Dafalla, Tarig Dafalla Mohamed; Kushniruk, Andre W; Borycki, Elizabeth M

2015-01-01

A pragmatic evaluation framework for evaluating the usability and usefulness of an e-learning intervention for a patient clinical information scheduling system is presented in this paper. The framework was conceptualized based on two different but related concepts (usability and usefulness) and selection of appropriate and valid methods of data collection and analysis that included: (1) Low-Cost Rapid Usability Engineering (LCRUE), (2) Cognitive Task Analysis (CTA), (3) Heuristic Evaluation (HE) criteria for web-based learning, and (4) Software Usability Measurement Inventory (SUMI). The results of the analysis showed some areas where usability that were related to General Interface Usability (GIU), instructional design and content was problematic; some of which might account for the poorly rated aspects of usability when subjectively measured. This paper shows that using a pragmatic framework can be a useful way, not only for measuring the usability and usefulness, but also for providing a practical objective evidences for learning and continuous quality improvement of e-learning systems. The findings should be of interest to educators, developers, designers, researchers, and usability practitioners involved in the development of e-learning systems in healthcare. This framework could be an appropriate method for assessing the usability, usefulness and safety of health information systems both in the laboratory and in the clinical context.

Integrated corridor management initiative : demonstration phase evaluation - final national evaluation framework.

DOT National Transportation Integrated Search

2012-05-01

This report provides an analytical framework for evaluating the two field deployments under the United States Department of Transportation (U.S. DOT) Integrated Corridor Management (ICM) Initiative Demonstration Phase. The San Diego Interstate 15 cor...

Case study of evaluations that go beyond clinical outcomes to assess quality improvement diabetes programmes using the Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE)

PubMed Central

Harris, Stewart B.; Naqshbandi Hayward, Mariam; Tompkins, Jordan W.

2016-01-01

Abstract Rationale, aims and objectives Investments in efforts to reduce the burden of diabetes on patients and health care are critical; however, more evaluation is needed to provide evidence that informs and supports future policies and programmes. The newly developed Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE) incorporates the theoretical concepts needed to facilitate the capture of critical information to guide investments, policy and programmatic decision making. The aim of the study is to assess the applicability and value of DEFINE in comprehensive real‐world evaluation. Method Using a critical and positivist approach, this intrinsic and collective case study retrospectively examines two naturalistic evaluations to demonstrate how DEFINE could be used when conducting real‐world comprehensive evaluations in health care settings. Results The variability between the cases and the evaluation designs are described and aligned to the DEFINE goals, steps and sub‐steps. The majority of the theoretical steps of DEFINE were exemplified in both cases, although limited for knowledge translation efforts. Application of DEFINE to evaluate diverse programmes that target various chronic diseases is needed to further test the inclusivity and built‐in flexibility of DEFINE and its role in encouraging more comprehensive knowledge translation. Conclusions This case study shows how DEFINE could be used to structure or guide comprehensive evaluations of programmes and initiatives implemented in health care settings and support scale‐up of successful innovations. Future use of the framework will continue to strengthen its value in guiding programme evaluation and informing health policy to reduce the burden of diabetes and other chronic diseases. PMID:26804339

An Online Social Constructivist Course: Toward a Framework for Usability Evaluations

ERIC Educational Resources Information Center

Phillips, Alana S.; Sheffield, Anneliese; Moore, Michelle; Robinson, Heather A.

2016-01-01

There is a need for a holistic usability evaluation framework that accommodates social constructivist online courses. Social knowledge construction may not be adequately evaluated using current frameworks. This qualitative research study examined the usability needs of a social constructivist online course. Data from an online course were analyzed…

A Concise and Practical Framework for the Development and Usability Evaluation of Patient Information Websites

PubMed Central

Knijnenburg, S.L.; Kremer, L.C.; Jaspers, M.W.M.

2015-01-01

Summary Background The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS). Objective To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population. Methods The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA. Results The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content. Conclusion Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations. PMID:26171083

Action semantics: A unifying conceptual framework for the selective use of multimodal and modality-specific object knowledge

NASA Astrophysics Data System (ADS)

van Elk, Michiel; van Schie, Hein; Bekkering, Harold

2014-06-01

Our capacity to use tools and objects is often considered one of the hallmarks of the human species. Many objects greatly extend our bodily capabilities to act in the physical world, such as when using a hammer or a saw. In addition, humans have the remarkable capability to use objects in a flexible fashion and to combine multiple objects in complex actions. We prepare coffee, cook dinner and drive our car. In this review we propose that humans have developed declarative and procedural knowledge, i.e. action semantics that enables us to use objects in a meaningful way. A state-of-the-art review of research on object use is provided, involving behavioral, developmental, neuropsychological and neuroimaging studies. We show that research in each of these domains is characterized by similar discussions regarding (1) the role of object affordances, (2) the relation between goals and means in object use and (3) the functional and neural organization of action semantics. We propose a novel conceptual framework of action semantics to address these issues and to integrate the previous findings. We argue that action semantics entails both multimodal object representations and modality-specific sub-systems, involving manipulation knowledge, functional knowledge and representations of the sensory and proprioceptive consequences of object use. Furthermore, we argue that action semantics are hierarchically organized and selectively activated and used depending on the action intention of the actor and the current task context. Our framework presents an integrative account of multiple findings and perspectives on object use that may guide future studies in this interdisciplinary domain.

Towards a Framework for Evaluating Mobile Mental Health Apps.

PubMed

Chan, Steven; Torous, John; Hinton, Ladson; Yellowlees, Peter

2015-12-01

Mobile phones are ubiquitous in society and owned by a majority of psychiatric patients, including those with severe mental illness. Their versatility as a platform can extend mental health services in the areas of communication, self-monitoring, self-management, diagnosis, and treatment. However, the efficacy and reliability of publicly available applications (apps) have yet to be demonstrated. Numerous articles have noted the need for rigorous evaluation of the efficacy and clinical utility of smartphone apps, which are largely unregulated. Professional clinical organizations do not provide guidelines for evaluating mobile apps. Guidelines and frameworks are needed to evaluate medical apps. Numerous frameworks and evaluation criteria exist from the engineering and informatics literature, as well as interdisciplinary organizations in similar fields such as telemedicine and healthcare informatics. We propose criteria for both patients and providers to use in assessing not just smartphone apps, but also wearable devices and smartwatch apps for mental health. Apps can be evaluated by their usefulness, usability, and integration and infrastructure. Apps can be categorized by their usability in one or more stages of a mental health provider's workflow. Ultimately, leadership is needed to develop a framework for describing apps, and guidelines are needed for both patients and mental health providers.

A Conceptual Framework for Evaluation of Public Health and Primary Care System Performance in Iran

PubMed Central

Jahanmehr, Nader; Rashidian, Arash; Khosravi, Ardeshir; Farzadfar, Farshad; Shariati, Mohammad; Majdzadeh, Reza; Sari, Ali Akbari; Mesdaghinia, Alireza

2015-01-01

Introduction: The main objective of this study was to design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial public health and primary care performance and to assess their share in the overall health impacts of the community. Methods: We used several tools and techniques, including system thinking, literature review to identify relevant attributes of health system performance framework and interview with the key stakeholders. The PubMed, Scopus, web of science, Google Scholar and two specialized databases of Persian language literature (IranMedex and SID) were searched using main terms and keywords. Following decision-making and collective agreement among the different stakeholders, 51 core indicators were chosen from among 602 obtained indicators in a four stage process, for monitoring and evaluation of Health Deputies. Results: We proposed a conceptual framework by identifying the performance area for Health Deputies between other determinants of health, as well as introducing a chain of results, for performance, consisting of Input, Process, Output and Outcome indicators. We also proposed 5 dimensions for measuring the performance of Health Deputies, consisting of efficiency, effectiveness, equity, access and improvement of health status. Conclusion: The proposed Conceptual Framework illustrates clearly the Health Deputies success in achieving best results and consequences of health in the country. Having the relative commitment of the ministry of health and Health Deputies at the University of Medical Sciences is essential for full implementation of this framework and providing the annual performance report. PMID:25946937

Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation.

PubMed

Fleming, Stephen M; Daw, Nathaniel D

2017-01-01

People are often aware of their mistakes, and report levels of confidence in their choices that correlate with objective performance. These metacognitive assessments of decision quality are important for the guidance of behavior, particularly when external feedback is absent or sporadic. However, a computational framework that accounts for both confidence and error detection is lacking. In addition, accounts of dissociations between performance and metacognition have often relied on ad hoc assumptions, precluding a unified account of intact and impaired self-evaluation. Here we present a general Bayesian framework in which self-evaluation is cast as a "second-order" inference on a coupled but distinct decision system, computationally equivalent to inferring the performance of another actor. Second-order computation may ensue whenever there is a separation between internal states supporting decisions and confidence estimates over space and/or time. We contrast second-order computation against simpler first-order models in which the same internal state supports both decisions and confidence estimates. Through simulations we show that second-order computation provides a unified account of different types of self-evaluation often considered in separate literatures, such as confidence and error detection, and generates novel predictions about the contribution of one's own actions to metacognitive judgments. In addition, the model provides insight into why subjects' metacognition may sometimes be better or worse than task performance. We suggest that second-order computation may underpin self-evaluative judgments across a range of domains. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Evaluating Frameworks That Provide Value Measures for Health Care Interventions.

PubMed

Mandelblatt, Jeanne S; Ramsey, Scott D; Lieu, Tracy A; Phelps, Charles E

2017-02-01

The recent acceleration of scientific discovery has led to greater choices in health care. New technologies, diagnostic tests, and pharmaceuticals have widely varying impact on patients and populations in terms of benefits, toxicities, and costs, stimulating a resurgence of interest in the creation of frameworks intended to measure value in health. Many of these are offered by providers and/or advocacy organizations with expertise and interest in specific diseases (e.g., cancer and heart disease). To help assess the utility of and the potential biases embedded in these frameworks, we created an evaluation taxonomy with seven basic components: 1) define the purpose; 2) detail the conceptual approach, including perspectives, methods for obtaining preferences of decision makers (e.g., patients), and ability to incorporate multiple dimensions of value; 3) discuss inclusions and exclusions of elements included in the framework, and whether the framework assumes clinical intervention or offers alternatives such as palliative care or watchful waiting; 4) evaluate data sources and their scientific validity; 5) assess the intervention's effect on total costs of treating a defined population; 6) analyze how uncertainty is incorporated; and 7) illuminate possible conflicts of interest among those creating the framework. We apply the taxonomy to four representative value frameworks recently published by professional organizations focused on treatment of cancer and heart disease and on vaccine use. We conclude that each of these efforts has strengths and weaknesses when evaluated using our taxonomy, and suggest pathways to enhance the utility of value-assessing frameworks for policy and clinical decision making. Copyright © 2017 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Action semantics: A unifying conceptual framework for the selective use of multimodal and modality-specific object knowledge.

PubMed

van Elk, Michiel; van Schie, Hein; Bekkering, Harold

2014-06-01

Our capacity to use tools and objects is often considered one of the hallmarks of the human species. Many objects greatly extend our bodily capabilities to act in the physical world, such as when using a hammer or a saw. In addition, humans have the remarkable capability to use objects in a flexible fashion and to combine multiple objects in complex actions. We prepare coffee, cook dinner and drive our car. In this review we propose that humans have developed declarative and procedural knowledge, i.e. action semantics that enables us to use objects in a meaningful way. A state-of-the-art review of research on object use is provided, involving behavioral, developmental, neuropsychological and neuroimaging studies. We show that research in each of these domains is characterized by similar discussions regarding (1) the role of object affordances, (2) the relation between goals and means in object use and (3) the functional and neural organization of action semantics. We propose a novel conceptual framework of action semantics to address these issues and to integrate the previous findings. We argue that action semantics entails both multimodal object representations and modality-specific sub-systems, involving manipulation knowledge, functional knowledge and representations of the sensory and proprioceptive consequences of object use. Furthermore, we argue that action semantics are hierarchically organized and selectively activated and used depending on the action intention of the actor and the current task context. Our framework presents an integrative account of multiple findings and perspectives on object use that may guide future studies in this interdisciplinary domain. Copyright © 2013 Elsevier B.V. All rights reserved.

Evaluating the Impact of Educational Interventions on Patients and Communities: A Conceptual Framework.

PubMed

Bzowyckyj, Andrew S; Dow, Alan; Knab, Mary S

2017-11-01

Health professions education programs can have direct effects on patients and communities as well as on learners. However, few studies have examined the patient and community outcomes of educational interventions. To better integrate education and health care delivery, educators and researchers would benefit from a unifying framework to guide the planning of educational interventions and evaluation of their impact on patients.The authors of this Perspective mirrored approaches from Miller's pyramid of educational assessment and Moore and colleagues' framework for evaluating continuing professional development to propose a conceptual framework for evaluating the impact of educational interventions on patients and communities. This proposed framework, which complements these existing frameworks for evaluating the impact of educational interventions on learners, includes four levels: (1) interaction; (2) acceptability; (3) individual outcomes (i.e., knowledge, skills, activation, behaviors, and individual health indicators); and (4) population outcomes (i.e., community health indicators, capacity, and disparities). The authors describe measures and outcomes at each level and provide an example of the application of their new conceptual framework.The authors encourage educators and researchers to use this conceptual framework to evaluate the impact of educational interventions on patients and to more clearly identify and define which educational interventions strengthen communities and enhance overall health outcomes.

Toward a multi-objective decision support framework to support regulations of unconventional oil and gas development

NASA Astrophysics Data System (ADS)

Alongi, M.; Howard, C.; Kasprzyk, J. R.; Ryan, J. N.

2015-12-01

Unconventional oil and gas development (UOGD) using hydraulic fracturing and horizontal drilling has recently fostered an unprecedented acceleration in energy development. Regulations seek to protect environmental quality of areas surrounding UOGD, while maintaining economic benefits. One such regulation is a setback distance, which dictates the minimum proximity between an oil and gas well and an object such as a residential or commercial building, property line, or water source. In general, most setback regulations have been strongly politically motivated without a clear scientific basis for understanding the relationship between the setback distance and various performance outcomes. This presentation discusses a new decision support framework for setback regulations, as part of a large NSF-funded sustainability research network (SRN) on UOGD. The goal of the decision support framework is to integrate a wide array of scientific information from the SRN into a coherent framework that can help inform policy regarding UOGD. The decision support framework employs multiobjective evolutionary algorithm (MOEA) optimization coupled with simulation models of air quality and other performance-based outcomes on UOGD. The result of the MOEA optimization runs are quantitative tradeoff curves among different objectives. For example, one such curve could demonstrate air pollution concentrations versus estimates of energy development profits, for different levels of setback distance. Our results will also inform policy-relevant discussions surrounding UOGD such as comparing single- and multi-well pads, as well as regulations on the density of well development over a spatial area.

A framework for evaluating and utilizing medical terminology mappings.

PubMed

Hussain, Sajjad; Sun, Hong; Sinaci, Anil; Erturkmen, Gokce Banu Laleci; Mead, Charles; Gray, Alasdair J G; McGuinness, Deborah L; Prud'Hommeaux, Eric; Daniel, Christel; Forsberg, Kerstin

2014-01-01

Use of medical terminologies and mappings across them are considered to be crucial pre-requisites for achieving interoperable eHealth applications. Built upon the outcomes of several research projects, we introduce a framework for evaluating and utilizing terminology mappings that offers a platform for i) performing various mappings strategies, ii) representing terminology mappings together with their provenance information, and iii) enabling terminology reasoning for inferring both new and erroneous mappings. We present the results of the introduced framework from SALUS project where we evaluated the quality of both existing and inferred terminology mappings among standard terminologies.

A Collaborative Evaluation Framework for Biometric Connected Devices in Healthcare.

PubMed

Farnia, Troskah; Jaulent, Marie Christine; Marchand, Guillaume; Yasini, Mobin

2017-01-01

A large number of biometric connected devices are currently available with a variety of designs. Healthcare users cannot easily choose the reliable ones that correspond the best to their healthcare problems. The existing evaluation methods do not consider at the same time aspects of connectivity and healthcare usage. In this study, a collaborative evaluation framework for biometric connected devices in healthcare usage is proposed. This framework contains six dimensions: medical validity, technical reliability, usability, ergonomy, legal compliance, and accuracy of measurements. In a first step, these dimensions were assessed by designing a self administered questionnaire answered by the stakeholders (patients, health professionals, payers, and manufacturers). A case study was then carried out in a second step to test this framework in a project of telemonitoring for heart failure patients. The results are in favor of the efficiency of the proposed framework as a decision making tool in healthcare usage.

Matrix evaluation of science objectives

NASA Technical Reports Server (NTRS)

Wessen, Randii R.

1994-01-01

The most fundamental objective of all robotic planetary spacecraft is to return science data. To accomplish this, a spacecraft is fabricated and built, software is planned and coded, and a ground system is designed and implemented. However, the quantitative analysis required to determine how the collection of science data drives ground system capabilities has received very little attention. This paper defines a process by which science objectives can be quantitatively evaluated. By applying it to the Cassini Mission to Saturn, this paper further illustrates the power of this technique. The results show which science objectives drive specific ground system capabilities. In addition, this process can assist system engineers and scientists in the selection of the science payload during pre-project mission planning; ground system designers during ground system development and implementation; and operations personnel during mission operations.

California Curriculum Frameworks: A Handbook for Production, Implementation, and Evaluation Activities.

ERIC Educational Resources Information Center

California State Dept. of Education, Sacramento.

This booklet describes the characteristics and role of curriculum frameworks and describes how they can be used in developing educational programs. It is designed as a guide for writers of frameworks, for educators who are responsible for implementing frameworks, or for evaluators of educational programs. It provides a concise description of the…

The Sloan-C Pillars and Boundary Objects As a Framework for Evaluating Blended Learning

ERIC Educational Resources Information Center

Laumakis, Mark; Graham, Charles; Dziuban, Chuck

2009-01-01

The authors contend that blended learning represents a boundary object; a construct that brings together constituencies from a variety of backgrounds with each of these cohorts defining the object somewhat differently. The Sloan-C Pillars (learning effectiveness, access, cost effectiveness, student satisfaction, and faculty satisfaction) provide…

A Conceptual Framework for Graduate Teaching Assistant Professional Development Evaluation and Research

PubMed Central

Reeves, Todd D.; Marbach-Ad, Gili; Miller, Kristen R.; Ridgway, Judith; Gardner, Grant E.; Schussler, Elisabeth E.; Wischusen, E. William

2016-01-01

Biology graduate teaching assistants (GTAs) are significant contributors to the educational mission of universities, particularly in introductory courses, yet there is a lack of empirical data on how to best prepare them for their teaching roles. This essay proposes a conceptual framework for biology GTA teaching professional development (TPD) program evaluation and research with three overarching variable categories for consideration: outcome variables, contextual variables, and moderating variables. The framework’s outcome variables go beyond GTA satisfaction and instead position GTA cognition, GTA teaching practice, and undergraduate learning outcomes as the foci of GTA TPD evaluation and research. For each GTA TPD outcome variable, key evaluation questions and example assessment instruments are introduced to demonstrate how the framework can be used to guide GTA TPD evaluation and research plans. A common conceptual framework is also essential to coordinating the collection and synthesis of empirical data on GTA TPD nationally. Thus, the proposed conceptual framework serves as both a guide for conducting GTA TPD evaluation at single institutions and as a means to coordinate research across institutions at a national level. PMID:27193291

Evaluation of the UNREST questionnaire for testing the social resistance framework.

PubMed

Factor, Roni; Kawachi, Ichiro; Williams, David R

2013-07-01

The recently developed social resistance framework addresses a widespread pattern whereby non-dominant minority groups, such as ethnic/racial minorities and people of low socioeconomic status, often engage in unhealthy and risky behaviours at higher rates compared with society at large. The framework suggests that power relations within society may encourage members of non-dominant minority groups to actively engage in acts of everyday resistance, which may include risky and unhealthy behaviours. The current paper develops and psychometrically evaluates a research tool to test this innovative framework. The UNREST questionnaire measures the key concepts of the framework, along with four high-risk and unhealthy behaviours, as well as demographic and socioeconomic characteristics. A pilot survey was conducted among representative subsamples of a non-dominant group (African-Americans) and a dominant group (Caucasians). Consistent with the general premises of the framework, the evaluation of the questionnaire produced six valid and reliable scales, which were significantly correlated with some criterion-related items as well as unhealthy and risky behaviours. The preliminary results of our pilot study suggest that the new tool may be useful for testing the framework. The results also provide support for the framework in general.

Subjective vs objective evaluations of smile esthetics.

PubMed

Schabel, Brian J; Franchi, Lorenzo; Baccetti, Tiziano; McNamara, James A

2009-04-01

The aim of this study was to analyze the relationships between subjective evaluations of posttreatment smiles captured with clinical photography and rated by a panel of orthodontists and parents of orthodontic patients, and objective evaluations of the same smiles from the Smile Mesh program (TDG Computing, Philadelphia, Pa). The clinical photographs of 48 orthodontically treated patients were rated by a panel of 25 experienced orthodontists and 20 parents of patients. Independent samples t tests were used to test whether objective measurements were significantly different between subjects with "attractive" and "unattractive" smiles, and those with the "most attractive" and "least attractive" smiles. Additionally, logistic regression was performed to evaluate whether the measurements could predict whether a smile captured with clinical photography would be attractive or unattractive. The comparison between groups showed no significant differences for any measurement. Subjects with the "most unattractive" smiles had a significantly greater distance between the incisal edge of the maxillary central incisors and the lower lip during smiling, and a significantly smaller smile index than did those with the "most attractive" smiles. As shown by the coefficients of logistic regression, smile attractiveness could not be predicted by any objectively gathered measurement. No objective measure of the smile could predict attractive or unattractive smiles as judged subjectively.

The conceptual framework of the International Tobacco Control (ITC) Policy Evaluation Project

PubMed Central

Fong, G T; Cummings, K M; Borland, R; Hastings, G; Hyland, A; Giovino, G A; Hammond, D; Thompson, M E

2006-01-01

This paper describes the conceptual model that underlies the International Tobacco Control Policy Evaluation Project (ITC Project), whose mission is to measure the psychosocial and behavioural impact of key policies of the Framework Convention on Tobacco Control (FCTC) among adult smokers, and in some countries, among adult non‐smokers and among youth. The evaluation framework utilises multiple country controls, a longitudinal design, and a pre‐specified, theory‐driven conceptual model to test hypotheses about the anticipated effects of specific policies. The ITC Project consists of parallel prospective cohort surveys of representative samples of adult smokers currently in nine countries (inhabited by over 45% of the world's smokers), with other countries being added in the future. Collectively, the ITC Surveys constitute the first‐ever international cohort study of tobacco use. The conceptual model of the ITC Project draws on the psychosocial and health communication literature and assumes that tobacco control policies influence tobacco related behaviours through a causal chain of psychological events, with some variables more closely related to the policy itself (policy‐specific variables) and other variables that are more downstream from the policy, which have been identified by health behaviour and social psychological theories as being important causal precursors of behaviour (psychosocial mediators). We discuss the objectives of the ITC Project and its potential for building the evidence base for the FCTC. PMID:16754944

Improvement of the R-SWAT-FME framework to support multiple variables and multi-objective functions

USGS Publications Warehouse

Wu, Yiping; Liu, Shu-Guang

2014-01-01

Application of numerical models is a common practice in the environmental field for investigation and prediction of natural and anthropogenic processes. However, process knowledge, parameter identifiability, sensitivity, and uncertainty analyses are still a challenge for large and complex mathematical models such as the hydrological/water quality model, Soil and Water Assessment Tool (SWAT). In this study, the previously developed R program language-SWAT-Flexible Modeling Environment (R-SWAT-FME) was improved to support multiple model variables and objectives at multiple time steps (i.e., daily, monthly, and annually). This expansion is significant because there is usually more than one variable (e.g., water, nutrients, and pesticides) of interest for environmental models like SWAT. To further facilitate its easy use, we also simplified its application requirements without compromising its merits, such as the user-friendly interface. To evaluate the performance of the improved framework, we used a case study focusing on both streamflow and nitrate nitrogen in the Upper Iowa River Basin (above Marengo) in the United States. Results indicated that the R-SWAT-FME performs well and is comparable to the built-in auto-calibration tool in multi-objective model calibration. Overall, the enhanced R-SWAT-FME can be useful for the SWAT community, and the methods we used can also be valuable for wrapping potential R packages with other environmental models.

ESDORA: A Data Archive Infrastructure Using Digital Object Model and Open Source Frameworks

NASA Astrophysics Data System (ADS)

Shrestha, Biva; Pan, Jerry; Green, Jim; Palanisamy, Giriprakash; Wei, Yaxing; Lenhardt, W.; Cook, R. Bob; Wilson, B. E.; Leggott, M.

2011-12-01

There are an array of challenges associated with preserving, managing, and using contemporary scientific data. Large volume, multiple formats and data services, and the lack of a coherent mechanism for metadata/data management are some of the common issues across data centers. It is often difficult to preserve the data history and lineage information, along with other descriptive metadata, hindering the true science value for the archived data products. In this project, we use digital object abstraction architecture as the information/knowledge framework to address these challenges. We have used the following open-source frameworks: Fedora-Commons Repository, Drupal Content Management System, Islandora (Drupal Module) and Apache Solr Search Engine. The system is an active archive infrastructure for Earth Science data resources, which include ingestion, archiving, distribution, and discovery functionalities. We use an ingestion workflow to ingest the data and metadata, where many different aspects of data descriptions (including structured and non-structured metadata) are reviewed. The data and metadata are published after reviewing multiple times. They are staged during the reviewing phase. Each digital object is encoded in XML for long-term preservation of the content and relations among the digital items. The software architecture provides a flexible, modularized framework for adding pluggable user-oriented functionality. Solr is used to enable word search as well as faceted search. A home grown spatial search module is plugged in to allow user to make a spatial selection in a map view. A RDF semantic store within the Fedora-Commons Repository is used for storing information on data lineage, dissemination services, and text-based metadata. We use the semantic notion "isViewerFor" to register internally or externally referenced URLs, which are rendered within the same web browser when possible. With appropriate mapping of content into digital objects, many

The DONE framework: Creation, evaluation, and updating of an interdisciplinary, dynamic framework 2.0 of determinants of nutrition and eating

PubMed Central

Stok, F. Marijn; Hoffmann, Stefan; Volkert, Dorothee; Boeing, Heiner; Ensenauer, Regina; Stelmach-Mardas, Marta; Kiesswetter, Eva; Weber, Alisa; Rohm, Harald; Lien, Nanna; Brug, Johannes; Holdsworth, Michelle; Renner, Britta

2017-01-01

The question of which factors drive human eating and nutrition is a key issue in many branches of science. We describe the creation, evaluation, and updating of an interdisciplinary, interactive, and evolving “framework 2.0” of Determinants Of Nutrition and Eating (DONE). The DONE framework was created by an interdisciplinary workgroup in a multiphase, multimethod process. Modifiability, relationship strength, and population-level effect of the determinants were rated to identify areas of priority for research and interventions. External experts positively evaluated the usefulness, comprehensiveness, and quality of the DONE framework. An approach to continue updating the framework with the help of experts was piloted. The DONE framework can be freely accessed (http://uni-konstanz.de/DONE) and used in a highly flexible manner: determinants can be sorted, filtered and visualized for both very specific research questions as well as more general queries. The dynamic nature of the framework allows it to evolve as experts can continually add new determinants and ratings. We anticipate this framework will be useful for research prioritization and intervention development. PMID:28152005

The DONE framework: Creation, evaluation, and updating of an interdisciplinary, dynamic framework 2.0 of determinants of nutrition and eating.

PubMed

Stok, F Marijn; Hoffmann, Stefan; Volkert, Dorothee; Boeing, Heiner; Ensenauer, Regina; Stelmach-Mardas, Marta; Kiesswetter, Eva; Weber, Alisa; Rohm, Harald; Lien, Nanna; Brug, Johannes; Holdsworth, Michelle; Renner, Britta

2017-01-01

The question of which factors drive human eating and nutrition is a key issue in many branches of science. We describe the creation, evaluation, and updating of an interdisciplinary, interactive, and evolving "framework 2.0" of Determinants Of Nutrition and Eating (DONE). The DONE framework was created by an interdisciplinary workgroup in a multiphase, multimethod process. Modifiability, relationship strength, and population-level effect of the determinants were rated to identify areas of priority for research and interventions. External experts positively evaluated the usefulness, comprehensiveness, and quality of the DONE framework. An approach to continue updating the framework with the help of experts was piloted. The DONE framework can be freely accessed (http://uni-konstanz.de/DONE) and used in a highly flexible manner: determinants can be sorted, filtered and visualized for both very specific research questions as well as more general queries. The dynamic nature of the framework allows it to evolve as experts can continually add new determinants and ratings. We anticipate this framework will be useful for research prioritization and intervention development.

Long-term Science Data Curation Using a Digital Object Model and Open-Source Frameworks

NASA Astrophysics Data System (ADS)

Pan, J.; Lenhardt, W.; Wilson, B. E.; Palanisamy, G.; Cook, R. B.

2010-12-01

Scientific digital content, including Earth Science observations and model output, has become more heterogeneous in format and more distributed across the Internet. In addition, data and metadata are becoming necessarily linked internally and externally on the Web. As a result, such content has become more difficult for providers to manage and preserve and for users to locate, understand, and consume. Specifically, it is increasingly harder to deliver relevant metadata and data processing lineage information along with the actual content consistently. Readme files, data quality information, production provenance, and other descriptive metadata are often separated in the storage level as well as in the data search and retrieval interfaces available to a user. Critical archival metadata, such as auditing trails and integrity checks, are often even more difficult for users to access, if they exist at all. We investigate the use of several open-source software frameworks to address these challenges. We use Fedora Commons Framework and its digital object abstraction as the repository, Drupal CMS as the user-interface, and the Islandora module as the connector from Drupal to Fedora Repository. With the digital object model, metadata of data description and data provenance can be associated with data content in a formal manner, so are external references and other arbitrary auxiliary information. Changes are formally audited on an object, and digital contents are versioned and have checksums automatically computed. Further, relationships among objects are formally expressed with RDF triples. Data replication, recovery, metadata export are supported with standard protocols, such as OAI-PMH. We provide a tentative comparative analysis of the chosen software stack with the Open Archival Information System (OAIS) reference model, along with our initial results with the existing terrestrial ecology data collections at NASA’s ORNL Distributed Active Archive Center for

An interactive framework for acquiring vision models of 3-D objects from 2-D images.

PubMed

Motai, Yuichi; Kak, Avinash

2004-02-01

This paper presents a human-computer interaction (HCI) framework for building vision models of three-dimensional (3-D) objects from their two-dimensional (2-D) images. Our framework is based on two guiding principles of HCI: 1) provide the human with as much visual assistance as possible to help the human make a correct input; and 2) verify each input provided by the human for its consistency with the inputs previously provided. For example, when stereo correspondence information is elicited from a human, his/her job is facilitated by superimposing epipolar lines on the images. Although that reduces the possibility of error in the human marked correspondences, such errors are not entirely eliminated because there can be multiple candidate points close together for complex objects. For another example, when pose-to-pose correspondence is sought from a human, his/her job is made easier by allowing the human to rotate the partial model constructed in the previous pose in relation to the partial model for the current pose. While this facility reduces the incidence of human-supplied pose-to-pose correspondence errors, such errors cannot be eliminated entirely because of confusion created when multiple candidate features exist close together. Each input provided by the human is therefore checked against the previous inputs by invoking situation-specific constraints. Different types of constraints (and different human-computer interaction protocols) are needed for the extraction of polygonal features and for the extraction of curved features. We will show results on both polygonal objects and object containing curved features.

A Framework for the Evaluation of CASE Tool Learnability in Educational Environments

ERIC Educational Resources Information Center

Senapathi, Mali

2005-01-01

The aim of the research is to derive a framework for the evaluation of Computer Aided Software Engineering (CASE) tool learnability in educational environments. Drawing from the literature of Human Computer Interaction and educational research, a framework for evaluating CASE tool learnability in educational environments is derived. The two main…

Proposed evaluation framework for assessing operator performance with multisensor displays

NASA Technical Reports Server (NTRS)

Foyle, David C.

1992-01-01

Despite aggressive work on the development of sensor fusion algorithms and techniques, no formal evaluation procedures have been proposed. Based on existing integration models in the literature, an evaluation framework is developed to assess an operator's ability to use multisensor, or sensor fusion, displays. The proposed evaluation framework for evaluating the operator's ability to use such systems is a normative approach: The operator's performance with the sensor fusion display can be compared to the models' predictions based on the operator's performance when viewing the original sensor displays prior to fusion. This allows for the determination as to when a sensor fusion system leads to: 1) poorer performance than one of the original sensor displays (clearly an undesirable system in which the fused sensor system causes some distortion or interference); 2) better performance than with either single sensor system alone, but at a sub-optimal (compared to the model predictions) level; 3) optimal performance (compared to model predictions); or, 4) super-optimal performance, which may occur if the operator were able to use some highly diagnostic 'emergent features' in the sensor fusion display, which were unavailable in the original sensor displays. An experiment demonstrating the usefulness of the proposed evaluation framework is discussed.

The ventral visual pathway: an expanded neural framework for the processing of object quality.

PubMed

Kravitz, Dwight J; Saleem, Kadharbatcha S; Baker, Chris I; Ungerleider, Leslie G; Mishkin, Mortimer

2013-01-01

Since the original characterization of the ventral visual pathway, our knowledge of its neuroanatomy, functional properties, and extrinsic targets has grown considerably. Here we synthesize this recent evidence and propose that the ventral pathway is best understood as a recurrent occipitotemporal network containing neural representations of object quality both utilized and constrained by at least six distinct cortical and subcortical systems. Each system serves its own specialized behavioral, cognitive, or affective function, collectively providing the raison d'être for the ventral visual pathway. This expanded framework contrasts with the depiction of the ventral visual pathway as a largely serial staged hierarchy culminating in singular object representations and more parsimoniously incorporates attentional, contextual, and feedback effects. Published by Elsevier Ltd.

Evaluating a multispecies adaptive management framework: Must uncertainty impede effective decision-making?

USGS Publications Warehouse

Smith, David R.; McGowan, Conor P.; Daily, Jonathan P.; Nichols, James D.; Sweka, John A.; Lyons, James E.

2013-01-01

Application of adaptive management to complex natural resource systems requires careful evaluation to ensure that the process leads to improved decision-making. As part of that evaluation, adaptive policies can be compared with alternative nonadaptive management scenarios. Also, the value of reducing structural (ecological) uncertainty to achieving management objectives can be quantified.A multispecies adaptive management framework was recently adopted by the Atlantic States Marine Fisheries Commission for sustainable harvest of Delaware Bay horseshoe crabs Limulus polyphemus, while maintaining adequate stopover habitat for migrating red knots Calidris canutus rufa, the focal shorebird species. The predictive model set encompassed the structural uncertainty in the relationships between horseshoe crab spawning, red knot weight gain and red knot vital rates. Stochastic dynamic programming was used to generate a state-dependent strategy for harvest decisions given that uncertainty. In this paper, we employed a management strategy evaluation approach to evaluate the performance of this adaptive management framework. Active adaptive management was used by including model weights as state variables in the optimization and reducing structural uncertainty by model weight updating.We found that the value of information for reducing structural uncertainty is expected to be low, because the uncertainty does not appear to impede effective management. Harvest policy responded to abundance levels of both species regardless of uncertainty in the specific relationship that generated those abundances. Thus, the expected horseshoe crab harvest and red knot abundance were similar when the population generating model was uncertain or known, and harvest policy was robust to structural uncertainty as specified.Synthesis and applications. The combination of management strategy evaluation with state-dependent strategies from stochastic dynamic programming was an informative approach to

A portfolio evaluation framework for air transportation improvement projects

NASA Astrophysics Data System (ADS)

Baik, Hyeoncheol

This thesis explores the application of portfolio theory to the Air Transportation System (ATS) improvement. The ATS relies on complexly related resources and different stakeholder groups. Moreover, demand for air travel is significantly increasing relative to capacity of air transportation. In this environment, improving the ATS is challenging. Many projects, which are defined as technologies or initiatives, for improvement have been proposed and some have been demonstrated in practice. However, there is no clear understanding of how well these projects work in different conditions nor of how they interact with each other or with existing systems. These limitations make it difficult to develop good project combinations, or portfolios that maximize improvement. To help address this gap, a framework for identifying good portfolios is proposed. The framework can be applied to individual projects or portfolios of projects. Projects or portfolios are evaluated using four different groups of factors (effectiveness, time-to-implement, scope of applicability, and stakeholder impacts). Portfolios are also evaluated in terms of interaction-determining factors (prerequisites, co-requisites, limiting factors, and amplifying factors) because, while a given project might work well in isolation, interdependencies between projects or with existing systems could result in lower overall performance in combination. Ways to communicate a portfolio to decision makers are also introduced. The framework is unique because (1) it allows using a variety of available data, and (2) it covers diverse benefit metrics. For demonstrating the framework, an application to ground delay management projects serves as a case study. The portfolio evaluation approach introduced in this thesis can aid decision makers and researchers at universities and aviation agencies such as Federal Aviation Administration (FAA), National Aeronautics and Space Administration (NASA), and Department of Defense (DoD), in

'Healthy Eating and Lifestyle in Pregnancy (HELP)' trial: Process evaluation framework.

PubMed

Simpson, Sharon A; Cassidy, Dunla; John, Elinor

2014-07-01

We developed and tested in a cluster RCT a theory-driven group-based intervention for obese pregnant women. It was designed to support women to moderate weight gain during pregnancy and reduce BMI one year after birth, in addition to targeting secondary health and wellbeing outcomes. In line with MRC guidance on developing and evaluating complex interventions in health, we conducted a process evaluation alongside the trial. This paper describes the development of the process evaluation framework. This cluster RCT recruited 598 pregnant women. Women in the intervention group were invited to attend a weekly weight-management group. Following a review of relevant literature, we developed a process evaluation framework which outlined key process indicators that we wanted to address and how we would measure these. Central to the process evaluation was to understand the mechanism of effect of the intervention. We utilised a logic-modelling approach to describe the intervention which helped us focus on what potential mediators of intervention effect to measure, and how. The resulting process evaluation framework was designed to address 9 core elements; context, reach, exposure, recruitment, fidelity, recruitment, retention, contamination and theory-testing. These were assessed using a variety of qualitative and quantitative approaches. The logic model explained the processes by which intervention components bring about change in target outcomes through various mediators and theoretical pathways including self-efficacy, social support, self-regulation and motivation. Process evaluation is a key element in assessing the effect of any RCT. We developed a process evaluation framework and logic model, and the results of analyses using these will offer insights into why the intervention is or is not effective. Copyright © 2014.

A Framework for Evaluation and Use of Automated Scoring

ERIC Educational Resources Information Center

Williamson, David M.; Xi, Xiaoming; Breyer, F. Jay

2012-01-01

A framework for evaluation and use of automated scoring of constructed-response tasks is provided that entails both evaluation of automated scoring as well as guidelines for implementation and maintenance in the context of constantly evolving technologies. Consideration of validity issues and challenges associated with automated scoring are…

Writing objectives and evaluating learning in the affective domain.

PubMed

Maier-Lorentz, M M

1999-01-01

Staff educators recognize the importance of affective competency for effective nursing practice. Inservice programs must include affective learning with objectives stated in measurable terms. Staff educators often express frustration in developing affective objectives and evaluating the learning outcome because attitudes and feelings are usually inferred from observations. This article presents affective learning objectives for a gerontological nursing inservice program and a rating scale that measures attitudes to evaluate the learning outcome.

A Need for a Framework for Curriculum Evaluation in Oman

ERIC Educational Resources Information Center

Al-Jardani, Khalid Salim; Siraj, Saedah; Abedalaziz, Nabeel

2012-01-01

The field of curriculum evaluation is a key part of the educational process. This means that this area needs to be developed continuously and requires ongoing research. This paper highlights curriculum evaluation in Oman, different evaluation procedures and methods and instruments used. The need for a framework for curriculum evaluation is a vital…

Virtual Workshop Environment (VWE): A Taxonomy and Service Oriented Architecture (SOA) Framework for Modularized Virtual Learning Environments (VLE)--Applying the Learning Object Concept to the VLE

ERIC Educational Resources Information Center

Paulsson, Fredrik; Naeve, Ambjorn

2006-01-01

Based on existing Learning Object taxonomies, this article suggests an alternative Learning Object taxonomy, combined with a general Service Oriented Architecture (SOA) framework, aiming to transfer the modularized concept of Learning Objects to modularized Virtual Learning Environments. The taxonomy and SOA-framework exposes a need for a clearer…

Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda.

PubMed

Hotchkiss, David R; Aqil, Anwer; Lippeveld, Theo; Mukooyo, Edward

2010-07-03

Sound policy, resource allocation and day-to-day management decisions in the health sector require timely information from routine health information systems (RHIS). In most low- and middle-income countries, the RHIS is viewed as being inadequate in providing quality data and continuous information that can be used to help improve health system performance. In addition, there is limited evidence on the effectiveness of RHIS strengthening interventions in improving data quality and use. The purpose of this study is to evaluate the usefulness of the newly developed Performance of Routine Information System Management (PRISM) framework, which consists of a conceptual framework and associated data collection and analysis tools to assess, design, strengthen and evaluate RHIS. The specific objectives of the study are: a) to assess the reliability and validity of the PRISM instruments and b) to assess the validity of the PRISM conceptual framework. Facility- and worker-level data were collected from 110 health care facilities in twelve districts in Uganda in 2004 and 2007 using records reviews, structured interviews and self-administered questionnaires. The analysis procedures include Cronbach's alpha to assess internal consistency of selected instruments, test-retest analysis to assess the reliability and sensitivity of the instruments, and bivariate and multivariate statistical techniques to assess validity of the PRISM instruments and conceptual framework. Cronbach's alpha analysis suggests high reliability (0.7 or greater) for the indices measuring a promotion of a culture of information, RHIS tasks self-efficacy and motivation. The study results also suggest that a promotion of a culture of information influences RHIS tasks self-efficacy, RHIS tasks competence and motivation, and that self-efficacy and the presence of RHIS staff have a direct influence on the use of RHIS information, a key aspect of RHIS performance. The study results provide some empirical support

Semantic framework for mapping object-oriented model to semantic web languages

PubMed Central

Ježek, Petr; Mouček, Roman

2015-01-01

The article deals with and discusses two main approaches in building semantic structures for electrophysiological metadata. It is the use of conventional data structures, repositories, and programming languages on one hand and the use of formal representations of ontologies, known from knowledge representation, such as description logics or semantic web languages on the other hand. Although knowledge engineering offers languages supporting richer semantic means of expression and technological advanced approaches, conventional data structures and repositories are still popular among developers, administrators and users because of their simplicity, overall intelligibility, and lower demands on technical equipment. The choice of conventional data resources and repositories, however, raises the question of how and where to add semantics that cannot be naturally expressed using them. As one of the possible solutions, this semantics can be added into the structures of the programming language that accesses and processes the underlying data. To support this idea we introduced a software prototype that enables its users to add semantically richer expressions into a Java object-oriented code. This approach does not burden users with additional demands on programming environment since reflective Java annotations were used as an entry for these expressions. Moreover, additional semantics need not to be written by the programmer directly to the code, but it can be collected from non-programmers using a graphic user interface. The mapping that allows the transformation of the semantically enriched Java code into the Semantic Web language OWL was proposed and implemented in a library named the Semantic Framework. This approach was validated by the integration of the Semantic Framework in the EEG/ERP Portal and by the subsequent registration of the EEG/ERP Portal in the Neuroscience Information Framework. PMID:25762923

Semantic framework for mapping object-oriented model to semantic web languages.

PubMed

Ježek, Petr; Mouček, Roman

2015-01-01

The article deals with and discusses two main approaches in building semantic structures for electrophysiological metadata. It is the use of conventional data structures, repositories, and programming languages on one hand and the use of formal representations of ontologies, known from knowledge representation, such as description logics or semantic web languages on the other hand. Although knowledge engineering offers languages supporting richer semantic means of expression and technological advanced approaches, conventional data structures and repositories are still popular among developers, administrators and users because of their simplicity, overall intelligibility, and lower demands on technical equipment. The choice of conventional data resources and repositories, however, raises the question of how and where to add semantics that cannot be naturally expressed using them. As one of the possible solutions, this semantics can be added into the structures of the programming language that accesses and processes the underlying data. To support this idea we introduced a software prototype that enables its users to add semantically richer expressions into a Java object-oriented code. This approach does not burden users with additional demands on programming environment since reflective Java annotations were used as an entry for these expressions. Moreover, additional semantics need not to be written by the programmer directly to the code, but it can be collected from non-programmers using a graphic user interface. The mapping that allows the transformation of the semantically enriched Java code into the Semantic Web language OWL was proposed and implemented in a library named the Semantic Framework. This approach was validated by the integration of the Semantic Framework in the EEG/ERP Portal and by the subsequent registration of the EEG/ERP Portal in the Neuroscience Information Framework.

A framework for the design, implementation, and evaluation of interprofessional education.

PubMed

Pardue, Karen T

2015-01-01

The growing emphasis on teamwork and care coordination within health care delivery is sparking interest in interprofessional education (IPE) among nursing and health profession faculty. Faculty often lack firsthand IPE experience, which hinders pedagogical reform. This article proposes a theoretically grounded framework for the design, implementation, and evaluation of IPE. Supporting literature and practical advice are interwoven. The proposed framework guides faculty in the successful creation and evaluation of collaborative learning experiences.

A framework for outcome-level evaluation of in-service training of health care workers.

PubMed

O'Malley, Gabrielle; Perdue, Thomas; Petracca, Frances

2013-10-01

In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President's Emergency Plan for AIDS Relief's Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the ability of evaluators to

A flexible object-oriented software framework for developing complex multimedia simulations.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sydelko, P. J.; Dolph, J. E.; Christiansen, J. H.

Decision makers involved in brownfields redevelopment and long-term stewardship must consider environmental conditions, future-use potential, site ownership, area infrastructure, funding resources, cost recovery, regulations, risk and liability management, community relations, and expected return on investment in a comprehensive and integrated fashion to achieve desired results. Successful brownfields redevelopment requires the ability to assess the impacts of redevelopment options on multiple interrelated aspects of the ecosystem, both natural and societal. Computer-based tools, such as simulation models, databases, and geographical information systems (GISs) can be used to address brownfields planning and project execution. The transparent integration of these tools into a comprehensivemore » and dynamic decision support system would greatly enhance the brownfields assessment process. Such a system needs to be able to adapt to shifting and expanding analytical requirements and contexts. The Dynamic Information Architecture System (DIAS) is a flexible, extensible, object-oriented framework for developing and maintaining complex multidisciplinary simulations of a wide variety of application domains. The modeling domain of a specific DIAS-based simulation is determined by (1) software objects that represent the real-world entities that comprise the problem space (atmosphere, watershed, human), and (2) simulation models and other data processing applications that express the dynamic behaviors of the domain entities. Models and applications used to express dynamic behaviors can be either internal or external to DIAS, including existing legacy models written in various languages (FORTRAN, C, etc.). The flexible design framework of DIAS makes the objects adjustable to the context of the problem without a great deal of recoding. The DIAS Spatial Data Set facility allows parameters to vary spatially depending on the simulation context according to any of a number of 1-D

A Conceptual Framework for Graduate Teaching Assistant Professional Development Evaluation and Research.

PubMed

Reeves, Todd D; Marbach-Ad, Gili; Miller, Kristen R; Ridgway, Judith; Gardner, Grant E; Schussler, Elisabeth E; Wischusen, E William

2016-01-01

Biology graduate teaching assistants (GTAs) are significant contributors to the educational mission of universities, particularly in introductory courses, yet there is a lack of empirical data on how to best prepare them for their teaching roles. This essay proposes a conceptual framework for biology GTA teaching professional development (TPD) program evaluation and research with three overarching variable categories for consideration: outcome variables, contextual variables, and moderating variables. The framework's outcome variables go beyond GTA satisfaction and instead position GTA cognition, GTA teaching practice, and undergraduate learning outcomes as the foci of GTA TPD evaluation and research. For each GTA TPD outcome variable, key evaluation questions and example assessment instruments are introduced to demonstrate how the framework can be used to guide GTA TPD evaluation and research plans. A common conceptual framework is also essential to coordinating the collection and synthesis of empirical data on GTA TPD nationally. Thus, the proposed conceptual framework serves as both a guide for conducting GTA TPD evaluation at single institutions and as a means to coordinate research across institutions at a national level. © 2016 T. D. Reeves et al. CBE—Life Sciences Education © 2016 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Privacy Protection by Masking Moving Objects for Security Cameras

NASA Astrophysics Data System (ADS)

Yabuta, Kenichi; Kitazawa, Hitoshi; Tanaka, Toshihisa

Because of an increasing number of security cameras, it is crucial to establish a system that protects the privacy of objects in the recorded images. To this end, we propose a framework of image processing and data hiding for security monitoring and privacy protection. First, we state the requirements of the proposed monitoring systems and suggest possible implementation that satisfies those requirements. The underlying concept of our proposed framework is as follows: (1) in the recorded images, the objects whose privacy should be protected are deteriorated by appropriate image processing; (2) the original objects are encrypted and watermarked into the output image, which is encoded using an image compression standard; (3) real-time processing is performed such that no future frame is required to generate on output bitstream. It should be noted that in this framework, anyone can observe the decoded image that includes the deteriorated objects that are unrecognizable or invisible. On the other hand, for crime investigation, this system allows a limited number of users to observe the original objects by using a special viewer that decrypts and decodes the watermarked objects with a decoding password. Moreover, the special viewer allows us to select the objects to be decoded and displayed. We provide an implementation example, experimental results, and performance evaluations to support our proposed framework.

Effective integrated frameworks for assessing mining sustainability.

PubMed

Virgone, K M; Ramirez-Andreotta, M; Mainhagu, J; Brusseau, M L

2018-05-28

The objectives of this research are to review existing methods used for assessing mining sustainability, analyze the limited prior research that has evaluated the methods, and identify key characteristics that would constitute an enhanced sustainability framework that would serve to improve sustainability reporting in the mining industry. Five of the most relevant frameworks were selected for comparison in this analysis, and the results show that there are many commonalities among the five, as well as some disparities. In addition, relevant components are missing from all five. An enhanced evaluation system and framework were created to provide a more holistic, comprehensive method for sustainability assessment and reporting. The proposed framework has five components that build from and encompass the twelve evaluation characteristics used in the analysis. The components include Foundation, Focus, Breadth, Quality Assurance, and Relevance. The enhanced framework promotes a comprehensive, location-specific reporting approach with a concise set of well-defined indicators. Built into the framework is quality assurance, as well as a defined method to use information from sustainability reports to inform decisions. The framework incorporates human health and socioeconomic aspects via initiatives such as community-engaged research, economic valuations, and community-initiated environmental monitoring.

A Python object-oriented framework for the CMS alignment and calibration data

NASA Astrophysics Data System (ADS)

Dawes, Joshua H.; CMS Collaboration

2017-10-01

The Alignment, Calibrations and Databases group at the CMS Experiment delivers Alignment and Calibration Conditions Data to a large set of workflows which process recorded event data and produce simulated events. The current infrastructure for releasing and consuming Conditions Data was designed in the two years of the first LHC long shutdown to respond to use cases from the preceding data-taking period. During the second run of the LHC, new use cases were defined. For the consumption of Conditions Metadata, no common interface existed for the detector experts to use in Python-based custom scripts, resulting in many different querying and transaction management patterns. A new framework has been built to address such use cases: a simple object-oriented tool that detector experts can use to read and write Conditions Metadata when using Oracle and SQLite databases, that provides a homogeneous method of querying across all services. The tool provides mechanisms for segmenting large sets of conditions while releasing them to the production database, allows for uniform error reporting to the client-side from the server-side and optimizes the data transfer to the server. The architecture of the new service has been developed exploiting many of the features made available by the metadata consumption framework to implement the required improvements. This paper presents the details of the design and implementation of the new metadata consumption and data upload framework, as well as analyses of the new upload service’s performance as the server-side state varies.

a New Object-Based Framework to Detect Shodows in High-Resolution Satellite Imagery Over Urban Areas

NASA Astrophysics Data System (ADS)

Tatar, N.; Saadatseresht, M.; Arefi, H.; Hadavand, A.

2015-12-01

In this paper a new object-based framework to detect shadow areas in high resolution satellite images is proposed. To produce shadow map in pixel level state of the art supervised machine learning algorithms are employed. Automatic ground truth generation based on Otsu thresholding on shadow and non-shadow indices is used to train the classifiers. It is followed by segmenting the image scene and create image objects. To detect shadow objects, a majority voting on pixel-based shadow detection result is designed. GeoEye-1 multi-spectral image over an urban area in Qom city of Iran is used in the experiments. Results shows the superiority of our proposed method over traditional pixel-based, visually and quantitatively.

An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit).

PubMed

Yusof, Maryati Mohd; Kuljis, Jasna; Papazafeiropoulou, Anastasia; Stergioulas, Lampros K

2008-06-01

The realization of Health Information Systems (HIS) requires rigorous evaluation that addresses technology, human and organization issues. Our review indicates that current evaluation methods evaluate different aspects of HIS and they can be improved upon. A new evaluation framework, human, organization and technology-fit (HOT-fit) was developed after having conducted a critical appraisal of the findings of existing HIS evaluation studies. HOT-fit builds on previous models of IS evaluation--in particular, the IS Success Model and the IT-Organization Fit Model. This paper introduces the new framework for HIS evaluation that incorporates comprehensive dimensions and measures of HIS and provides a technological, human and organizational fit. Literature review on HIS and IS evaluation studies and pilot testing of developed framework. The framework was used to evaluate a Fundus Imaging System (FIS) of a primary care organization in the UK. The case study was conducted through observation, interview and document analysis. The main findings show that having the right user attitude and skills base together with good leadership, IT-friendly environment and good communication can have positive influence on the system adoption. Comprehensive, specific evaluation factors, dimensions and measures in the new framework (HOT-fit) are applicable in HIS evaluation. The use of such a framework is argued to be useful not only for comprehensive evaluation of the particular FIS system under investigation, but potentially also for any Health Information System in general.

A framework for evaluating national space activity

NASA Astrophysics Data System (ADS)

Wood, Danielle; Weigel, Annalisa

2012-04-01

Space technology and resources are used around the world to address societal challenges. Space provides valuable satellite services, unique scientific discoveries, surprising technology applications and new economic opportunities. Many developing countries formally recognize the advantages of space resources and pursue national level activity to harness them. There is limited data or documentation on the space activities of developing countries. Meanwhile, traditional approaches to summarize national space activity do not necessarily capture the types of activity that developing countries pursue in space. This is especially true if they do not have a formal national space program or office. Developing countries pursue national space activity through activities of many types—from national satellite programs to commercial use of satellite services to involvement with international space institutions. This research aims to understand and analyze these trends. This paper introduces two analytical frameworks for evaluating space activity at the national level. The frameworks are specifically designed to capture the activity of countries that have traditionally been less involved in space. They take a broad view of space related activity across multiple societal sectors and disciplines. The discussion explains the approach for using the frameworks as well as illustrative examples of how they can be applied as part of a research process. The first framework is called the Mission and Management Ladders. This framework considers specific space projects within countries and ranks them on "Ladders" that measure technical challenge and managerial autonomy. This first method is at a micro level of analysis. The second framework is called the Space Participation Metric (SPM). The SPM can be used to assign a Space Participation score to countries based on their involvement in various space related activities. This second method uses a macro level of analysis. The authors

An Evaluation of the Effects of Experimenter Control of Objects on Individuals' Engagement in Object Stereotypy

ERIC Educational Resources Information Center

Stangeland, Lindsay A.; Smith, Dean P.; Rapp, John T.

2012-01-01

In two experiments, the authors evaluated the extent to which (a) individuals preferred engaging in object stereotypy versus observing an experimenter while the experimenter engaged in object stereotypy and (b) an experimenter's engagement in object stereotypy decreased the participants' engagement in object stereotypy. Results of Experiment 1…

Choices, Frameworks and Refinement

NASA Technical Reports Server (NTRS)

Campbell, Roy H.; Islam, Nayeem; Johnson, Ralph; Kougiouris, Panos; Madany, Peter

1991-01-01

In this paper we present a method for designing operating systems using object-oriented frameworks. A framework can be refined into subframeworks. Constraints specify the interactions between the subframeworks. We describe how we used object-oriented frameworks to design Choices, an object-oriented operating system.

An audience-channel-message-evaluation (ACME) framework for health communication campaigns.

PubMed

Noar, Seth M

2012-07-01

Recent reviews of the literature have indicated that a number of health communication campaigns continue to fail to adhere to principles of effective campaign design. The lack of an integrated, organizing framework for the design, implementation, and evaluation of health communication campaigns may contribute to this state of affairs. The current article introduces an audience-channel-message-evaluation (ACME) framework that organizes the major principles of health campaign design, implementation, and evaluation. ACME also explicates the relationships and linkages between the varying principles. Insights from ACME include the following: The choice of audience segment(s) to focus on in a campaign affects all other campaign design choices, including message strategy and channel/component options. Although channel selection influences options for message design, choice of message design also influences channel options. Evaluation should not be thought of as a separate activity, but rather should be infused and integrated throughout the campaign design and implementation process, including formative, process, and outcome evaluation activities. Overall, health communication campaigns that adhere to this integrated set of principles of effective campaign design will have a greater chance of success than those using principles idiosyncratically. These design, implementation, and evaluation principles are embodied in the ACME framework.

Using the Evaluative Linguistic Framework for Questionnaires to Assess Comprehensibility of Self-Report Health Questionnaires.

PubMed

Clerehan, Rosemary; Guillemin, Francis; Epstein, Jonathan; Buchbinder, Rachelle

2016-06-01

The Evaluative Linguistic Framework (ELF) was developed to judge the quality of health care texts for patients, based on systemic functional linguistic theory. This approach considers key variables such as context and structure, known to be important for communication. Our objective was to adapt the ELF to evaluate the quality of self-report questionnaires. We reviewed the Health Literacy Questionnaire using the ELF. On the basis of these data, we drafted the preliminary version of the Evaluative Linguistic Framework for Questionnaires (ELF-Q) and applied it to English- and French-language versions of two arthritis self-report questionnaires and to Spanish, Dutch, and Turkish versions of an arthritis questionnaire. The developed ELF-Q was found to be effective for evaluating questionnaires in English and in four other languages. It contains nine items with new descriptions and assessment probes. These include overall organizational or generic structure of the questionnaire, metadiscourse (or text about the text), headings, rhetorical elements (function of each "move" or stage of the text in relation to the reader), the writer-reader relationship, technicality of vocabulary, lexical density (proportion of content words in the text), format, and overall judgment. We added assessment responses on a two- or three-point Likert scale to complement the assessment probes and make the intent and meaning of the probes fully explicit for the questionnaire developer or assessor. The ELF-Q is a framework practical to use for the development or assessment of any type of self-report questionnaire. Its application can identify features of a self-report questionnaire that could be improved to optimize its comprehensibility. Copyright © 2016 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

An evaluation framework for participatory modelling

NASA Astrophysics Data System (ADS)

Krueger, T.; Inman, A.; Chilvers, J.

2012-04-01

Strong arguments for participatory modelling in hydrology can be made on substantive, instrumental and normative grounds. These arguments have led to increasingly diverse groups of stakeholders (here anyone affecting or affected by an issue) getting involved in hydrological research and the management of water resources. In fact, participation has become a requirement of many research grants, programs, plans and policies. However, evidence of beneficial outcomes of participation as suggested by the arguments is difficult to generate and therefore rare. This is because outcomes are diverse, distributed, often tacit, and take time to emerge. In this paper we develop an evaluation framework for participatory modelling focussed on learning outcomes. Learning encompasses many of the potential benefits of participation, such as better models through diversity of knowledge and scrutiny, stakeholder empowerment, greater trust in models and ownership of subsequent decisions, individual moral development, reflexivity, relationships, social capital, institutional change, resilience and sustainability. Based on the theories of experiential, transformative and social learning, complemented by practitioner experience our framework examines if, when and how learning has occurred. Special emphasis is placed on the role of models as learning catalysts. We map the distribution of learning between stakeholders, scientists (as a subgroup of stakeholders) and models. And we analyse what type of learning has occurred: instrumental learning (broadly cognitive enhancement) and/or communicative learning (change in interpreting meanings, intentions and values associated with actions and activities; group dynamics). We demonstrate how our framework can be translated into a questionnaire-based survey conducted with stakeholders and scientists at key stages of the participatory process, and show preliminary insights from applying the framework within a rural pollution management situation in

Contributions to Objective Measurement and Evaluation of Trainee Competency.

ERIC Educational Resources Information Center

Moonan, William J.

The purpose of this paper is to lay a basis for and discuss the components of a system, called COMET, designed to objectively measure and evaluate the competency of trainees in military training enterprises. COMET is an acronym for "Computerized Objective Measurement and Evaluation of Trainees." These goals will be accomplished by: (a)…

Framework and criteria for program evaluation in the Office of Conservation and Renewable Energy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

This study addresses the development of a framework and generic criteria for conducting program evaluation in the Office of Conservation and Renewable Energy. The evaluation process is intended to provide the Assistant Secretary with comprehensive and consistent evaluation data for management decisions regarding policy and strategy, crosscutting energy impacts and resource allocation and justification. The study defines evaluation objectives, identifies basic information requirements (criteria), and identifies a process for collecting evaluation results at the basic program level, integrating the results, and summarizing information upward through the CE organization to the Assistant Secretary. Methods are described by which initial criteria weremore » tested, analyzed, and refined for CE program applicability. General guidelines pertaining to evaluation and the Sunset Review requirements are examined and various types, designs, and models for evaluation are identified. Existing CE evaluation reports are reviewed and comments on their adequacy for meeting current needs are provided. An inventory and status survey of CE program evaluation activities is presented, as are issues, findings, and recommendations pertaining to CE evaluation and Sunset Review requirements. Also, sources of data for use in evaluation and the Sunset Review response are identified. An inventory of CE evaluation-related documents and reports is provided.« less

A framework for outcome-level evaluation of in-service training of health care workers

PubMed Central

2013-01-01

Background In-service training is a key strategic approach to addressing the severe shortage of health care workers in many countries. However, there is a lack of evidence linking these health care worker trainings to improved health outcomes. In response, the United States President’s Emergency Plan for AIDS Relief’s Human Resources for Health Technical Working Group initiated a project to develop an outcome-focused training evaluation framework. This paper presents the methods and results of that project. Methods A general inductive methodology was used for the conceptualization and development of the framework. Fifteen key informant interviews were conducted to explore contextual factors, perceived needs, barriers and facilitators affecting the evaluation of training outcomes. In addition, a thematic analysis of 70 published articles reporting health care worker training outcomes identified key themes and categories. These were integrated, synthesized and compared to several existing training evaluation models. This formed an overall typology which was used to draft a new framework. Finally, the framework was refined and validated through an iterative process of feedback, pilot testing and revision. Results The inductive process resulted in identification of themes and categories, as well as relationships among several levels and types of outcomes. The resulting framework includes nine distinct types of outcomes that can be evaluated, which are organized within three nested levels: individual, organizational and health system/population. The outcome types are: (1) individual knowledge, attitudes and skills; (2) individual performance; (3) individual patient health; (4) organizational systems; (5) organizational performance; (6) organizational-level patient health; (7) health systems; (8) population-level performance; and (9) population-level health. The framework also addresses contextual factors which may influence the outcomes of training, as well as the

Methodology Evaluation Framework for Component-Based System Development.

ERIC Educational Resources Information Center

Dahanayake, Ajantha; Sol, Henk; Stojanovic, Zoran

2003-01-01

Explains component-based development (CBD) for distributed information systems and presents an evaluation framework, which highlights the extent to which a methodology is component oriented. Compares prominent CBD methods, discusses ways of modeling, and suggests that this is a first step towards a components-oriented systems development…

The AgMIP Framework to Evaluate Agricultural Pathways

NASA Technical Reports Server (NTRS)

Ruane, Alex

2015-01-01

This talk will describe the community and research framework that AgMIP has built to enable evidence-based adaptation investment. We provide expertise on the ground and connect various disciplines in order to allow specific adaptations to be evaluated for their biophysical and socio-economic ramifications.

JSEM: A Framework for Identifying and Evaluating Indicators.

ERIC Educational Resources Information Center

Hyman, Jeffrey B.; Leibowitz, Scott G.

2001-01-01

Presents an approach to identifying and evaluating combinations of indicators when the mathematical relationships between the indicators and an endpoint may not be quantified, a limitation common to many ecological assessments. Uses the framework of Structural Equation Modeling (SEM), which combines path analysis with measurement model, to…

Deep Spatial-Temporal Joint Feature Representation for Video Object Detection.

PubMed

Zhao, Baojun; Zhao, Boya; Tang, Linbo; Han, Yuqi; Wang, Wenzheng

2018-03-04

With the development of deep neural networks, many object detection frameworks have shown great success in the fields of smart surveillance, self-driving cars, and facial recognition. However, the data sources are usually videos, and the object detection frameworks are mostly established on still images and only use the spatial information, which means that the feature consistency cannot be ensured because the training procedure loses temporal information. To address these problems, we propose a single, fully-convolutional neural network-based object detection framework that involves temporal information by using Siamese networks. In the training procedure, first, the prediction network combines the multiscale feature map to handle objects of various sizes. Second, we introduce a correlation loss by using the Siamese network, which provides neighboring frame features. This correlation loss represents object co-occurrences across time to aid the consistent feature generation. Since the correlation loss should use the information of the track ID and detection label, our video object detection network has been evaluated on the large-scale ImageNet VID dataset where it achieves a 69.5% mean average precision (mAP).

Assessing Vital Signs: Applying Two Participatory Evaluation Frameworks to the Evaluation of a College of Nursing

ERIC Educational Resources Information Center

Connors, Susan C.; Magilvy, Joan K.

2011-01-01

Evaluation research has been in progress to clarify the concept of participatory evaluation and to assess its impact. Recently, two theoretical frameworks have been offered--Daigneault and Jacob's participatory evaluation measurement index and Champagne and Smits' model of practical participatory evaluation. In this case report, we apply these…

A data fusion framework for meta-evaluation of intelligent transportation system effectiveness

DOT National Transportation Integrated Search

This study presents a framework for the meta-evaluation of Intelligent Transportation System effectiveness. The framework is based on data fusion approaches that adjust for data biases and violations of other standard statistical assumptions. Operati...

The ventral visual pathway: An expanded neural framework for the processing of object quality

PubMed Central

Kravitz, Dwight J.; Saleem, Kadharbatcha S.; Baker, Chris I.; Ungerleider, Leslie G.; Mishkin, Mortimer

2012-01-01

Since the original characterization of the ventral visual pathway our knowledge of its neuroanatomy, functional properties, and extrinsic targets has grown considerably. Here we synthesize this recent evidence and propose that the ventral pathway is best understood as a recurrent occipitotemporal network containing neural representations of object quality both utilized and constrained by at least six distinct cortical and subcortical systems. Each system serves its own specialized behavioral, cognitive, or affective function, collectively providing the raison d’etre for the ventral visual pathway. This expanded framework contrasts with the depiction of the ventral visual pathway as a largely serial staged hierarchy that culminates in singular object representations for utilization mainly by ventrolateral prefrontal cortex and, more parsimoniously than this account, incorporates attentional, contextual, and feedback effects. PMID:23265839

An Application of the Impact Evaluation Process for Designing a Performance Measurement and Evaluation Framework in K-12 Environments

ERIC Educational Resources Information Center

Guerra-Lopez, Ingrid; Toker, Sacip

2012-01-01

This article illustrates the application of the Impact Evaluation Process for the design of a performance measurement and evaluation framework for an urban high school. One of the key aims of this framework is to enhance decision-making by providing timely feedback about the effectiveness of various performance improvement interventions. The…

The development of an evaluation framework for injury surveillance systems

PubMed Central

Mitchell, Rebecca J; Williamson, Ann M; O'Connor, Rod

2009-01-01

Background Access to good quality information from injury surveillance is essential to develop and monitor injury prevention activities. To determine if information obtained from surveillance is of high quality, the limitations and strengths of a surveillance system are often examined. Guidelines have been developed to assist in evaluating certain types of surveillance systems. However, to date, no standard guidelines have been developed to specifically evaluate an injury surveillance system. The aim of this research is to develop a framework to guide the evaluation of injury surveillance systems. Methods The development of an Evaluation Framework for Injury Surveillance Systems (EFISS) involved a four stage process. First, a literature review was conducted to identify an initial set of characteristics that were recognised as important and/or had been recommended to be assessed in an evaluation of a surveillance system. Second, this set of characteristics was assessed using SMART criteria. Third, those surviving were presented to an expert panel using a two round modified-Delphi study to gain an alternative perspective on characteristic definitions, practicality of assessment, and characteristic importance. Finally, a rating system was created for the EFISS characteristics. Results The resulting EFISS consisted of 18 characteristics that assess three areas of an injury surveillance system – five characteristics assess data quality, nine characteristics assess the system's operation, and four characteristics assess the practical capability of an injury surveillance system. A rating system assesses the performance of each characteristic. Conclusion The development of the EFISS builds upon existing evaluation guidelines for surveillance systems and provides a framework tailored to evaluate an injury surveillance system. Ultimately, information obtained through an evaluation of an injury data collection using the EFISS would be useful for agencies to recommend how a

POLE.VAULT: A Semantic Framework for Health Policy Evaluation and Logical Testing.

PubMed

Shaban-Nejad, Arash; Okhmatovskaia, Anya; Shin, Eun Kyong; Davis, Robert L; Buckeridge, David L

2017-01-01

The major goal of our study is to provide an automatic evaluation framework that aligns the results generated through semantic reasoning with the best available evidence regarding effective interventions to support the logical evaluation of public health policies. To this end, we have designed the POLicy EVAlUation & Logical Testing (POLE.VAULT) Framework to assist different stakeholders and decision-makers in making informed decisions about different health-related interventions, programs and ultimately policies, based on the contextual knowledge and the best available evidence at both individual and aggregate levels.

Lessons learned about collaborative evaluation using the Capacity for Applying Project Evaluation (CAPE) framework with school and district leaders.

PubMed

Corn, Jenifer O; Byrom, Elizabeth; Knestis, Kirk; Matzen, Nita; Thrift, Beth

2012-11-01

Schools, districts, and state-level educational organizations are experiencing a great shift in the way they do the business of education. This shift focuses on accountability, specifically through the expectation of the effective utilization of evaluative-focused efforts to guide and support decisions about educational program implementation. In as much, education leaders need specific guidance and training on how to plan, implement, and use evaluation to critically examine district and school-level initiatives. One specific effort intended to address this need is through the Capacity for Applying Project Evaluation (CAPE) framework. The CAPE framework is composed of three crucial components: a collection of evaluation resources; a professional development model; and a conceptual framework that guides the work to support evaluation planning and implementation in schools and districts. School and district teams serve as active participants in the professional development and ultimately as formative evaluators of their own school or district-level programs by working collaboratively with evaluation experts. The CAPE framework involves the school and district staff in planning and implementing their evaluation. They are the ones deciding what evaluation questions to ask, which instruments to use, what data to collect, and how and to whom results should be reported. Initially this work is done through careful scaffolding by evaluation experts, where supports are slowly pulled away as the educators gain experience and confidence in their knowledge and skills as evaluators. Since CAPE engages all stakeholders in all stages of the evaluation, the philosophical intentions of these efforts to build capacity for formative evaluation strictly aligns with the collaborative evaluation approach. Copyright © 2012 Elsevier Ltd. All rights reserved.

Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation

PubMed Central

Wootton, Richard; Vladzymyrskyy, Anton; Zolfo, Maria; Bonnardot, Laurent

2011-01-01

Background Telemedicine has been used for many years to support doctors in the developing world. Several networks provide services in different settings and in different ways. However, to draw conclusions about which telemedicine networks are successful requires a method of evaluating them. No general consensus or validated framework exists for this purpose. Objective To define a basic method of performance measurement that can be used to improve and compare teleconsultation networks; to employ the proposed framework in an evaluation of three existing networks; to make recommendations about the future implementation and follow-up of such networks. Methods Analysis based on the experience of three telemedicine networks (in operation for 7–10 years) that provide services to doctors in low-resource settings and which employ the same basic design. Findings Although there are many possible indicators and metrics that might be relevant, five measures for each of the three user groups appear to be sufficient for the proposed framework. In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The proposed performance measurement framework was applied to three mature telemedicine networks. Despite their differences in terms of activity, size and objectives, their performance in certain respects is very similar. For example, the time to first reply from an expert is about 24 hours for each network. Although all three networks had systems in place to collect data from the user perspective, none of them collected information about the coordinator's time required or about ease of system usage. They had only limited information about quality and cost. Conclusion Measuring the performance of a telemedicine network is essential in understanding whether the network is working as intended and what effect it is having. Based on long-term field experience, the suggested framework is a practical tool that will permit

A framework for evaluating proposals for scientific activities in wilderness

Treesearch

Peter Landres

2000-01-01

This paper presents a structured framework for evaluating proposals for scientific activities in wilderness. Wilderness managers receive proposals for scientific activities ranging from unobtrusive inventorying of plants and animals to the use of chainsaws and helicopters for collecting information. Currently, there is no consistent process for evaluating proposals,...

Evaluation in Cross-Cultural Contexts: Proposing a Framework for International Education and Training Project Evaluations.

ERIC Educational Resources Information Center

bin Yahya, Ismail; And Others

This paper focuses on the need for increased sensitivity and responsiveness in international education and training project evaluations, particularly those in Third World countries. A conceptual-theoretical framework for designing and developing models appropriate for evaluating education and training projects in non-Western cultures is presented.…

Analyzing Electronic Question/Answer Services: Framework and Evaluations of Selected Services.

ERIC Educational Resources Information Center

White, Marilyn Domas, Ed.

This report develops an analytical framework based on systems analysis for evaluating electronic question/answer or AskA services operated by a wide range of types of organizations, including libraries. Version 1.0 of this framework was applied in June 1999 to a selective sample of 11 electronic question/answer services, which cover a range of…

Beyond the Rhetoric: A Framework for Evaluating Improvements to the Student Experience

ERIC Educational Resources Information Center

Baird, Jeanette; Gordon, George

2009-01-01

A framework is described to assist institutions in evaluating the extent to which activities described as "quality improvements" or "quality enhancements" are likely to directly improve the student experience. The framework classifies ways of improving the student experience into "coaching improvements",…

Framework of outcome measures recommended for use in the evaluation of childhood obesity treatment interventions: the CoOR framework.

PubMed

Bryant, M; Ashton, L; Nixon, J; Jebb, S; Wright, J; Roberts, K; Brown, J

2014-12-01

Consensus is lacking in determining appropriate outcome measures for assessment of childhood obesity treatments. Inconsistency in the use and reporting of such measures impedes comparisons between treatments and limits consideration of effectiveness. This study aimed to produce a framework of recommended outcome measures: the Childhood obesity treatment evaluation Outcomes Review (CoOR) framework. A systematic review including two searches was conducted to identify (1) existing trial outcome measures and (2) manuscripts describing development/evaluation of outcome measures. Outcomes included anthropometry, diet, eating behaviours, physical activity, sedentary time/behaviour, fitness, physiology, environment, psychological well-being and health-related quality of life. Eligible measures were appraised by the internal team using a system developed from international guidelines, followed by appraisal from national external expert collaborators. A total of 25,486 papers were identified through both searches. Eligible search 1 trial papers cited 417 additional papers linked to outcome measures, of which 56 were eligible. A further 297 outcome development/evaluation papers met eligibility criteria from search 2. Combined, these described 191 outcome measures. After internal and external appraisal, 52 measures across 10 outcomes were recommended for inclusion in the CoOR framework. Application of the CoOR framework will ensure greater consistency in choosing robust outcome measures that are appropriate to population characteristics. © 2014 The Authors. Pediatric Obesity © 2014 International Association for the Study of Obesity.

Diversity training for the community aged care workers: A conceptual framework for evaluation.

PubMed

Appannah, Arti; Meyer, Claudia; Ogrin, Rajna; McMillan, Sally; Barrett, Elizabeth; Browning, Colette

2017-08-01

Older Australians are an increasingly diverse population, with variable characteristics such as culture, sexual orientation, socioeconomic status, and physical capabilities potentially influencing their participation in healthcare. In response, community aged care workers may need to increase skills and uptake of knowledge into practice regarding diversity through appropriate training interventions. Diversity training (DT) programs have traditionally existed in the realm of business, with little research attention devoted to scientifically evaluating the outcomes of training directed at community aged care workers. A DT workshop has been developed for community aged care workers, and this paper focuses on the construction of a formative evaluative framework for the workshop. Key evaluation concepts and measures relating to DT have been identified in the literature and integrated into the framework, focusing on five categories: Training needs analysis; Reactions; Learning outcomes, Behavioural outcomes and Results The use of a mixed methods approach in the framework provides an additional strength, by evaluating long-term behavioural change and improvements in service delivery. As little is known about the effectiveness of DT programs for community aged care workers, the proposed framework will provide an empirical and consistent method of evaluation, to assess their impact on enhancing older people's experience of healthcare. Copyright © 2017 Elsevier Ltd. All rights reserved.

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation

PubMed Central

Barasa, Edwine W.; Molyneux, Sassy; English, Mike; Cleary, Susan

2015-01-01

Background: Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices. Methods: We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken. Results: Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values. Conclusion: Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these

Setting Healthcare Priorities at the Macro and Meso Levels: A Framework for Evaluation.

PubMed

Barasa, Edwine W; Molyneux, Sassy; English, Mike; Cleary, Susan

2015-09-16

Priority setting in healthcare is a key determinant of health system performance. However, there is no widely accepted priority setting evaluation framework. We reviewed literature with the aim of developing and proposing a framework for the evaluation of macro and meso level healthcare priority setting practices. We systematically searched Econlit, PubMed, CINAHL, and EBSCOhost databases and supplemented this with searches in Google Scholar, relevant websites and reference lists of relevant papers. A total of 31 papers on evaluation of priority setting were identified. These were supplemented by broader theoretical literature related to evaluation of priority setting. A conceptual review of selected papers was undertaken. Based on a synthesis of the selected literature, we propose an evaluative framework that requires that priority setting practices at the macro and meso levels of the health system meet the following conditions: (1) Priority setting decisions should incorporate both efficiency and equity considerations as well as the following outcomes; (a) Stakeholder satisfaction, (b) Stakeholder understanding, (c) Shifted priorities (reallocation of resources), and (d) Implementation of decisions. (2) Priority setting processes should also meet the procedural conditions of (a) Stakeholder engagement, (b) Stakeholder empowerment, (c) Transparency, (d) Use of evidence, (e) Revisions, (f) Enforcement, and (g) Being grounded on community values. Available frameworks for the evaluation of priority setting are mostly grounded on procedural requirements, while few have included outcome requirements. There is, however, increasing recognition of the need to incorporate both consequential and procedural considerations in priority setting practices. In this review, we adapt an integrative approach to develop and propose a framework for the evaluation of priority setting practices at the macro and meso levels that draws from these complementary schools of thought. © 2015

Evaluating Comprehensive State Tobacco Prevention and Control Programs Using an Outcome Indicator Framework.

PubMed

Fulmer, Erika; Rogers, Todd; Glasgow, LaShawn; Brown, Susan; Kuiper, Nicole

2018-03-01

The outcome indicator framework helps tobacco prevention and control programs (TCPs) plan and implement theory-driven evaluations of their efforts to reduce and prevent tobacco use. Tobacco use is the single-most preventable cause of morbidity and mortality in the United States. The implementation of public health best practices by comprehensive state TCPs has been shown to prevent the initiation of tobacco use, reduce tobacco use prevalence, and decrease tobacco-related health care expenditures. Achieving and sustaining program goals require TCPs to evaluate the effectiveness and impact of their programs. To guide evaluation efforts by TCPs, the Centers for Disease Control and Prevention's Office on Smoking and Health developed an outcome indicator framework that includes a high-level logic model and evidence-based outcome indicators for each tobacco prevention and control goal area. In this article, we describe how TCPs and other community organizations can use the outcome indicator framework in their evaluation efforts. We also discuss how the framework is used at the national level to unify tobacco prevention and control efforts across varying state contexts, identify promising practices, and expand the public health evidence base.

Object-oriented fault tree evaluation program for quantitative analyses

NASA Technical Reports Server (NTRS)

Patterson-Hine, F. A.; Koen, B. V.

1988-01-01

Object-oriented programming can be combined with fault free techniques to give a significantly improved environment for evaluating the safety and reliability of large complex systems for space missions. Deep knowledge about system components and interactions, available from reliability studies and other sources, can be described using objects that make up a knowledge base. This knowledge base can be interrogated throughout the design process, during system testing, and during operation, and can be easily modified to reflect design changes in order to maintain a consistent information source. An object-oriented environment for reliability assessment has been developed on a Texas Instrument (TI) Explorer LISP workstation. The program, which directly evaluates system fault trees, utilizes the object-oriented extension to LISP called Flavors that is available on the Explorer. The object representation of a fault tree facilitates the storage and retrieval of information associated with each event in the tree, including tree structural information and intermediate results obtained during the tree reduction process. Reliability data associated with each basic event are stored in the fault tree objects. The object-oriented environment on the Explorer also includes a graphical tree editor which was modified to display and edit the fault trees.

Evaluation Framework and Analyses for Thermal Energy Storage Integrated with Packaged Air Conditioning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kung, F.; Deru, M.; Bonnema, E.

2013-10-01

Few third-party guidance documents or tools are available for evaluating thermal energy storage (TES) integrated with packaged air conditioning (AC), as this type of TES is relatively new compared to TES integrated with chillers or hot water systems. To address this gap, researchers at the National Renewable Energy Laboratory conducted a project to improve the ability of potential technology adopters to evaluate TES technologies. Major project outcomes included: development of an evaluation framework to describe key metrics, methodologies, and issues to consider when assessing the performance of TES systems integrated with packaged AC; application of multiple concepts from the evaluationmore » framework to analyze performance data from four demonstration sites; and production of a new simulation capability that enables modeling of TES integrated with packaged AC in EnergyPlus. This report includes the evaluation framework and analysis results from the project.« less

Examining Readers' Evaluations of Objectivity and Bias in News Discourse

ERIC Educational Resources Information Center

Cramer, Peter; Eisenhart, Christopher

2014-01-01

Readers' objectivity and bias evaluations of news texts were investigated in order to better understand the process by which readers make these kinds of judgments and the evidence on which they base them. Readers were primed to evaluate news texts for objectivity and bias, and their selections and metacommentary were analyzed. Readers detected…

Health policy in times of austerity-A conceptual framework for evaluating effects of policy on efficiency and equity illustrated with examples from Europe since 2008.

PubMed

Wenzl, Martin; Naci, Huseyin; Mossialos, Elias

2017-09-01

The objective of this paper is to provide a framework for evaluation of changes in health policy against overarching health system goals. We propose a categorisation of policies into seven distinct health system domains. We then develop existing analytical concepts of insurance coverage and cost-effectiveness further to evaluate the effects of policies in each domain on equity and efficiency. The framework is illustrated with likely effects of policy changes implemented in a sample of European countries since 2008. Our illustrative analysis suggests that cost containment has been the main focus and that countries have implemented a mix of measures that are efficient or efficiency neutral. Similarly, policies are likely to have mixed effects on equity. Additional user charges were a common theme but these were frequently accompanied by additional exemptions, making their likely effects on equity difficult to evaluate. We provide a framework for future, and more detailed, evaluations of changes in health policy. Copyright © 2017 The Authors. Published by Elsevier B.V. All rights reserved.

Development of an Object-Oriented Turbomachinery Analysis Code within the NPSS Framework

NASA Technical Reports Server (NTRS)

Jones, Scott M.

2014-01-01

During the preliminary or conceptual design phase of an aircraft engine, the turbomachinery designer has a need to estimate the effects of a large number of design parameters such as flow size, stage count, blade count, radial position, etc. on the weight and efficiency of a turbomachine. Computer codes are invariably used to perform this task however, such codes are often very old, written in outdated languages with arcane input files, and rarely adaptable to new architectures or unconventional layouts. Given the need to perform these kinds of preliminary design trades, a modern 2-D turbomachinery design and analysis code has been written using the Numerical Propulsion System Simulation (NPSS) framework. This paper discusses the development of the governing equations and the structure of the primary objects used in OTAC.

Evaluation of Conceptual Frameworks Applicable to the Study of Isolation Precautions Effectiveness

PubMed Central

Crawford, Catherine; Shang, Jingjing

2015-01-01

Aims A discussion of conceptual frameworks applicable to the study of isolation precautions effectiveness according to Fawcett and DeSanto-Madeya’s (2013) evaluation technique and their relative merits and drawbacks for this purpose Background Isolation precautions are recommended to control infectious diseases with high morbidity and mortality, but effectiveness is not established due to numerous methodological challenges. These challenges, such as identifying empirical indicators and refining operational definitions, could be alleviated though use of an appropriate conceptual framework. Design Discussion paper Data Sources In mid-April 2014, the primary author searched five electronic, scientific literature databases for conceptual frameworks applicable to study isolation precautions, without limiting searches by publication date. Implications for Nursing By reviewing promising conceptual frameworks to support isolation precautions effectiveness research, this paper exemplifies the process to choose an appropriate conceptual framework for empirical research. Hence, researchers may build on these analyses to improve study design of empirical research in multiple disciplines, which may lead to improved research and practice. Conclusion Three frameworks were reviewed: the epidemiologic triad of disease, Donabedian’s healthcare quality framework and the Quality Health Outcomes model. Each has been used in nursing research to evaluate health outcomes and contains concepts relevant to nursing domains. Which framework can be most useful likely depends on whether the study question necessitates testing multiple interventions, concerns pathogen-specific characteristics and yields cross-sectional or longitudinal data. The Quality Health Outcomes model may be slightly preferred as it assumes reciprocal relationships, multi-level analysis and is sensitive to cultural inputs. PMID:26179813

Evidence-Based Evaluation of Practice and Innovation in Physical Therapy Using the IDEAL-Physio Framework.

PubMed

Beard, David; Hamilton, David; Davies, Loretta; Cook, Jonathan; Hirst, Allison; McCulloch, Peter; Paez, Arsenio

2018-02-01

The IDEAL framework is an established method for initial and ongoing evaluations of innovation and practice for complex health care interventions. First derived for surgical sciences and embedded at a global level for evaluating surgery/surgical devices, the IDEAL framework is based on the principle that innovation and evaluation in clinical practice can, and should, evolve together in an ordered manner: from conception to development and then to validation by appropriate clinical studies and, finally, longer-term follow-up. This framework is highly suited to other complex, nonpharmacological interventions, such as physical therapist interventions. This perspective outlines the application of IDEAL to physical therapy in the new IDEAL-Physio framework. The IDEAL-Physio framework comprises 5 stages. In stage 1, the idea phase, formal data collection should begin. Stage 2a is the phase for iterative improvement and adjustment with thorough data recording. Stage 2b involves the onset of formal evaluation using systematically collected group or cohort data. Stage 3 is the phase for formal comparative assessment of treatment, usually involving randomized studies. Stage 4 involves long-term follow-up. The IDEAL-Physio framework is recommended as a method for guiding and evaluating both innovation and practice in physical therapy, with the overall goal of providing better evidence-based care. © 2017 American Physical Therapy Association.

Child Support Enforcement: A Framework for Evaluating Costs, Benefits, and Effects.

DTIC Science & Technology

1991-03-01

efforts to gain and enforce child support awards might yield additional collections on behalf of these children, they would surely entail additional...framework for evaluating the full costs and ne . effects of child support enforcement.I This framework could assist your office and others in planning...following results of our develop- S . ............. .. mental work: (1) models of the child support enforcement system activi- AvajiabilitY Codes. ties

An Evaluation Framework for Lossy Compression of Genome Sequencing Quality Values.

PubMed

Alberti, Claudio; Daniels, Noah; Hernaez, Mikel; Voges, Jan; Goldfeder, Rachel L; Hernandez-Lopez, Ana A; Mattavelli, Marco; Berger, Bonnie

2016-01-01

This paper provides the specification and an initial validation of an evaluation framework for the comparison of lossy compressors of genome sequencing quality values. The goal is to define reference data, test sets, tools and metrics that shall be used to evaluate the impact of lossy compression of quality values on human genome variant calling. The functionality of the framework is validated referring to two state-of-the-art genomic compressors. This work has been spurred by the current activity within the ISO/IEC SC29/WG11 technical committee (a.k.a. MPEG), which is investigating the possibility of starting a standardization activity for genomic information representation.

[Bio-objects and biological methods of space radiation effects evaluation].

PubMed

Kaminskaia, E V; Nevzgodina, L V; Platova, N G

2009-01-01

The unique conditions of space experiments place austere requirements to bio-objects and biological methods of radiation effects evaluation. The paper discusses suitability of a number of bio-objects varying in stage of evolution and metabolism for space researches aimed to state common patterns of the radiation damage caused by heavy ions (HI), and character of HI-cell interaction. Physical detectors in space experiments of the BIOBLOCK series make it possible to identify bio-objects hit by space HI and to set correlation between HI track topography and biological effect. The paper provides an all-round description of the bio-objects chosen for two BIOBLOCK experiments (population of hydrophyte Wolffia arrhiza (fam. duckweed) and Lactuca sativa seeds) and the method of evaluating effects from single space radiation HI. Direct effects of heavy ions on cells can be determined by the criteria of chromosomal aberrations and delayed morphologic abnormalities. The evaluation results are compared with the data about human blood lymphocytes. Consideration is being given to the procedures of test-objects' treatment and investigation.

Real-time tracking of visually attended objects in virtual environments and its application to LOD.

PubMed

Lee, Sungkil; Kim, Gerard Jounghyun; Choi, Seungmoon

2009-01-01

This paper presents a real-time framework for computationally tracking objects visually attended by the user while navigating in interactive virtual environments. In addition to the conventional bottom-up (stimulus-driven) saliency map, the proposed framework uses top-down (goal-directed) contexts inferred from the user's spatial and temporal behaviors, and identifies the most plausibly attended objects among candidates in the object saliency map. The computational framework was implemented using GPU, exhibiting high computational performance adequate for interactive virtual environments. A user experiment was also conducted to evaluate the prediction accuracy of the tracking framework by comparing objects regarded as visually attended by the framework to actual human gaze collected with an eye tracker. The results indicated that the accuracy was in the level well supported by the theory of human cognition for visually identifying single and multiple attentive targets, especially owing to the addition of top-down contextual information. Finally, we demonstrate how the visual attention tracking framework can be applied to managing the level of details in virtual environments, without any hardware for head or eye tracking.

Towards an IMU Evaluation Framework for Human Body Tracking.

PubMed

Venek, Verena; Kremser, Wolfgang; Schneider, Cornelia

2018-01-01

Existing full-body tracking systems, which use Inertial Measurement Units (IMUs) as sensing unit, require expert knowledge for setup and data collection. Thus, the daily application for human body tracking is difficult. In particular, in the field of active and assisted living (AAL), tracking human movements would enable novel insights not only into the quantity but also into the quality of human movement, for example by monitoring functional training. While the current market offers a wide range of products with vastly different properties, literature lacks guidelines for choosing IMUs for body tracking applications. Therefore, this paper introduces developments towards an IMU evaluation framework for human body tracking which compares IMUs against five requirement areas that consider device features and data quality. The data quality is assessed by conducting a static and a dynamic error analysis. In a first application to four IMUs of different component consumption, the IMU evaluation framework convinced as promising tool for IMU selection.

A novel framework to evaluate pedestrian safety at non-signalized locations.

PubMed

Fu, Ting; Miranda-Moreno, Luis; Saunier, Nicolas

2018-02-01

This paper proposes a new framework to evaluate pedestrian safety at non-signalized crosswalk locations. In the proposed framework, the yielding maneuver of a driver in response to a pedestrian is split into the reaction and braking time. Hence, the relationship of the distance required for a yielding maneuver and the approaching vehicle speed depends on the reaction time of the driver and deceleration rate that the vehicle can achieve. The proposed framework is represented in the distance-velocity (DV) diagram and referred as the DV model. The interactions between approaching vehicles and pedestrians showing the intention to cross are divided in three categories: i) situations where the vehicle cannot make a complete stop, ii) situations where the vehicle's ability to stop depends on the driver reaction time, and iii) situations where the vehicle can make a complete stop. Based on these classifications, non-yielding maneuvers are classified as "non-infraction non-yielding" maneuvers, "uncertain non-yielding" maneuvers and "non-yielding" violations, respectively. From the pedestrian perspective, crossing decisions are classified as dangerous crossings, risky crossings and safe crossings accordingly. The yielding compliance and yielding rate, as measures of the yielding behavior, are redefined based on these categories. Time to crossing and deceleration rate required for the vehicle to stop are used to measure the probability of collision. Finally, the framework is demonstrated through a case study in evaluating pedestrian safety at three different types of non-signalized crossings: a painted crosswalk, an unprotected crosswalk, and a crosswalk controlled by stop signs. Results from the case study suggest that the proposed framework works well in describing pedestrian-vehicle interactions which helps in evaluating pedestrian safety at non-signalized crosswalk locations. Copyright © 2017 Elsevier Ltd. All rights reserved.

Urban Partnership Agreement and Congestion Reduction Demonstration : National Evaluation Framework

DOT National Transportation Integrated Search

2008-11-21

This report provides an analytical framework for evaluating six deployments under the United States Department of Transportation (U.S. DOT) Urban Partnership Agreement (UPA) and Congestion Reduction Demonstration (CRD) Programs. The six UPA/CRD sites...

Standardization in the Handling and Evaluation of Objective Examinations.

ERIC Educational Resources Information Center

Sass, M. Burke

1978-01-01

In response to requests for standardization on testing and grading, a pilot program for the administration and evaluation of objective examinations was instituted. Outlined are objectives, initial test item collection, procedural flow for examinations, faculty responsibilities, support staff responsibilities, and project coordinator services. (LBH)

A common evaluation framework for the African Health Initiative.

PubMed

Bryce, Jennifer; Requejo, Jennifer Harris; Moulton, Lawrence H; Ram, Malathi; Black, Robert E

2013-01-01

The African Health Initiative includes highly diverse partnerships in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia), each of which is working to improve population health by strengthening health systems and to evaluate the results. One aim of the Initiative is to generate cross-site learning that can inform implementation in the five partnerships during the project period and identify lessons that may be generalizable to other countries in the region. Collaborators in the Initiative developed a common evaluation framework as a basis for this cross-site learning. This paper describes the components of the framework; this includes the conceptual model, core metrics to be measured in all sites, and standard guidelines for reporting on the implementation of partnership activities and contextual factors that may affect implementation, or the results it produces. We also describe the systems that have been put in place for data management, data quality assessments, and cross-site analysis of results. The conceptual model for the Initiative highlights points in the causal chain between health system strengthening activities and health impact where evidence produced by the partnerships can contribute to learning. This model represents an important advance over its predecessors by including contextual factors and implementation strength as potential determinants, and explicitly including equity as a component of both outcomes and impact. Specific measurement challenges include the prospective documentation of program implementation and contextual factors. Methodological issues addressed in the development of the framework include the aggregation of data collected using different methods and the challenge of evaluating a complex set of interventions being improved over time based on continuous monitoring and intermediate results.

A common evaluation framework for the African Health Initiative

PubMed Central

2013-01-01

Background The African Health Initiative includes highly diverse partnerships in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia), each of which is working to improve population health by strengthening health systems and to evaluate the results. One aim of the Initiative is to generate cross-site learning that can inform implementation in the five partnerships during the project period and identify lessons that may be generalizable to other countries in the region. Collaborators in the Initiative developed a common evaluation framework as a basis for this cross-site learning. Methods This paper describes the components of the framework; this includes the conceptual model, core metrics to be measured in all sites, and standard guidelines for reporting on the implementation of partnership activities and contextual factors that may affect implementation, or the results it produces. We also describe the systems that have been put in place for data management, data quality assessments, and cross-site analysis of results. Results and conclusions The conceptual model for the Initiative highlights points in the causal chain between health system strengthening activities and health impact where evidence produced by the partnerships can contribute to learning. This model represents an important advance over its predecessors by including contextual factors and implementation strength as potential determinants, and explicitly including equity as a component of both outcomes and impact. Specific measurement challenges include the prospective documentation of program implementation and contextual factors. Methodological issues addressed in the development of the framework include the aggregation of data collected using different methods and the challenge of evaluating a complex set of interventions being improved over time based on continuous monitoring and intermediate results. PMID:23819778

A framework for selecting indicators of bioenergy sustainability

DOE PAGES

Dale, Virginia H.; Efroymson, Rebecca Ann; Kline, Keith L.; ...

2015-05-11

A framework for selecting and evaluating indicators of bioenergy sustainability is presented. This framework is designed to facilitate decision-making about which indicators are useful for assessing sustainability of bioenergy systems and supporting their deployment. Efforts to develop sustainability indicators in the United States and Europe are reviewed. The first steps of the framework for indicator selection are defining the sustainability goals and other goals for a bioenergy project or program, gaining an understanding of the context, and identifying the values of stakeholders. From the goals, context, and stakeholders, the objectives for analysis and criteria for indicator selection can be developed.more » The user of the framework identifies and ranks indicators, applies them in an assessment, and then evaluates their effectiveness, while identifying gaps that prevent goals from being met, assessing lessons learned, and moving toward best practices. The framework approach emphasizes that the selection of appropriate criteria and indicators is driven by the specific purpose of an analysis. Realistic goals and measures of bioenergy sustainability can be developed systematically with the help of the framework presented here.« less

Introduction of blended learning in a master program: Developing an integrative mixed method evaluation framework.

PubMed

Chmiel, Aviva S; Shaha, Maya; Schneider, Daniel K

2017-01-01

The aim of this research is to develop a comprehensive evaluation framework involving all actors in a higher education blended learning (BL) program. BL evaluation usually either focuses on students, faculty, technological or institutional aspects. Currently, no validated comprehensive monitoring tool exists that can support introduction and further implementation of BL in a higher education context. Starting from established evaluation principles and standards, concepts that were to be evaluated were firstly identified and grouped. In a second step, related BL evaluation tools referring to students, faculty and institutional level were selected. This allowed setting up and implementing an evaluation framework to monitor the introduction of BL during two succeeding recurrences of the program. The results of the evaluation allowed documenting strengths and weaknesses of the BL format in a comprehensive way, involving all actors. It has led to improvements at program, faculty and course level. The evaluation process and the reporting of the results proved to be demanding in time and personal resources. The evaluation framework allows measuring the most significant dimensions influencing the success of a BL implementation at program level. However, this comprehensive evaluation is resource intensive. Further steps will be to refine the framework towards a sustainable and transferable BL monitoring tool that finds a balance between comprehensiveness and efficiency. Copyright © 2016 Elsevier Ltd. All rights reserved.

Using a Learning Progression Framework to Assess and Evaluate Student Growth

ERIC Educational Resources Information Center

Briggs, Derek C.; Diaz-Bilello, Elena; Peck, Fred; Alzen, Jessica; Chattergoon, Rajendra; Johnson, Raymond

2015-01-01

This report describes the use of a Learning Progression Framework (LPF) to support the Student Learning Objectives (SLO) process. The report highlights a few common threats we currently see in the SLO process implemented at various states and districts, and offers the LPF as a possible solution for addressing these threats. This report was…

Rapid Prototyping of an Aircraft Model in an Object-Oriented Simulation

NASA Technical Reports Server (NTRS)

Kenney, P. Sean

2003-01-01

A team was created to participate in the Mars Scout Opportunity. Trade studies determined that an aircraft provided the best opportunity to complete the science objectives of the team. A high fidelity six degree of freedom flight simulation was required to provide credible evidence that the aircraft design fulfilled mission objectives and to support the aircraft design process by providing performance evaluations. The team created the simulation using the Langley Standard Real-Time Simulation in C++ (LaSRS++) application framework. A rapid prototyping approach was necessary because the team had only three months to both develop the aircraft simulation model and evaluate aircraft performance as the design and mission parameters matured. The design of LaSRS++ enabled rapid-prototyping in several ways. First, the framework allowed component models to be designed, implemented, unit-tested, and integrated quickly. Next, the framework provides a highly reusable infrastructure that allowed developers to maximize code reuse while concentrating on aircraft and mission specific features. Finally, the framework reduces risk by providing reusable components that allow developers to build a quality product with a compressed testing cycle that relies heavily on unit testing of new components.

The training for health equity network evaluation framework: a pilot study at five health professional schools.

PubMed

Ross, Simone J; Preston, Robyn; Lindemann, Iris C; Matte, Marie C; Samson, Rex; Tandinco, Filedito D; Larkins, Sarah L; Palsdottir, Bjorg; Neusy, Andre-Jacques

2014-01-01

The Training for Health Equity Network (THEnet), a group of diverse health professional schools aspiring toward social accountability, developed and pilot tested a comprehensive evaluation framework to assess progress toward socially accountable health professions education. The evaluation framework provides criteria for schools to assess their level of social accountability within their organization and planning; education, research and service delivery; and the direct and indirect impacts of the school and its graduates, on the community and health system. This paper describes the pilot implementation of testing the evaluation framework across five THEnet schools, and examines whether the evaluation framework was practical and feasible across contexts for the purposes of critical reflection and continuous improvement in terms of progress towards social accountability. In this pilot study, schools utilized the evaluation framework using a mixed method approach of data collection comprising of workshops, qualitative interviews and focus group discussions, document review and collation and analysis of existing quantitative data. The evaluation framework allowed each school to contextually gather evidence on how it was meeting the aspirational goals of social accountability across a range of school activities, and to identify strengths and areas for improvement and development. The evaluation framework pilot study demonstrated how social accountability can be assessed through a critically reflective and comprehensive process. As social accountability focuses on the relationship between health professions schools and health system and health population outcomes, each school was able to demonstrate to students, health professionals, governments, accrediting bodies, communities and other stakeholders how current and future health care needs of populations are addressed in terms of education, research, and service learning.

Towards a hierarchical optimization modeling framework for ...

EPA Pesticide Factsheets

Background:Bilevel optimization has been recognized as a 2-player Stackelberg game where players are represented as leaders and followers and each pursue their own set of objectives. Hierarchical optimization problems, which are a generalization of bilevel, are especially difficult because the optimization is nested, meaning that the objectives of one level depend on solutions to the other levels. We introduce a hierarchical optimization framework for spatially targeting multiobjective green infrastructure (GI) incentive policies under uncertainties related to policy budget, compliance, and GI effectiveness. We demonstrate the utility of the framework using a hypothetical urban watershed, where the levels are characterized by multiple levels of policy makers (e.g., local, regional, national) and policy followers (e.g., landowners, communities), and objectives include minimization of policy cost, implementation cost, and risk; reduction of combined sewer overflow (CSO) events; and improvement in environmental benefits such as reduced nutrient run-off and water availability. Conclusions: While computationally expensive, this hierarchical optimization framework explicitly simulates the interaction between multiple levels of policy makers (e.g., local, regional, national) and policy followers (e.g., landowners, communities) and is especially useful for constructing and evaluating environmental and ecological policy. Using the framework with a hypothetical urba

A unified framework for evaluating the risk of re-identification of text de-identification tools.

PubMed

Scaiano, Martin; Middleton, Grant; Arbuckle, Luk; Kolhatkar, Varada; Peyton, Liam; Dowling, Moira; Gipson, Debbie S; El Emam, Khaled

2016-10-01

It has become regular practice to de-identify unstructured medical text for use in research using automatic methods, the goal of which is to remove patient identifying information to minimize re-identification risk. The metrics commonly used to determine if these systems are performing well do not accurately reflect the risk of a patient being re-identified. We therefore developed a framework for measuring the risk of re-identification associated with textual data releases. We apply the proposed evaluation framework to a data set from the University of Michigan Medical School. Our risk assessment results are then compared with those that would be obtained using a typical contemporary micro-average evaluation of recall in order to illustrate the difference between the proposed evaluation framework and the current baseline method. We demonstrate how this framework compares against common measures of the re-identification risk associated with an automated text de-identification process. For the probability of re-identification using our evaluation framework we obtained a mean value for direct identifiers of 0.0074 and a mean value for quasi-identifiers of 0.0022. The 95% confidence interval for these estimates were below the relevant thresholds. The threshold for direct identifier risk was based on previously used approaches in the literature. The threshold for quasi-identifiers was determined based on the context of the data release following commonly used de-identification criteria for structured data. Our framework attempts to correct for poorly distributed evaluation corpora, accounts for the data release context, and avoids the often optimistic assumptions that are made using the more traditional evaluation approach. It therefore provides a more realistic estimate of the true probability of re-identification. This framework should be used as a basis for computing re-identification risk in order to more realistically evaluate future text de-identification tools

Matching methods evaluation framework for stereoscopic breast x-ray images.

PubMed

Rousson, Johanna; Naudin, Mathieu; Marchessoux, Cédric

2016-01-01

Three-dimensional (3-D) imaging has been intensively studied in the past few decades. Depth information is an important added value of 3-D systems over two-dimensional systems. Special focuses were devoted to the development of stereo matching methods for the generation of disparity maps (i.e., depth information within a 3-D scene). Dedicated frameworks were designed to evaluate and rank the performance of different stereo matching methods but never considering x-ray medical images. Yet, 3-D x-ray acquisition systems and 3-D medical displays have already been introduced into the diagnostic market. To access the depth information within x-ray stereoscopic images, computing accurate disparity maps is essential. We aimed at developing a framework dedicated to x-ray stereoscopic breast images used to evaluate and rank several stereo matching methods. A multiresolution pyramid optimization approach was integrated to the framework to increase the accuracy and the efficiency of the stereo matching techniques. Finally, a metric was designed to score the results of the stereo matching compared with the ground truth. Eight methods were evaluated and four of them [locally scaled sum of absolute differences (LSAD), zero mean sum of absolute differences, zero mean sum of squared differences, and locally scaled mean sum of squared differences] appeared to perform equally good with an average error score of 0.04 (0 is the perfect matching). LSAD was selected for generating the disparity maps.

CognitionMaster: an object-based image analysis framework

PubMed Central

2013-01-01

Background Automated image analysis methods are becoming more and more important to extract and quantify image features in microscopy-based biomedical studies and several commercial or open-source tools are available. However, most of the approaches rely on pixel-wise operations, a concept that has limitations when high-level object features and relationships between objects are studied and if user-interactivity on the object-level is desired. Results In this paper we present an open-source software that facilitates the analysis of content features and object relationships by using objects as basic processing unit instead of individual pixels. Our approach enables also users without programming knowledge to compose “analysis pipelines“ that exploit the object-level approach. We demonstrate the design and use of example pipelines for the immunohistochemistry-based cell proliferation quantification in breast cancer and two-photon fluorescence microscopy data about bone-osteoclast interaction, which underline the advantages of the object-based concept. Conclusions We introduce an open source software system that offers object-based image analysis. The object-based concept allows for a straight-forward development of object-related interactive or fully automated image analysis solutions. The presented software may therefore serve as a basis for various applications in the field of digital image analysis. PMID:23445542

Attitudes as Object-Evaluation Associations of Varying Strength

PubMed Central

Fazio, Russell H.

2009-01-01

Historical developments regarding the attitude concept are reviewed, and set the stage for consideration of a theoretical perspective that views attitude, not as a hypothetical construct, but as evaluative knowledge. A model of attitudes as object-evaluation associations of varying strength is summarized, along with research supporting the model’s contention that at least some attitudes are represented in memory and activated automatically upon the individual’s encountering the attitude object. The implications of the theoretical perspective for a number of recent discussions related to the attitude concept are elaborated. Among these issues are the notion of attitudes as “constructions,” the presumed malleability of automatically-activated attitudes, correspondence between implicit and explicit measures of attitude, and postulated dual or multiple attitudes. PMID:19424447

Learning of perceptual grouping for object segmentation on RGB-D data☆

PubMed Central

Richtsfeld, Andreas; Mörwald, Thomas; Prankl, Johann; Zillich, Michael; Vincze, Markus

2014-01-01

Object segmentation of unknown objects with arbitrary shape in cluttered scenes is an ambitious goal in computer vision and became a great impulse with the introduction of cheap and powerful RGB-D sensors. We introduce a framework for segmenting RGB-D images where data is processed in a hierarchical fashion. After pre-clustering on pixel level parametric surface patches are estimated. Different relations between patch-pairs are calculated, which we derive from perceptual grouping principles, and support vector machine classification is employed to learn Perceptual Grouping. Finally, we show that object hypotheses generation with Graph-Cut finds a globally optimal solution and prevents wrong grouping. Our framework is able to segment objects, even if they are stacked or jumbled in cluttered scenes. We also tackle the problem of segmenting objects when they are partially occluded. The work is evaluated on publicly available object segmentation databases and also compared with state-of-the-art work of object segmentation. PMID:24478571

An open-source framework for large-scale, flexible evaluation of biomedical text mining systems.

PubMed

Baumgartner, William A; Cohen, K Bretonnel; Hunter, Lawrence

2008-01-29

Improved evaluation methodologies have been identified as a necessary prerequisite to the improvement of text mining theory and practice. This paper presents a publicly available framework that facilitates thorough, structured, and large-scale evaluations of text mining technologies. The extensibility of this framework and its ability to uncover system-wide characteristics by analyzing component parts as well as its usefulness for facilitating third-party application integration are demonstrated through examples in the biomedical domain. Our evaluation framework was assembled using the Unstructured Information Management Architecture. It was used to analyze a set of gene mention identification systems involving 225 combinations of system, evaluation corpus, and correctness measure. Interactions between all three were found to affect the relative rankings of the systems. A second experiment evaluated gene normalization system performance using as input 4,097 combinations of gene mention systems and gene mention system-combining strategies. Gene mention system recall is shown to affect gene normalization system performance much more than does gene mention system precision, and high gene normalization performance is shown to be achievable with remarkably low levels of gene mention system precision. The software presented in this paper demonstrates the potential for novel discovery resulting from the structured evaluation of biomedical language processing systems, as well as the usefulness of such an evaluation framework for promoting collaboration between developers of biomedical language processing technologies. The code base is available as part of the BioNLP UIMA Component Repository on SourceForge.net.

An innovative strategy in evaluation: using a student engagement framework to evaluate a role-based simulation.

PubMed

Smith, Morgan; Warland, Jane; Smith, Colleen

2012-03-01

Online role-play has the potential to actively engage students in authentic learning experiences and help develop their clinical reasoning skills. However, evaluation of student learning for this kind of simulation focuses mainly on the content and outcome of learning, rather than on the process of learning through student engagement. This article reports on the use of a student engagement framework to evaluate an online role-play offered as part of a course in Bachelor of Nursing and Bachelor of Midwifery programs. Instruments that measure student engagement to date have targeted large numbers of students at program and institutional levels, rather than at the level of a specific learning activity. Although the framework produced some useful findings for evaluation purposes, further refinement of the questions is required to be certain that deep learning results from the engagement that occurs with course-level learning initiatives. Copyright 2012, SLACK Incorporated.

Development of a Dynamically Configurable,Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation

NASA Technical Reports Server (NTRS)

Afjeh, Abdollah A.; Reed, John A.

2003-01-01

This research is aimed at developing a neiv and advanced simulation framework that will significantly improve the overall efficiency of aerospace systems design and development. This objective will be accomplished through an innovative integration of object-oriented and Web-based technologies ivith both new and proven simulation methodologies. The basic approach involves Ihree major areas of research: Aerospace system and component representation using a hierarchical object-oriented component model which enables the use of multimodels and enforces component interoperability. Collaborative software environment that streamlines the process of developing, sharing and integrating aerospace design and analysis models. . Development of a distributed infrastructure which enables Web-based exchange of models to simplify the collaborative design process, and to support computationally intensive aerospace design and analysis processes. Research for the first year dealt with the design of the basic architecture and supporting infrastructure, an initial implementation of that design, and a demonstration of its application to an example aircraft engine system simulation.

Risks posed by climate change to the delivery of Water Framework Directive objectives in the UK.

PubMed

Wilby, R L; Orr, H G; Hedger, M; Forrow, D; Blackmore, M

2006-12-01

The EU Water Framework Directive (WFD) is novel because it integrates water quality, water resources, physical habitat and, to some extent, flooding for all surface and groundwaters and takes forward river basin management. However, the WFD does not explicitly mention risks posed by climate change to the achievement of its environmental objectives. This is despite the fact that the time scale for the implementation process and achieving particular objectives extends into the 2020s, when climate models project changes in average temperature and precipitation. This paper begins by reviewing the latest UK climate change scenarios and the wider policy and science context of the WFD. We then examine the potential risks of climate change to key phases of the River Basin Management Process that underpin the WFD (such as characterisation of river basins and their water bodies, risk assessments to identify pressures and impacts, programmes of measures (POMs) options appraisal, monitoring and modelling, policy and management activities). Despite these risks the WFD could link new policy and participative mechanisms (being established for the River Basin Management Plans) to the emerging framework of national and regional climate change adaptation policy. The risks are identified with a view to informing policy opportunities, objective setting, adaptation strategies and the research agenda. Key knowledge gaps have already been identified during the implementation of the WFD, such as the links between hydromorphology and ecosystem status, but the overarching importance of linking climate change to these considerations needs to be highlighted. The next generation of (probabilistic) climate change scenarios will present new opportunities and challenges for risk analysis and policy-making.

Generalizable open source urban water portfolio simulation framework demonstrated using a multi-objective risk-based planning benchmark problem.

NASA Astrophysics Data System (ADS)

Trindade, B. C.; Reed, P. M.

2017-12-01

The growing access and reduced cost for computing power in recent years has promoted rapid development and application of multi-objective water supply portfolio planning. As this trend continues there is a pressing need for flexible risk-based simulation frameworks and improved algorithm benchmarking for emerging classes of water supply planning and management problems. This work contributes the Water Utilities Management and Planning (WUMP) model: a generalizable and open source simulation framework designed to capture how water utilities can minimize operational and financial risks by regionally coordinating planning and management choices, i.e. making more efficient and coordinated use of restrictions, water transfers and financial hedging combined with possible construction of new infrastructure. We introduce the WUMP simulation framework as part of a new multi-objective benchmark problem for planning and management of regionally integrated water utility companies. In this problem, a group of fictitious water utilities seek to balance the use of the mentioned reliability driven actions (e.g., restrictions, water transfers and infrastructure pathways) and their inherent financial risks. Several traits of this problem make it ideal for a benchmark problem, namely the presence of (1) strong non-linearities and discontinuities in the Pareto front caused by the step-wise nature of the decision making formulation and by the abrupt addition of storage through infrastructure construction, (2) noise due to the stochastic nature of the streamflows and water demands, and (3) non-separability resulting from the cooperative formulation of the problem, in which decisions made by stakeholder may substantially impact others. Both the open source WUMP simulation framework and its demonstration in a challenging benchmarking example hold value for promoting broader advances in urban water supply portfolio planning for regions confronting change.

Learning Objects and Virtual Learning Environments Technical Evaluation Criteria

ERIC Educational Resources Information Center

Kurilovas, Eugenijus; Dagiene, Valentina

2009-01-01

The main scientific problems investigated in this article deal with technical evaluation of quality attributes of the main components of e-Learning systems (referred here as DLEs--Digital Libraries of Educational Resources and Services), i.e., Learning Objects (LOs) and Virtual Learning Environments (VLEs). The main research object of the work is…

Litigation and complaints procedures: objectives, effectiveness and alternatives.

PubMed Central

Whelan, C J

1988-01-01

Recent debates about redress mechanisms for medical accident victims have been sidetracked by fears of an American-style medical malpractice crisis. What is required is a framework within which the debate can resume. This paper proposes such a framework by focusing on the compensation and deterrence objectives and placing them in the wider context of the social costs of providing medical services. The framework is then used to assess and compare the effectiveness of differing approaches. In particular, the American and British experiences of litigation, including the concept of 'defensive medicine', are evaluated. Also discussed briefly are alternatives to court-based complaints procedures including 'no-fault' schemes, professional ethics and internal complaints mechanisms. PMID:3392721

A framework for evaluating complex networks measurements

NASA Astrophysics Data System (ADS)

Comin, Cesar H.; Silva, Filipi N.; Costa, Luciano da F.

2015-06-01

A good deal of current research in complex networks involves the characterization and/or classification of the topological properties of given structures, which has motivated several respective measurements. This letter proposes a framework for evaluating the quality of complex-network measurements in terms of their effective resolution, degree of degeneracy and discriminability. The potential of the suggested approach is illustrated with respect to comparing the characterization of several model and real-world networks by using concentric and symmetry measurements. The results indicate a markedly superior performance for the latter type of mapping.

An Ecosystem Evaluation Framework for Global Seamount Conservation and Management

PubMed Central

Taranto, Gerald H.; Kvile, Kristina Ø.; Pitcher, Tony J.; Morato, Telmo

2012-01-01

In the last twenty years, several global targets for protection of marine biodiversity have been adopted but have failed. The Convention on Biological Diversity (CBD) aims at preserving 10% of all the marine biomes by 2020. For achieving this goal, ecologically or biologically significant areas (EBSA) have to be identified in all biogeographic regions. However, the methodologies for identifying the best suitable areas are still to be agreed. Here, we propose a framework for applying the CBD criteria to locate potential ecologically or biologically significant seamount areas based on the best information currently available. The framework combines the likelihood of a seamount constituting an EBSA and its level of human impact and can be used at global, regional and local scales. This methodology allows the classification of individual seamounts into four major portfolio conservation categories which can help optimize management efforts toward the protection of the most suitable areas. The framework was tested against 1000 dummy seamounts and satisfactorily assigned seamounts to proper EBSA and threats categories. Additionally, the framework was applied to eight case study seamounts that were included in three out of four portfolio categories: areas highly likely to be identified as EBSA with high degree of threat; areas highly likely to be EBSA with low degree of threat; and areas with a low likelihood of being EBSA with high degree of threat. This framework will allow managers to identify seamount EBSAs and to prioritize their policies in terms of protecting undisturbed areas, disturbed areas for recovery of habitats and species, or both based on their management objectives. It also identifies seamount EBSAs and threats considering different ecological groups in both pelagic and benthic communities. Therefore, this framework may represent an important tool to mitigate seamount biodiversity loss and to achieve the 2020 CBD goals. PMID:22905190

An ecosystem evaluation framework for global seamount conservation and management.

PubMed

Taranto, Gerald H; Kvile, Kristina Ø; Pitcher, Tony J; Morato, Telmo

2012-01-01

In the last twenty years, several global targets for protection of marine biodiversity have been adopted but have failed. The Convention on Biological Diversity (CBD) aims at preserving 10% of all the marine biomes by 2020. For achieving this goal, ecologically or biologically significant areas (EBSA) have to be identified in all biogeographic regions. However, the methodologies for identifying the best suitable areas are still to be agreed. Here, we propose a framework for applying the CBD criteria to locate potential ecologically or biologically significant seamount areas based on the best information currently available. The framework combines the likelihood of a seamount constituting an EBSA and its level of human impact and can be used at global, regional and local scales. This methodology allows the classification of individual seamounts into four major portfolio conservation categories which can help optimize management efforts toward the protection of the most suitable areas. The framework was tested against 1000 dummy seamounts and satisfactorily assigned seamounts to proper EBSA and threats categories. Additionally, the framework was applied to eight case study seamounts that were included in three out of four portfolio categories: areas highly likely to be identified as EBSA with high degree of threat; areas highly likely to be EBSA with low degree of threat; and areas with a low likelihood of being EBSA with high degree of threat. This framework will allow managers to identify seamount EBSAs and to prioritize their policies in terms of protecting undisturbed areas, disturbed areas for recovery of habitats and species, or both based on their management objectives. It also identifies seamount EBSAs and threats considering different ecological groups in both pelagic and benthic communities. Therefore, this framework may represent an important tool to mitigate seamount biodiversity loss and to achieve the 2020 CBD goals.

SurF: an innovative framework in biosecurity and animal health surveillance evaluation.

PubMed

Muellner, Petra; Watts, Jonathan; Bingham, Paul; Bullians, Mark; Gould, Brendan; Pande, Anjali; Riding, Tim; Stevens, Paul; Vink, Daan; Stärk, Katharina Dc

2018-05-16

Surveillance for biosecurity hazards is being conducted by the New Zealand Competent Authority, the Ministry for Primary Industries (MPI) to support New Zealand's biosecurity system. Surveillance evaluation should be an integral part of the surveillance life cycle, as it provides a means to identify and correct problems and to sustain and enhance the existing strengths of a surveillance system. The surveillance evaluation Framework (SurF) presented here was developed to provide a generic framework within which the MPI biosecurity surveillance portfolio, and all of its components, can be consistently assessed. SurF is an innovative, cross-sectoral effort that aims to provide a common umbrella for surveillance evaluation in the animal, plant, environment and aquatic sectors. It supports the conduct of the following four distinct components of an evaluation project: (i) motivation for the evaluation, (ii) scope of the evaluation, (iii) evaluation design and implementation and (iv) reporting and communication of evaluation outputs. Case studies, prepared by MPI subject matter experts, are included in the framework to guide users in their assessment. Three case studies were used in the development of SurF in order to assure practical utility and to confirm usability of SurF across all included sectors. It is anticipated that the structured approach and information provided by SurF will not only be of benefit to MPI but also to other New Zealand stakeholders. Although SurF was developed for internal use by MPI, it could be applied to any surveillance system in New Zealand or elsewhere. © 2018 2018 The Authors. Transboundary and Emerging Diseases Published by Blackwell Verlag GmbH.

A framework for evaluating electronic health record vendor user-centered design and usability testing processes.

PubMed

Ratwani, Raj M; Zachary Hettinger, A; Kosydar, Allison; Fairbanks, Rollin J; Hodgkins, Michael L

2017-04-01

Currently, there are few resources for electronic health record (EHR) purchasers and end users to understand the usability processes employed by EHR vendors during product design and development. We developed a framework, based on human factors literature and industry standards, to systematically evaluate the user-centered design processes and usability testing methods used by EHR vendors. We reviewed current usability certification requirements and the human factors literature to develop a 15-point framework for evaluating EHR products. The framework is based on 3 dimensions: user-centered design process, summative testing methodology, and summative testing results. Two vendor usability reports were retrieved from the Office of the National Coordinator's Certified Health IT Product List and were evaluated using the framework. One vendor scored low on the framework (5 pts) while the other vendor scored high on the framework (15 pts). The 2 scored vendor reports demonstrate the framework's ability to discriminate between the variabilities in vendor processes and to determine which vendors are meeting best practices. The framework provides a method to more easily comprehend EHR vendors' usability processes and serves to highlight where EHR vendors may be falling short in terms of best practices. The framework provides a greater level of transparency for both purchasers and end users of EHRs. The framework highlights the need for clearer certification requirements and suggests that the authorized certification bodies that examine vendor usability reports may need to be provided with clearer guidance. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework.

PubMed

Wolterink, Jelmer M; Leiner, Tim; de Vos, Bob D; Coatrieux, Jean-Louis; Kelm, B Michael; Kondo, Satoshi; Salgado, Rodrigo A; Shahzad, Rahil; Shu, Huazhong; Snoeren, Miranda; Takx, Richard A P; van Vliet, Lucas J; van Walsum, Theo; Willems, Tineke P; Yang, Guanyu; Zheng, Yefeng; Viergever, Max A; Išgum, Ivana

2016-05-01

The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiac CT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiac CT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiac CT using a publicly available standardized framework. Cardiac CT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CT scanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient. Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen's kappa

An open-source framework for large-scale, flexible evaluation of biomedical text mining systems

PubMed Central

Baumgartner, William A; Cohen, K Bretonnel; Hunter, Lawrence

2008-01-01

Background Improved evaluation methodologies have been identified as a necessary prerequisite to the improvement of text mining theory and practice. This paper presents a publicly available framework that facilitates thorough, structured, and large-scale evaluations of text mining technologies. The extensibility of this framework and its ability to uncover system-wide characteristics by analyzing component parts as well as its usefulness for facilitating third-party application integration are demonstrated through examples in the biomedical domain. Results Our evaluation framework was assembled using the Unstructured Information Management Architecture. It was used to analyze a set of gene mention identification systems involving 225 combinations of system, evaluation corpus, and correctness measure. Interactions between all three were found to affect the relative rankings of the systems. A second experiment evaluated gene normalization system performance using as input 4,097 combinations of gene mention systems and gene mention system-combining strategies. Gene mention system recall is shown to affect gene normalization system performance much more than does gene mention system precision, and high gene normalization performance is shown to be achievable with remarkably low levels of gene mention system precision. Conclusion The software presented in this paper demonstrates the potential for novel discovery resulting from the structured evaluation of biomedical language processing systems, as well as the usefulness of such an evaluation framework for promoting collaboration between developers of biomedical language processing technologies. The code base is available as part of the BioNLP UIMA Component Repository on SourceForge.net. PMID:18230184

Electronic immunization data collection systems: application of an evaluation framework.

PubMed

Heidebrecht, Christine L; Kwong, Jeffrey C; Finkelstein, Michael; Quan, Sherman D; Pereira, Jennifer A; Quach, Susan; Deeks, Shelley L

2014-01-14

Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units. The Centers for Disease Control and Prevention's Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations. Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers' immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform. Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.

Objective speech quality evaluation of real-time speech coders

NASA Astrophysics Data System (ADS)

Viswanathan, V. R.; Russell, W. H.; Huggins, A. W. F.

1984-02-01

This report describes the work performed in two areas: subjective testing of a real-time 16 kbit/s adaptive predictive coder (APC) and objective speech quality evaluation of real-time coders. The speech intelligibility of the APC coder was tested using the Diagnostic Rhyme Test (DRT), and the speech quality was tested using the Diagnostic Acceptability Measure (DAM) test, under eight operating conditions involving channel error, acoustic background noise, and tandem link with two other coders. The test results showed that the DRT and DAM scores of the APC coder equalled or exceeded the corresponding test scores fo the 32 kbit/s CVSD coder. In the area of objective speech quality evaluation, the report describes the development, testing, and validation of a procedure for automatically computing several objective speech quality measures, given only the tape-recordings of the input speech and the corresponding output speech of a real-time speech coder.

Validity evidence for the Simulated Colonoscopy Objective Performance Evaluation scoring system.

PubMed

Trinca, Kristen D; Cox, Tiffany C; Pearl, Jonathan P; Ritter, E Matthew

2014-02-01

Low-cost, objective systems to assess and train endoscopy skills are needed. The aim of this study was to evaluate the ability of Simulated Colonoscopy Objective Performance Evaluation to assess the skills required to perform endoscopy. Thirty-eight subjects were included in this study, all of whom performed 4 tasks. The scoring system measured performance by calculating precision and efficiency. Data analysis assessed the relationship between colonoscopy experience and performance on each task and the overall score. Endoscopic trainees' Simulated Colonoscopy Objective Performance Evaluation scores correlated significantly with total colonoscopy experience (r = .61, P = .003) and experience in the past 12 months (r = .63, P = .002). Significant differences were seen among practicing endoscopists, nonendoscopic surgeons, and trainees (P < .0001). When the 4 tasks were analyzed, each showed significant correlation with colonoscopy experience (scope manipulation, r = .44, P = .044; tool targeting, r = .45, P = .04; loop management, r = .47, P = .032; mucosal inspection, r = .65, P = .001) and significant differences in performance between the endoscopist groups, except for mucosal inspection (scope manipulation, P < .0001; tool targeting, P = .002; loop management, P = .0008; mucosal inspection, P = .27). Simulated Colonoscopy Objective Performance Evaluation objectively assesses the technical skills required to perform endoscopy and shows promise as a platform for proficiency-based skills training. Published by Elsevier Inc.

The Climate Change Education Evidence Base: Lessons Learned from NOAA's Monitoring and Evaluation Framework Implementation

NASA Astrophysics Data System (ADS)

Baek, J.

2012-12-01

Federal science mission agencies are under increased pressure to ensure that their STEM education investments accomplish several objectives, including the identification and use of evidence-based approaches. Climate change education and climate literacy programs fall under these broader STEM initiatives. This paper is designed as a primer for climate change education evaluators and researchers to understand the policy context on the use of evidence. Recent initiatives, that include the National Science Foundation (NSF), the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration (NOAA), point to a need for shared goals and measurements amongst the climate change education community. The Tri-agency Climate Change Education (CCE) collaboration, which includes NSF, NASA, and NOAA, developed the Tri-Agency Climate Change Education Common Evaluation Framework Initiative Stakeholder Statement (2012). An excerpt: From the perspective of the tri-agency collaboration, and its individual agency members, the goal of the common framework is not to build a required evaluation scheme or a set of new requirements for our funded climate change education initiatives. Rather, the collaboration would be strengthened by the development of a framework that includes tools, instruments, and/or documentation to: ● Help the agencies see and articulate the relationships between the individual pieces of the tri-agency CCE portfolio; ● Guide the agencies in reporting on the progress, lessons learned, and impacts of the collaboration between the three agencies in developing a coordinated portfolio of climate education initiatives; and ● Help the individual projects, as part of this broader portfolio, understand where they fit into a larger picture. The accomplishments of this initiative to date have been based on the collaborative nature of evaluators the climate change education community within the tri-agency portfolio. While this

Framework of behavioral indicators evaluating TB health promotion outcomes: a modified Delphi study of TB policymakers and health workers.

PubMed

Li, Ying; Ehiri, John; Hu, Daiyu; Oren, Eyal; Cao, Jia

2015-12-15

Although TB health promotion directed at policy makers and healthcare workers (HCWs) is considered important to tuberculosis (TB) control, no indicators currently assess the impact of such promotional activities. This article is the second in a series of papers that seek to establish a framework of behavioral indicators for outcome evaluation of TB health promotion, using the Delphi method. In the first article, we sought to establish a framework of behavioral indicators for outcome evaluation of TB health promotion among TB suspects and patients. The objective of this second article is to present an indicator framework that can be used to assess behavioral outcomes of TB health promotion directed at policy makers and HCWs. A two-round, modified Delphi method was used to establish the indicators. Sixteen experts who were knowledgeable and experienced in the field of TB control were consulted in Delphi surveys. A questionnaire was developed following 4 steps, and involved ranking indicators on a five-point Likert scale. The consensus level was 70 %. Median, mode, and Coefficient of variation (CV) were used to describe expert responses. An authority coefficient (Cr) was used to assess the degree of each expert's authority. Consensus was achieved following the two survey rounds and several iterations among the experts. For TB health-promotion activities directed at policymakers, the experts reached consensus on 2 domains ("Resource inputs" and "Policymaking and monitoring behaviors"), 4 subdomains ("Human resources" among others), and 13 indicators ("Human resources per 100,000 person" among others). For TB health-promotion activities directed at HCWs, the experts reached consensus on 5 domains ("Self-protective behaviors" among others), 6 sub-domains ("Preventing infection" among others), and 15 indicators ("Average hours of daily workplace disinfection by ultraviolet radiation" among others). This study identified a conceptual framework of core behavioral indicators

A framework for evaluating forest landscape model predictions using empirical data and knowledge

Treesearch

Wen J. Wang; Hong S. He; Martin A. Spetich; Stephen R. Shifley; Frank R. Thompson; William D. Dijak; Qia Wang

2014-01-01

Evaluation of forest landscape model (FLM) predictions is indispensable to establish the credibility of predictions. We present a framework that evaluates short- and long-term FLM predictions at site and landscape scales. Site-scale evaluation is conducted through comparing raster cell-level predictions with inventory plot data whereas landscape-scale evaluation is...

A Modified Importance-Performance Framework for Evaluating Recreation-Based Experiential Learning Programs

ERIC Educational Resources Information Center

Pitas, Nicholas; Murray, Alison; Olsen, Max; Graefe, Alan

2017-01-01

This article describes a modified importance-performance framework for use in evaluation of recreation-based experiential learning programs. Importance-performance analysis (IPA) provides an effective and readily applicable means of evaluating many programs, but the near universal satisfaction associated with recreation inhibits the use of IPA in…

A conceptual framework for evaluating data suitability for observational studies.

PubMed

Shang, Ning; Weng, Chunhua; Hripcsak, George

2017-09-08

To contribute a conceptual framework for evaluating data suitability to satisfy the research needs of observational studies. Suitability considerations were derived from a systematic literature review on researchers' common data needs in observational studies and a scoping review on frequent clinical database design considerations, and were harmonized to construct a suitability conceptual framework using a bottom-up approach. The relationships among the suitability categories are explored from the perspective of 4 facets of data: intrinsic, contextual, representational, and accessible. A web-based national survey of domain experts was conducted to validate the framework. Data suitability for observational studies hinges on the following key categories: Explicitness of Policy and Data Governance, Relevance, Availability of Descriptive Metadata and Provenance Documentation, Usability, and Quality. We describe 16 measures and 33 sub-measures. The survey uncovered the relevance of all categories, with a 5-point Likert importance score of 3.9 ± 1.0 for Explicitness of Policy and Data Governance, 4.1 ± 1.0 for Relevance, 3.9 ± 0.9 for Availability of Descriptive Metadata and Provenance Documentation, 4.2 ± 1.0 for Usability, and 4.0 ± 0.9 for Quality. The suitability framework evaluates a clinical data source's fitness for research use. Its construction reflects both researchers' points of view and data custodians' design features. The feedback from domain experts rated Usability, Relevance, and Quality categories as the most important considerations. © The Author 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Critical evaluation of international health programs: Reframing global health and evaluation.

PubMed

Chi, Chunhuei; Tuepker, Anaïs; Schoon, Rebecca; Núñez Mondaca, Alicia

2018-04-01

Striking changes in the funding and implementation of international health programs in recent decades have stimulated debate about the role of communities in deciding which health programs to implement. An important yet neglected piece of that discussion is the need to change norms in program evaluation so that analysis of community ownership, beyond various degrees of "participation," is seen as central to strong evaluation practices. This article challenges mainstream evaluation practices and proposes a framework of Critical Evaluation with 3 levels: upstream evaluation assessing the "who" and "how" of programming decisions; midstream evaluation focusing on the "who" and "how" of selecting program objectives; and downstream evaluation, the focus of current mainstream evaluation, which assesses whether the program achieved its stated objectives. A vital tenet of our framework is that a community possesses the right to determine the path of its health development. A prerequisite of success, regardless of technical outcomes, is that programs must address communities' high priority concerns. Current participatory methods still seldom practice community ownership of program selection because they are vulnerable to funding agencies' predetermined priorities. In addition to critiquing evaluation practices and proposing an alternative framework, we acknowledge likely challenges and propose directions for future research. Copyright © 2018 John Wiley & Sons, Ltd.

Using the "Indicators of Engaged Learning Online" Framework to Evaluate Online Course Quality

ERIC Educational Resources Information Center

Bigatel, Paula M.; Edel-Malizia, Stephanie

2018-01-01

This article is a case study of the use of the "Indicators of Engaged Learning Online" (IELO) framework (See Appendix 1) as a guide to evaluate the quality of online courses. The framework lends itself well to measures of engagement, particularly, in terms of online course design because of its comprehensiveness. Six online courses were…

Object-oriented models of cognitive processing.

PubMed

Mather, G

2001-05-01

Information-processing models of vision and cognition are inspired by procedural programming languages. Models that emphasize object-based representations are closely related to object-oriented programming languages. The concepts underlying object-oriented languages provide a theoretical framework for cognitive processing that differs markedly from that offered by procedural languages. This framework is well-suited to a system designed to deal flexibly with discrete objects and unpredictable events in the world.

Supporting the Evaluation and Implementation of Musculoskeletal Models of Care: A Globally Informed Framework for Judging Readiness and Success.

PubMed

Briggs, Andrew M; Jordan, Joanne E; Jennings, Matthew; Speerin, Robyn; Bragge, Peter; Chua, Jason; Woolf, Anthony D; Slater, Helen

2017-04-01

To develop a globally informed framework to evaluate readiness for implementation and success after implementation of musculoskeletal models of care (MOCs). Three phases were undertaken: 1) a qualitative study with 27 Australian subject matter experts (SMEs) to develop a draft framework; 2) an eDelphi study with an international panel of 93 SMEs across 30 nations to evaluate face validity, and refine and establish consensus on the framework components; and 3) translation of the framework into a user-focused resource and evaluation of its acceptability with the eDelphi panel. A comprehensive evaluation framework was developed for judging the readiness and success of musculoskeletal MOCs. The framework consists of 9 domains, with each domain containing a number of themes underpinned by detailed elements. In the first Delphi round, scores of "partly agree" or "completely agree" with the draft framework ranged 96.7%-100%. In the second round, "essential" scores ranged 58.6%-98.9%, resulting in 14 of 34 themes being classified as essential. SMEs strongly agreed or agreed that the final framework was useful (98.8%), usable (95.1%), credible (100%) and appealing (93.9%). Overall, 96.3% strongly supported or supported the final structure of the framework as it was presented, while 100%, 96.3%, and 100% strongly supported or supported the content within the readiness, initiating implementation, and success streams, respectively. An empirically derived framework to evaluate the readiness and success of musculoskeletal MOCs was strongly supported by an international panel of SMEs. The framework provides an important internationally applicable benchmark for the development, implementation, and evaluation of musculoskeletal MOCs. © 2016, American College of Rheumatology.

Evaluation of the suitability of root cause analysis frameworks for the investigation of community-acquired pressure ulcers: a systematic review and documentary analysis.

PubMed

McGraw, Caroline; Drennan, Vari M

2015-02-01

To evaluate the suitability of root cause analysis frameworks for the investigation of community-acquired pressure ulcers. The objective was to identify the extent to which these frameworks take account of the setting where the ulcer originated as being the person's home rather than a hospital setting. Pressure ulcers involving full-thickness skin loss are increasingly being regarded as indicators of nursing patient safety failure, requiring investigation using root cause analysis frameworks. Evidence suggests that root cause analysis frameworks developed in hospital settings ignore the unique dimensions of risk in home healthcare settings. A systematic literature review and documentary analysis of frameworks used to investigate community-acquired grade three and four pressure ulcers by home nursing services in England. No published papers were identified for inclusion in the review. Fifteen patient safety investigative frameworks were collected and analysed. Twelve of the retrieved frameworks were intended for the investigation of community-acquired pressure ulcers; seven of which took account of the setting where the ulcer originated as being the patient's home. This study provides evidence to suggest that many of the root cause analysis frameworks used to investigate community-acquired pressure ulcers in England are unsuitable for this purpose. This study provides researchers and practitioners with evidence of the need to develop appropriate home nursing root cause analysis frameworks to investigate community-acquired pressure ulcers. © 2014 John Wiley & Sons Ltd.

A framework for developing objective and measurable recovery criteria for threatened and endangered species.

PubMed

Himes Boor, Gina K

2014-02-01

For species listed under the U.S. Endangered Species Act (ESA), the U.S. Fish and Wildlife Service and National Marine Fisheries Service are tasked with writing recovery plans that include "objective, measurable criteria" that define when a species is no longer at risk of extinction, but neither the act itself nor agency guidelines provide an explicit definition of objective, measurable criteria. Past reviews of recovery plans, including one published in 2012, show that many criteria lack quantitative metrics with clear biological rationale and are not meeting the measureable and objective mandate. I reviewed how objective, measureable criteria have been defined implicitly and explicitly in peer-reviewed literature, the ESA, other U.S. statutes, and legal decisions. Based on a synthesis of these sources, I propose the following 6 standards be used as minimum requirements for objective, measurable criteria: contain a quantitative threshold with calculable units, stipulate a timeframe over which they must be met, explicitly define the spatial extent or population to which they apply, specify a sampling procedure that includes sample size, specify a statistical significance level, and include justification by providing scientific evidence that the criteria define a species whose extinction risk has been reduced to the desired level. To meet these 6 standards, I suggest that recovery plans be explicitly guided by and organized around a population viability modeling framework even if data or agency resources are too limited to complete a viability model. When data and resources are available, recovery criteria can be developed from the population viability model results, but when data and resources are insufficient for model implementation, extinction risk thresholds can be used as criteria. A recovery-planning approach centered on viability modeling will also yield appropriately focused data-acquisition and monitoring plans and will facilitate a seamless transition

System modeling with the DISC framework: evidence from safety-critical domains.

PubMed

Reiman, Teemu; Pietikäinen, Elina; Oedewald, Pia; Gotcheva, Nadezhda

2012-01-01

The objective of this paper is to illustrate the development and application of the Design for Integrated Safety Culture (DISC) framework for system modeling by evaluating organizational potential for safety in nuclear and healthcare domains. The DISC framework includes criteria for good safety culture and a description of functions that the organization needs to implement in order to orient the organization toward the criteria. Three case studies will be used to illustrate the utilization of the DISC framework in practice.

An evaluation framework for obesity prevention policy interventions.

PubMed

Leeman, Jennifer; Sommers, Janice; Vu, Maihan; Jernigan, Jan; Payne, Gayle; Thompson, Diane; Heiser, Claire; Farris, Rosanne; Ammerman, Alice

2012-01-01

As the emphasis on preventing obesity has grown, so have calls for interventions that extend beyond individual behaviors and address changes in environments and policies. Despite the need for policy action, little is known about policy approaches that are most effective at preventing obesity. The Centers for Disease Control and Prevention (CDC) and others are funding the implementation and evaluation of new obesity prevention policies, presenting a distinct opportunity to learn from these practice-based initiatives and build the body of evidence-based approaches. However, contributions from this policy activity are limited by the incomplete and inconsistent evaluation data collected on policy processes and outcomes. We present a framework developed by the CDC-funded Center of Excellence for Training and Research Translation that public health practitioners can use to evaluate policy interventions and identify the practice-based evidence needed to fill the gaps in effective policy approaches to obesity prevention.

Proposed Framework for the Evaluation of Standalone Corpora Processing Systems: An Application to Arabic Corpora

PubMed Central

Al-Thubaity, Abdulmohsen; Alqifari, Reem

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework. PMID:25610910

Proposed framework for the evaluation of standalone corpora processing systems: an application to Arabic corpora.

PubMed

Al-Thubaity, Abdulmohsen; Al-Khalifa, Hend; Alqifari, Reem; Almazrua, Manal

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework.

Framework of behavioral indicators for outcome evaluation of TB health promotion: a Delphi study of TB suspects and Tb patients.

PubMed

Li, Ying; Ehiri, John; Hu, Daiyu; Zhang, Yanqi; Wang, Qingya; Zhang, Shun; Cao, Jia

2014-05-16

Health promotion for prevention and control of Tuberculosis (TB) is implemented worldwide because of its importance, but few reports have evaluated its impact on behavior due to a lack of standard outcome indicators. The objective of this study was to establish a framework of behavioral indicators for outcome evaluation of TB health promotion among TB suspects and patients. A two-round modified Delphi method involving sixteen TB control experts was used to establish a framework of behavioral indicators for outcome evaluation of TB health promotion targeted at TB suspects and patients. Sixteen of seventeen invited experts in TB control (authority score of 0.91 on a 1.0 scale) participated in round 1 survey. All sixteen experts also participated in a second round survey. After two rounds of surveys and several iterations among the experts, there was consensus on a framework of indicators for measuring outcomes of TB health promotion for TB suspects and patients. For TB suspects, the experts reached consensus on 2 domains ("Healthcare seeking behavior" and "Transmission prevention"), 3 subdomains ("Seeking care after onset of TB symptoms", "Pathways of seeking care" and "Interpersonal contact etiquette"), and 8 indicators (including among others, "Length of patient delay"). For TB patients, consensus was reached on 3 domains ("Adherence to treatment", "Healthy lifestyle" and "Transmission prevention"), 8 subdomains (including among others, "Adherence to their medication"), and 14 indicators (including "Percentage of patients who adhered to their medication"). Operational definitions and data sources were provided for each indicator. The findings of this study provide the basis for debate among international experts on a framework for achieving global consensus on outcome indicators for TB health promotion interventions targeted at TB patients and suspects. Such consensus will help to increase effectiveness of TB health promotion, while ensuring international

New model framework and structure and the commonality evaluation model. [concerning unmanned spacecraft projects

NASA Technical Reports Server (NTRS)

1977-01-01

The development of a framework and structure for shuttle era unmanned spacecraft projects and the development of a commonality evaluation model is documented. The methodology developed for model utilization in performing cost trades and comparative evaluations for commonality studies is discussed. The model framework consists of categories of activities associated with the spacecraft system's development process. The model structure describes the physical elements to be treated as separate identifiable entities. Cost estimating relationships for subsystem and program-level components were calculated.

Valuing hydrological alteration in multi-objective water resources management

NASA Astrophysics Data System (ADS)

Bizzi, Simone; Pianosi, Francesca; Soncini-Sessa, Rodolfo

2012-11-01

SummaryThe management of water through the impoundment of rivers by dams and reservoirs is necessary to support key human activities such as hydropower production, agriculture and flood risk mitigation. Advances in multi-objective optimization techniques and ever growing computing power make it possible to design reservoir operating policies that represent Pareto-optimal tradeoffs between multiple interests. On the one hand, such optimization methods can enhance performances of commonly targeted objectives (such as hydropower production or water supply), on the other hand they risk strongly penalizing all the interests not directly (i.e. mathematically) included in the optimization algorithm. The alteration of the downstream hydrological regime is a well established cause of ecological degradation and its evaluation and rehabilitation is commonly required by recent legislation (as the Water Framework Directive in Europe). However, it is rarely embedded in reservoir optimization routines and, even when explicitly considered, the criteria adopted for its evaluation are doubted and not commonly trusted, undermining the possibility of real implementation of environmentally friendly policies. The main challenges in defining and assessing hydrological alterations are: how to define a reference state (referencing); how to define criteria upon which to build mathematical indicators of alteration (measuring); and finally how to aggregate the indicators in a single evaluation index (valuing) that can serve as objective function in the optimization problem. This paper aims to address these issues by: (i) discussing the benefits and constrains of different approaches to referencing, measuring and valuing hydrological alteration; (ii) testing two alternative indices of hydrological alteration, one based on the established framework of Indicators of Hydrological Alteration (Richter et al., 1996), and one satisfying the mathematical properties required by widely used optimization

Picasso Paintings, Moon Rocks, and Hand-Written Beatles Lyrics: Adults' Evaluations of Authentic Objects.

PubMed

Frazier, Brandy N; Gelman, Susan A; Wilson, Alice; Hood, Bruce

2009-01-01

Authentic objects are those that have an historical link to a person, event, time, or place of some significance (e.g., original Picasso painting; gown worn by Princess Diana; your favorite baby blanket). The current study examines everyday beliefs about authentic objects, with three primary goals: to determine the scope of adults' evaluation of authentic objects, to examine such evaluation in two distinct cultural settings, and to determine whether a person's attachment history (i.e., whether or not they owned an attachment object as a child) predicts evaluation of authentic objects. We found that college students in the U.K. (N = 125) and U.S. (N = 119) consistently evaluate a broad range of authentic items as more valuable than matched control (inauthentic) objects, more desirable to keep, and more desirable to touch, though only non-personal authentic items were judged to be more appropriate for display in a museum. These patterns were remarkably similar across the two cultural contexts. Additionally, those who had an attachment object as a child evaluated objects more favorably, and in particular judged authentic objects to be more valuable. Altogether, these results demonstrate broad endorsement of "positive contagion" among college-educated adults.

Picasso Paintings, Moon Rocks, and Hand-Written Beatles Lyrics: Adults’ Evaluations of Authentic Objects

PubMed Central

Frazier, Brandy N.; Gelman, Susan A.; Wilson, Alice; Hood, Bruce

2010-01-01

Authentic objects are those that have an historical link to a person, event, time, or place of some significance (e.g., original Picasso painting; gown worn by Princess Diana; your favorite baby blanket). The current study examines everyday beliefs about authentic objects, with three primary goals: to determine the scope of adults’ evaluation of authentic objects, to examine such evaluation in two distinct cultural settings, and to determine whether a person’s attachment history (i.e., whether or not they owned an attachment object as a child) predicts evaluation of authentic objects. We found that college students in the U.K. (N = 125) and U.S. (N = 119) consistently evaluate a broad range of authentic items as more valuable than matched control (inauthentic) objects, more desirable to keep, and more desirable to touch, though only non-personal authentic items were judged to be more appropriate for display in a museum. These patterns were remarkably similar across the two cultural contexts. Additionally, those who had an attachment object as a child evaluated objects more favorably, and in particular judged authentic objects to be more valuable. Altogether, these results demonstrate broad endorsement of "positive contagion" among college-educated adults. PMID:20631919

A framework and case studies for evaluation of enzyme ontogeny in children's health risk evaluation.

PubMed

Ginsberg, Gary; Vulimiri, Suryanarayana V; Lin, Yu-Sheng; Kancherla, Jayaram; Foos, Brenda; Sonawane, Babasaheb

2017-01-01

Knowledge of the ontogeny of Phase I and Phase II metabolizing enzymes may be used to inform children's vulnerability based upon likely differences in internal dose from xenobiotic exposure. This might provide a qualitative assessment of toxicokinetic (TK) variability and uncertainty pertinent to early lifestages and help scope a more quantitative physiologically based toxicokinetic (PBTK) assessment. Although much is known regarding the ontogeny of metabolizing systems, this is not commonly utilized in scoping and problem formulation stage of human health risk evaluation. A framework is proposed for introducing this information into problem formulation which combines data on enzyme ontogeny and chemical-specific TK to explore potential child/adult differences in internal dose and whether such metabolic differences may be important factors in risk evaluation. The framework is illustrated with five case study chemicals, including some which are data rich and provide proof of concept, while others are data poor. Case studies for toluene and chlorpyrifos indicate potentially important child/adult TK differences while scoping for acetaminophen suggests enzyme ontogeny is unlikely to increase early-life risks. Scoping for trichloroethylene and aromatic amines indicates numerous ways that enzyme ontogeny may affect internal dose which necessitates further evaluation. PBTK modeling is a critical and feasible next step to further evaluate child-adult differences in internal dose for a number of these chemicals.

A framework to evaluate proposals for scientific activities in wilderness

Treesearch

Peter Landres

2010-01-01

Every year, the four Federal wilderness management agencies - U.S. DOI Bureau of Land Management, Fish and Wildlife Service, National Park Service, and the USDA Forest Service - receive hundreds of proposals to conduct scientific studies within wilderness. There is no consistent and comprehensive framework for evaluating such proposals that accounts for the unique...

Green and sustainable remediation (GSR) evaluation: framework, standards, and tool. A case study in Taiwan.

PubMed

Huang, Wen-Yen; Hung, Weiteng; Vu, Chi Thanh; Chen, Wei-Ting; Lai, Jhih-Wei; Lin, Chitsan

2016-11-01

Taiwan has a large number of poorly managed contaminated sites in need of remediation. This study proposes a framework, a set of standards, and a spreadsheet-based evaluation tool for implementing green and sustainable principles into remediation projects and evaluating the projects from this perspective. We performed a case study to understand how the framework would be applied. For the case study, we used a spreadsheet-based evaluation tool (SEFA) and performed field scale cultivation tests on a site contaminated with total petroleum hydrocarbons (TPHs). The site was divided into two lots: one treated by chemical oxidation and the other by bioremediation. We evaluated five core elements of green and sustainable remediation (GSR): energy, air, water resources, materials and wastes, and land and ecosystem. The proposed evaluation tool and field scale cultivation test were found to efficiently assess the effectiveness of the two remediation alternatives. The framework and related tools proposed herein can potentially be used to support decisions about the remediation of contaminated sites taking into account engineering management, cost effectiveness, and social reconciliation.

Benefit-risk Evaluation for Diagnostics: A Framework (BED-FRAME).

PubMed

Evans, Scott R; Pennello, Gene; Pantoja-Galicia, Norberto; Jiang, Hongyu; Hujer, Andrea M; Hujer, Kristine M; Manca, Claudia; Hill, Carol; Jacobs, Michael R; Chen, Liang; Patel, Robin; Kreiswirth, Barry N; Bonomo, Robert A

2016-09-15

The medical community needs systematic and pragmatic approaches for evaluating the benefit-risk trade-offs of diagnostics that assist in medical decision making. Benefit-Risk Evaluation of Diagnostics: A Framework (BED-FRAME) is a strategy for pragmatic evaluation of diagnostics designed to supplement traditional approaches. BED-FRAME evaluates diagnostic yield and addresses 2 key issues: (1) that diagnostic yield depends on prevalence, and (2) that different diagnostic errors carry different clinical consequences. As such, evaluating and comparing diagnostics depends on prevalence and the relative importance of potential errors. BED-FRAME provides a tool for communicating the expected clinical impact of diagnostic application and the expected trade-offs of diagnostic alternatives. BED-FRAME is a useful fundamental supplement to the standard analysis of diagnostic studies that will aid in clinical decision making. © The Author 2016. Published by Oxford University Press for the Infectious Diseases Society of America. All rights reserved. For permissions, e-mail journals.permissions@oup.com.

An operational structured decision making framework for ...

EPA Pesticide Factsheets

Pressure to develop an operational framework for decision makers to employ the concepts of ecosystem goods and services for assessing changes to human well-being has been increasing since these concepts gained widespread notoriety after the Millennium Ecosystem Assessment Report. Many conceptual frameworks have been proposed, but most do not propose methodologies and tools to make this approach to decision making implementable. Building on common components of existing conceptual frameworks for ecosystem services and human well-being assessment we apply a structured decision making approach to develop a standardized operational framework and suggest tools and methods for completing each step. The structured decision making approach consists of six steps: 1) Clarify the Decision Context 2) Define Objectives and Evaluation Criteria 3) Develop Alternatives 4) Estimate Consequences 5) Evaluate Trade-Offs and Select and 6) Implement and Monitor. These six steps include the following activities, and suggested tools, when applied to ecosystem goods and services and human well-being conceptual frameworks: 1) Characterization of decision specific human beneficiaries using the Final Ecosystem Goods and Services (FEGS) approach and Classification System (FEGS-CS) 2) Determine beneficiaries’ relative priorities for human well-being domains in the Human Well-Being Index (HWBI) through stakeholder engagement and identify beneficiary-relevant metrics of FEGS using the Nat

Object-Oriented Algorithm For Evaluation Of Fault Trees

NASA Technical Reports Server (NTRS)

Patterson-Hine, F. A.; Koen, B. V.

1992-01-01

Algorithm for direct evaluation of fault trees incorporates techniques of object-oriented programming. Reduces number of calls needed to solve trees with repeated events. Provides significantly improved software environment for such computations as quantitative analyses of safety and reliability of complicated systems of equipment (e.g., spacecraft or factories).

An Evaluation Framework for Obesity Prevention Policy Interventions

PubMed Central

Sommers, Janice; Vu, Maihan; Jernigan, Jan; Payne, Gayle; Thompson, Diane; Heiser, Claire; Farris, Rosanne; Ammerman, Alice

2012-01-01

As the emphasis on preventing obesity has grown, so have calls for interventions that extend beyond individual behaviors and address changes in environments and policies. Despite the need for policy action, little is known about policy approaches that are most effective at preventing obesity. The Centers for Disease Control and Prevention (CDC) and others are funding the implementation and evaluation of new obesity prevention policies, presenting a distinct opportunity to learn from these practice-based initiatives and build the body of evidence-based approaches. However, contributions from this policy activity are limited by the incomplete and inconsistent evaluation data collected on policy processes and outcomes. We present a framework developed by the CDC-funded Center of Excellence for Training and Research Translation that public health practitioners can use to evaluate policy interventions and identify the practice-based evidence needed to fill the gaps in effective policy approaches to obesity prevention. PMID:22742594

Assessing the Spread and Uptake of a Framework for Introducing and Evaluating Advanced Practice Nursing Roles.

PubMed

Boyko, Jennifer A; Carter, Nancy; Bryant-Lukosius, Denise

2016-08-01

Health system researchers must ensure that the products of their work meet the needs of various stakeholder groups (e.g., patients, practitioners, and policy makers). Evidence-based frameworks can support the uptake and spread of research evidence; however, their existence as knowledge translation tools does not ensure their uptake and it is difficult to ascertain their spread into research, practice, and policy using existing methods. The purpose of this article is to report results of a study on the spread and uptake of an evidence-based framework (i.e., the participatory, evidence-based, patient-focused process for advanced practice nursing [PEPPA] framework) into research, practice, and policies relevant to the introduction and evaluation of advanced practice nursing roles. We also reflect on the utility of using a modified citation methodology to evaluate knowledge translation efforts. We searched four databases for literature published between 2004 and 2014 citing the original paper in which the PEPPA framework was published, and carried out an Internet search for grey literature using keywords. Relevant data were extracted from sources and organized using NVivo software. We analysed results descriptively. Our search yielded 164 unique sources of which 69.5% were from published literature and the majority (83.4%) of these were published in nursing journals. Most frequently (71.5%), the framework was used by researchers and students in research studies. A smaller number of citations (11.3%) reflected use of the PEPPA framework in practice settings with a focus on role development, implementation, evaluation, or a combination of these. This study demonstrates that the PEPPA framework has been used to varying degrees as intended, and provides guidance on how to evaluate the spread and uptake of research outputs (e.g., theoretical frameworks). Further research is needed about ways to determine whether evidence-informed research tools such as frameworks have been

Measuring social accountability in health professional education: development and international pilot testing of an evaluation framework.

PubMed

Larkins, Sarah L; Preston, Robyn; Matte, Marie C; Lindemann, Iris C; Samson, Rex; Tandinco, Filedito D; Buso, David; Ross, Simone J; Pálsdóttir, Björg; Neusy, André-Jacques

2013-01-01

Health professional schools are responsible for producing graduates with competencies and attitudes to address health inequities and respond to priority health needs. Health professional schools striving towards social accountability founded the Training for Health Equity Network (THEnet). This article describes the development of THEnet evaluation framework for socially accountable health professional education, presents the framework to be used as a tool by other schools and discusses the findings of pilot implementation at five schools. The framework was designed collaboratively and built on Boelen and Woollard's conceptualization, production and usability model. It includes key components, linked to aspirational statements, indicators and suggested measurement tools. Five schools completed pilot implementation, involving workshops, document/data review and focus group discussions with faculty, students and community members. Three sections of the framework consider: How does our school work?; What do we do? and What difference do we make? Pilot testing proved that the evaluation framework was acceptable and feasible across contexts and produced findings useful at school level and to compare schools. The framework is designed as a formative exercise to help schools take a critical look at their performance and progress towards social accountability. Initiatives to implement the framework more widely are underway. The framework effectively aids in identifying strengths, weaknesses and gaps, with a view to schools striving for continuous self-improvement. THEnet evaluation framework is applicable and useful across contexts. It is possible and desirable to assess progress towards social accountability in health professional schools and this is an important step in producing health professionals with knowledge, attitudes, and skills to meet the challenges of priority health needs of underserved populations.

Evaluating cloud retrieval algorithms with the ARM BBHRP framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mlawer,E.; Dunn,M.; Mlawer, E.

2008-03-10

Climate and weather prediction models require accurate calculations of vertical profiles of radiative heating. Although heating rate calculations cannot be directly validated due to the lack of corresponding observations, surface and top-of-atmosphere measurements can indirectly establish the quality of computed heating rates through validation of the calculated irradiances at the atmospheric boundaries. The ARM Broadband Heating Rate Profile (BBHRP) project, a collaboration of all the working groups in the program, was designed with these heating rate validations as a key objective. Given the large dependence of radiative heating rates on cloud properties, a critical component of BBHRP radiative closure analysesmore » has been the evaluation of cloud microphysical retrieval algorithms. This evaluation is an important step in establishing the necessary confidence in the continuous profiles of computed radiative heating rates produced by BBHRP at the ARM Climate Research Facility (ACRF) sites that are needed for modeling studies. This poster details the continued effort to evaluate cloud property retrieval algorithms within the BBHRP framework, a key focus of the project this year. A requirement for the computation of accurate heating rate profiles is a robust cloud microphysical product that captures the occurrence, height, and phase of clouds above each ACRF site. Various approaches to retrieve the microphysical properties of liquid, ice, and mixed-phase clouds have been processed in BBHRP for the ACRF Southern Great Plains (SGP) and the North Slope of Alaska (NSA) sites. These retrieval methods span a range of assumptions concerning the parameterization of cloud location, particle density, size, shape, and involve different measurement sources. We will present the radiative closure results from several different retrieval approaches for the SGP site, including those from Microbase, the current 'reference' retrieval approach in BBHRP. At the NSA, mixed

Towards an evaluation framework for information quality management (IQM) practices for health information systems--evaluation criteria for effective IQM practices.

PubMed

Mohammed, Siti Asma; Yusof, Maryati Mohd

2013-04-01

Poor information quality (IQ) must be understood as a business problem rather than systems problem. In health care organization, what is required is an effective quality management that continuously manages and reviews the factors influencing IQ in health information systems (HIS) so as to achieve the desired outcomes. Hence, in order to understand the issues of information quality management (IQM) practices in health care organizations, a more holistic evaluation study should be undertaken to investigate the IQM practices in health care organizations. It is the aim of this paper to identify the significant evaluation criteria that influence the production of good IQ in HIS. Six selected frameworks and best practices both from health informatics and information systems literature have been reviewed to identify the evaluation criteria from the perspective of human, organizational and technological factors. From the review, it was found that human and organization factors are of greater significance in influencing HIS IQ. Our review depicts that there is still shortage in finding a comprehensive IQM evaluation framework. Thus, the criteria from the frameworks reviewed can be used in combination for more comprehensive evaluation criteria. Integrated IQM evaluation criteria for HIS are then proposed in this study. Poor IQ is the result of complex interdependency within sociotechnical factors in health care organization and lack of formal and structured IQM practices. Thus, a feedback mechanism such as evaluation is needed to understand the issues in depth in the future. © 2012 Blackwell Publishing Ltd.

Multiple-objective evaluation of wastewater treatment plant control alternatives.

PubMed

Flores-Alsina, Xavier; Gallego, Alejandro; Feijoo, Gumersindo; Rodriguez-Roda, Ignasi

2010-05-01

Besides the evaluation of the environmental issues, the correct assessment of wastewater treatment plants (WWTP) should take into account several objectives such as: economic e.g. operation costs; technical e.g. risk of suffering microbiology-related TSS separation problems; or legal e.g. accomplishment with the effluent standards in terms of the different pollution loads. For this reason, the main objective of this paper is to show the benefits of complementing the environmental assessment carried out by life cycle assessment with economical, technical and legal criteria. Using a preliminary version of the BSM2 as a case study, different combinations of controllers are implemented, simulated and evaluated. In the following step, the resulting multi-criteria matrix is mined using multivariate statistical techniques. The results showed that the presence of an external carbon source addition, the type of aeration system and the TSS controller are the key elements creating the differences amongst the alternatives. Also, it was possible to characterize the different control strategies according to a set of aggregated criteria. Additionally, the existing synergies amongst different objectives and their consequent trade-offs were identified. Finally, it was discovered that from the initial extensive list of evaluation criteria, only a small set of five are really discriminant, being useful to differentiate within the generated alternatives. Copyright 2010 Elsevier Ltd. All rights reserved.

An evaluation framework for effective public participation in EIA in Pakistan

DOE Office of Scientific and Technical Information (OSTI.GOV)

Nadeem, Obaidullah, E-mail: obaidnadeem@yahoo.co; Fischer, Thomas B., E-mail: fischer@liverpool.ac.u

2011-01-15

Evaluating the effectiveness of public participation in EIA related decisions is of crucial importance for developing a better understanding of overall EIA effectiveness. This paper aims to contribute to the professional debate by establishing a country specific evaluation framework for Pakistan, which, it is suggested, could also potentially be used in other developing countries. The framework is used to evaluate performance of public participation in EIA in terms of 40 attributes for four selected projects from the province of Punjab. The evaluation is based on interviews with stakeholders, review of EIA reports as well as public hearing proceedings and environmentalmore » approval conditions. The evaluation of the selected projects revealed an overall weak influence of public participation on substantive quality of EIA and on the final decision. Overall, EIA public participation has succeeded in providing a more egalitarian environment. Furthermore, it appears fair to say that sufficient time for submitting written comments on EIA reports as well as for raising concerns during public hearings had been given. Also, public consultation was significantly contributing to educating participants. Despite some impediments, it is argued that public participation in EIA is gradually gaining ground in Pakistan. Recommendations to enhance EIA public participation effectiveness in Pakistan include applying a more proactive approach which should take place before EIA is conducted and before site selection for development projects is happening.« less

Phenotype Instance Verification and Evaluation Tool (PIVET): A Scaled Phenotype Evidence Generation Framework Using Web-Based Medical Literature

PubMed Central

Ke, Junyuan; Ho, Joyce C; Ghosh, Joydeep; Wallace, Byron C

2018-01-01

Background Researchers are developing methods to automatically extract clinically relevant and useful patient characteristics from raw healthcare datasets. These characteristics, often capturing essential properties of patients with common medical conditions, are called computational phenotypes. Being generated by automated or semiautomated, data-driven methods, such potential phenotypes need to be validated as clinically meaningful (or not) before they are acceptable for use in decision making. Objective The objective of this study was to present Phenotype Instance Verification and Evaluation Tool (PIVET), a framework that uses co-occurrence analysis on an online corpus of publically available medical journal articles to build clinical relevance evidence sets for user-supplied phenotypes. PIVET adopts a conceptual framework similar to the pioneering prototype tool PheKnow-Cloud that was developed for the phenotype validation task. PIVET completely refactors each part of the PheKnow-Cloud pipeline to deliver vast improvements in speed without sacrificing the quality of the insights PheKnow-Cloud achieved. Methods PIVET leverages indexing in NoSQL databases to efficiently generate evidence sets. Specifically, PIVET uses a succinct representation of the phenotypes that corresponds to the index on the corpus database and an optimized co-occurrence algorithm inspired by the Aho-Corasick algorithm. We compare PIVET’s phenotype representation with PheKnow-Cloud’s by using PheKnow-Cloud’s experimental setup. In PIVET’s framework, we also introduce a statistical model trained on domain expert–verified phenotypes to automatically classify phenotypes as clinically relevant or not. Additionally, we show how the classification model can be used to examine user-supplied phenotypes in an online, rather than batch, manner. Results PIVET maintains the discriminative power of PheKnow-Cloud in terms of identifying clinically relevant phenotypes for the same corpus with

A Contextual Factors Framework to Inform Implementation and Evaluation of Public Health Initiatives

ERIC Educational Resources Information Center

Vanderkruik, Rachel; McPherson, Marianne E.

2017-01-01

Evaluating initiatives implemented across multiple settings can elucidate how various contextual factors may influence both implementation and outcomes. Understanding context is especially critical when the same program has varying levels of success across settings. We present a framework for evaluating contextual factors affecting an initiative…

Towards a Framework for Evaluating and Comparing Diagnosis Algorithms

NASA Technical Reports Server (NTRS)

Kurtoglu, Tolga; Narasimhan, Sriram; Poll, Scott; Garcia,David; Kuhn, Lukas; deKleer, Johan; vanGemund, Arjan; Feldman, Alexander

2009-01-01

Diagnostic inference involves the detection of anomalous system behavior and the identification of its cause, possibly down to a failed unit or to a parameter of a failed unit. Traditional approaches to solving this problem include expert/rule-based, model-based, and data-driven methods. Each approach (and various techniques within each approach) use different representations of the knowledge required to perform the diagnosis. The sensor data is expected to be combined with these internal representations to produce the diagnosis result. In spite of the availability of various diagnosis technologies, there have been only minimal efforts to develop a standardized software framework to run, evaluate, and compare different diagnosis technologies on the same system. This paper presents a framework that defines a standardized representation of the system knowledge, the sensor data, and the form of the diagnosis results and provides a run-time architecture that can execute diagnosis algorithms, send sensor data to the algorithms at appropriate time steps from a variety of sources (including the actual physical system), and collect resulting diagnoses. We also define a set of metrics that can be used to evaluate and compare the performance of the algorithms, and provide software to calculate the metrics.

Real-time visual tracking of less textured three-dimensional objects on mobile platforms

NASA Astrophysics Data System (ADS)

Seo, Byung-Kuk; Park, Jungsik; Park, Hanhoon; Park, Jong-Il

2012-12-01

Natural feature-based approaches are still challenging for mobile applications (e.g., mobile augmented reality), because they are feasible only in limited environments such as highly textured and planar scenes/objects, and they need powerful mobile hardware for fast and reliable tracking. In many cases where conventional approaches are not effective, three-dimensional (3-D) knowledge of target scenes would be beneficial. We present a well-established framework for real-time visual tracking of less textured 3-D objects on mobile platforms. Our framework is based on model-based tracking that efficiently exploits partially known 3-D scene knowledge such as object models and a background's distinctive geometric or photometric knowledge. Moreover, we elaborate on implementation in order to make it suitable for real-time vision processing on mobile hardware. The performance of the framework is tested and evaluated on recent commercially available smartphones, and its feasibility is shown by real-time demonstrations.

Evaluating QR Code Case Studies Using a Mobile Learning Framework

ERIC Educational Resources Information Center

Rikala, Jenni

2014-01-01

The aim of this study was to evaluate the feasibility of Quick Response (QR) codes and mobile devices in the context of Finnish basic education. The feasibility was analyzed through a mobile learning framework, which includes the core characteristics of mobile learning. The study is part of a larger research where the aim is to develop a…

Object segmentation using graph cuts and active contours in a pyramidal framework

NASA Astrophysics Data System (ADS)

Subudhi, Priyambada; Mukhopadhyay, Susanta

2018-03-01

Graph cuts and active contours are two very popular interactive object segmentation techniques in the field of computer vision and image processing. However, both these approaches have their own well-known limitations. Graph cut methods perform efficiently giving global optimal segmentation result for smaller images. However, for larger images, huge graphs need to be constructed which not only takes an unacceptable amount of memory but also increases the time required for segmentation to a great extent. On the other hand, in case of active contours, initial contour selection plays an important role in the accuracy of the segmentation. So a proper selection of initial contour may improve the complexity as well as the accuracy of the result. In this paper, we have tried to combine these two approaches to overcome their above-mentioned drawbacks and develop a fast technique of object segmentation. Here, we have used a pyramidal framework and applied the mincut/maxflow algorithm on the lowest resolution image with the least number of seed points possible which will be very fast due to the smaller size of the image. Then, the obtained segmentation contour is super-sampled and and worked as the initial contour for the next higher resolution image. As the initial contour is very close to the actual contour, so fewer number of iterations will be required for the convergence of the contour. The process is repeated for all the high-resolution images and experimental results show that our approach is faster as well as memory efficient as compare to both graph cut or active contour segmentation alone.

The Effect of Instructional Objectives and General Objectives on Student Self-Evaluation of Psychomotor Performance in Power Mechanics.

ERIC Educational Resources Information Center

Janeczko, Robert John

The major purpose of this study was to ascertain the relative effects of student exposure to instructional objectives upon student self-evaluation of psychomotor activities in a college-level power mechanics course. A randomized posttest-only control group design was used with two different approaches to the statement of the objectives. Four…

A unified computational model of the development of object unity, object permanence, and occluded object trajectory perception.

PubMed

Franz, A; Triesch, J

2010-12-01

The perception of the unity of objects, their permanence when out of sight, and the ability to perceive continuous object trajectories even during occlusion belong to the first and most important capacities that infants have to acquire. Despite much research a unified model of the development of these abilities is still missing. Here we make an attempt to provide such a unified model. We present a recurrent artificial neural network that learns to predict the motion of stimuli occluding each other and that develops representations of occluded object parts. It represents completely occluded, moving objects for several time steps and successfully predicts their reappearance after occlusion. This framework allows us to account for a broad range of experimental data. Specifically, the model explains how the perception of object unity develops, the role of the width of the occluders, and it also accounts for differences between data for moving and stationary stimuli. We demonstrate that these abilities can be acquired by learning to predict the sensory input. The model makes specific predictions and provides a unifying framework that has the potential to be extended to other visual event categories. Copyright © 2010 Elsevier Inc. All rights reserved.

Developing Objective Criteria for Evaluating Student Athletic Trainers.

ERIC Educational Resources Information Center

Treadway, Linda

In devising a form for the evaluation of students preparing to become athletic trainers, it is helpful to have a checklist in which objectives and behavioral responses are organized into categories, such as prevention of injury, first aid, emergency care, treatment, rehabilitation, and taping and wrapping. It is also important to have records and…

Early object labels: the case for a developmental lexical principles framework.

PubMed

Golinkoff, R M; Mervis, C B; Hirsh-Pasek, K

1994-02-01

Universally, object names make up the largest proportion of any word type found in children's early lexicons. Here we present and critically evaluate a set of six lexical principles (some previously proposed and some new) for making object label learning a manageable task. Overall, the principles have the effect of reducing the amount of information that language-learning children must consider for what a new word might mean. These principles are constructed by children in a two-tiered developmental sequence, as a function of their sensitivity to linguistic input, contextual information, and social-interactional cues. Thus, the process of lexical acquisition changes as a result of the particular principles a given child has at his or her disposal. For children who have only the principles of the first tier (reference, extendibility, and object scope), word learning has a deliberate and laborious look. The principles of the second tier (categorical scope, novel name-nameless category' or N3C, and conventionality) enable the child to acquire many new labels rapidly. The present unified account is argued to have a number of advantages over treating such principles separately and non-developmentally. Further, the explicit recognition that the acquisition and operation of these principles is influenced by the child's interpretation of both linguistic and non-linguistic input is seen as an advance.

Organic food quality: a framework for concept, definition and evaluation from the European perspective.

PubMed

Kahl, Johannes; Baars, Ton; Bügel, Susanne; Busscher, Nicolaas; Huber, Machteld; Kusche, Daniel; Rembiałkowska, Ewa; Schmid, Otto; Seidel, Kathrin; Taupier-Letage, Bruno; Velimirov, Alberta; Załecka, Aneta

2012-11-01

Consumers buy organic food because they believe in the high quality of the product. Furthermore, the EU legal regulatory framework for organic food and farming defines high quality of the products as an important goal of production. A major challenge is the need to define food quality concepts and methods for determination. A background is described which allows embedding of the quality definitions as well as evaluation methods into a conceptual framework connected to the vision and mission of organic agriculture and food production. Organic food quality is defined through specific aspects and criteria. For evaluation each criterion has to be described by indicators. The determination of indicators should be through parameters, where parameters are described by methods. Conversely, the conceptual framework is described according to underlying principles and starting definitions are given, but further work has do be done on the detailed scientific description of the indicators. Furthermore, parameters have to be defined for the evaluation of suitability of these indicators for organic food production. Copyright © 2012 Society of Chemical Industry.

Developing a framework for digital objects in the Big Data to Knowledge (BD2K) commons: Report from the Commons Framework Pilots workshop.

PubMed

Jagodnik, Kathleen M; Koplev, Simon; Jenkins, Sherry L; Ohno-Machado, Lucila; Paten, Benedict; Schurer, Stephan C; Dumontier, Michel; Verborgh, Ruben; Bui, Alex; Ping, Peipei; McKenna, Neil J; Madduri, Ravi; Pillai, Ajay; Ma'ayan, Avi

2017-07-01

The volume and diversity of data in biomedical research have been rapidly increasing in recent years. While such data hold significant promise for accelerating discovery, their use entails many challenges including: the need for adequate computational infrastructure, secure processes for data sharing and access, tools that allow researchers to find and integrate diverse datasets, and standardized methods of analysis. These are just some elements of a complex ecosystem that needs to be built to support the rapid accumulation of these data. The NIH Big Data to Knowledge (BD2K) initiative aims to facilitate digitally enabled biomedical research. Within the BD2K framework, the Commons initiative is intended to establish a virtual environment that will facilitate the use, interoperability, and discoverability of shared digital objects used for research. The BD2K Commons Framework Pilots Working Group (CFPWG) was established to clarify goals and work on pilot projects that address existing gaps toward realizing the vision of the BD2K Commons. This report reviews highlights from a two-day meeting involving the BD2K CFPWG to provide insights on trends and considerations in advancing Big Data science for biomedical research in the United States. Copyright © 2017 Elsevier Inc. All rights reserved.

High Impact Anti-Crime Program: A Framework for Assessing Project-Level Evaluation Plans.

ERIC Educational Resources Information Center

Kupersmith, Gerrie

The pamphlet describes an evaluation model for crime reduction projects developed by National Institute of Law Enforcement and Criminal Justice sponsored research. It is meant to assist justice agency and project managers in determining completeness of evaluation planning by providing a framework against which to measure their evaluation…

Expected frontiers: Incorporating weather uncertainty into a policy analysis using an integrated bi-level multi-objective optimization framework

EPA Science Inventory

Weather is the main driver in both plant use of nutrients and fate and transport of nutrients in the environment. In previous work, we evaluated a green tax for control of agricultural nutrients in a bi-level optimization framework that linked deterministic models. In this study,...

A novel framework for analyzing conservation impacts: evaluation, theory, and marine protected areas.

PubMed

Mascia, Michael B; Fox, Helen E; Glew, Louise; Ahmadia, Gabby N; Agrawal, Arun; Barnes, Megan; Basurto, Xavier; Craigie, Ian; Darling, Emily; Geldmann, Jonas; Gill, David; Holst Rice, Susie; Jensen, Olaf P; Lester, Sarah E; McConney, Patrick; Mumby, Peter J; Nenadovic, Mateja; Parks, John E; Pomeroy, Robert S; White, Alan T

2017-07-01

Environmental conservation initiatives, including marine protected areas (MPAs), have proliferated in recent decades. Designed to conserve marine biodiversity, many MPAs also seek to foster sustainable development. As is the case for many other environmental policies and programs, the impacts of MPAs are poorly understood. Social-ecological systems, impact evaluation, and common-pool resource governance are three complementary scientific frameworks for documenting and explaining the ecological and social impacts of conservation interventions. We review key components of these three frameworks and their implications for the study of conservation policy, program, and project outcomes. Using MPAs as an illustrative example, we then draw upon these three frameworks to describe an integrated approach for rigorous empirical documentation and causal explanation of conservation impacts. This integrated three-framework approach for impact evaluation of governance in social-ecological systems (3FIGS) accounts for alternative explanations, builds upon and advances social theory, and provides novel policy insights in ways that no single approach affords. Despite the inherent complexity of social-ecological systems and the difficulty of causal inference, the 3FIGS approach can dramatically advance our understanding of, and the evidentiary basis for, effective MPAs and other conservation initiatives. © 2017 New York Academy of Sciences.

How to Deal with Emotional Abuse and Neglect--Further Development of a Conceptual Framework (FRAMEA)

ERIC Educational Resources Information Center

Glaser, Danya

2011-01-01

Objective: To develop further the understanding of emotional abuse and neglect. Methods: Building on previous work, this paper describes the further development of a conceptual framework for the recognition and management of emotional abuse and neglect. Training in this framework is currently being evaluated. The paper also briefly reviews more…

Quality Evaluation of Zirconium Dioxide Frameworks Produced in Five Dental Laboratories from Different Countries.

PubMed

Schneebeli, Esther; Brägger, Urs; Scherrer, Susanne S; Keller, Andrea; Wittneben, Julia G; Hicklin, Stefan P

2017-07-01

The aim of this study was to assess and compare quality as well as economic aspects of CAD/CAM high strength ceramic three-unit FDP frameworks ordered from dental laboratories located in emerging countries and Switzerland. The master casts of six cases were sent to five dental laboratories located in Thailand (Bangkok), China (Peking and Shenzhen), Turkey (Izmir), and Switzerland (Bern). Each laboratory was using a different CAD/CAM system. The clinical fit of the frameworks was qualitatively assessed, and the thickness of the framework material, the connector height, the width, and the diameter were evaluated using a measuring sensor. The analysis of the internal fit of the frameworks was performed by means of a replica technique, whereas the inner and outer surfaces of the frameworks were evaluated for traces of postprocessing and damage to the intaglio surface with light and electronic microscopes. Groups (dental laboratories and cases) were compared for statistically significant differences using Mann-Whitney U-tests after Bonferroni correction. An acceptable clinical fit was found at 97.9% of the margins produced in laboratory E, 87.5% in B, 93.7% in C, 79.2% in A, and 62.5% in D. The mean framework thicknesses were not statistically significantly different for the premolar regions; however, for the molar area 4/8 of the evaluated sites were statistically significantly different. Circumference, surface, and width of the connectors produced in the different laboratories were statistically significantly different but not the height. There were great differences in the designs for the pontic and connector regions, and some of the frameworks would not be recommended for clinical use. Traces of heavy postprocessing were found in frameworks from some of the laboratories. The prices per framework ranged from US$177 to US$896. By ordering laboratory work in developing countries, a considerable price reduction was obtained compared to the price level in Switzerland

Experience with low-cost telemedicine in three different settings. Recommendations based on a proposed framework for network performance evaluation.

PubMed

Wootton, Richard; Vladzymyrskyy, Anton; Zolfo, Maria; Bonnardot, Laurent

2011-01-01

Telemedicine has been used for many years to support doctors in the developing world. Several networks provide services in different settings and in different ways. However, to draw conclusions about which telemedicine networks are successful requires a method of evaluating them. No general consensus or validated framework exists for this purpose. To define a basic method of performance measurement that can be used to improve and compare teleconsultation networks; to employ the proposed framework in an evaluation of three existing networks; to make recommendations about the future implementation and follow-up of such networks. Analysis based on the experience of three telemedicine networks (in operation for 7-10 years) that provide services to doctors in low-resource settings and which employ the same basic design. Although there are many possible indicators and metrics that might be relevant, five measures for each of the three user groups appear to be sufficient for the proposed framework. In addition, from the societal perspective, information about clinical- and cost-effectiveness is also required. The proposed performance measurement framework was applied to three mature telemedicine networks. Despite their differences in terms of activity, size and objectives, their performance in certain respects is very similar. For example, the time to first reply from an expert is about 24 hours for each network. Although all three networks had systems in place to collect data from the user perspective, none of them collected information about the coordinator's time required or about ease of system usage. They had only limited information about quality and cost. Measuring the performance of a telemedicine network is essential in understanding whether the network is working as intended and what effect it is having. Based on long-term field experience, the suggested framework is a practical tool that will permit organisations to assess the performance of their own networks

Developing an Evaluation Framework of Spatial Understanding through GIS Analysis of Volunteered Geographic Information (VGI)

ERIC Educational Resources Information Center

Wu, Bing Sheng

2013-01-01

This study integrates volunteered geographic information (VGI) into GIS and contextual analyses, and develops a framework to evaluate students' understanding of "locations and places in order to set national and international events within a geographical framework and to understand basic spatial relationships" as proposed by the…

Elementary Integrated Curriculum Framework

ERIC Educational Resources Information Center

Montgomery County Public Schools, 2010

2010-01-01

The Elementary Integrated Curriculum (EIC) Framework is the guiding curriculum document for the Elementary Integrated Curriculum and represents the elementary portion of the Montgomery County (Maryland) Public Schools (MCPS) Pre-K-12 Curriculum Frameworks. The EIC Framework contains the detailed indicators and objectives that describe what…

Towards a Holistic Framework for the Evaluation of Emergency Plans in Indoor Environments

PubMed Central

Serrano, Emilio; Poveda, Geovanny; Garijo, Mercedes

2014-01-01

One of the most promising fields for ambient intelligence is the implementation of intelligent emergency plans. Because the use of drills and living labs cannot reproduce social behaviors, such as panic attacks, that strongly affect these plans, the use of agent-based social simulation provides an approach to evaluate these plans more thoroughly. (1) The hypothesis presented in this paper is that there has been little interest in describing the key modules that these simulators must include, such as formally represented knowledge and a realistic simulated sensor model, and especially in providing researchers with tools to reuse, extend and interconnect modules from different works. This lack of interest hinders researchers from achieving a holistic framework for evaluating emergency plans and forces them to reconsider and to implement the same components from scratch over and over. In addition to supporting this hypothesis by considering over 150 simulators, this paper: (2) defines the main modules identified and proposes the use of semantic web technologies as a cornerstone for the aforementioned holistic framework; (3) provides a basic methodology to achieve the framework; (4) identifies the main challenges; and (5) presents an open and free software tool to hint at the potential of such a holistic view of emergency plan evaluation in indoor environments. PMID:24662453

Applying a Framework to Evaluate Assignment Marking Software: A Case Study on Lightwork

ERIC Educational Resources Information Center

Heinrich, Eva; Milne, John

2012-01-01

This article presents the findings of a qualitative evaluation on the effect of a specialised software tool on the efficiency and quality of assignment marking. The software, Lightwork, combines with the Moodle learning management system and provides support through marking rubrics and marker allocations. To enable the evaluation a framework has…

Testing thermal comfort of trekking boots: an objective and subjective evaluation.

PubMed

Arezes, P M; Neves, M M; Teixeira, S F; Leão, C P; Cunha, J L

2013-07-01

The study of the thermal comfort of the feet when using a specific type of shoe is of paramount importance, in particular if the main goal of the study is to attend to the needs of users. The main aim of this study was to propose a test battery for thermal comfort analysis and to apply it to the analysis of trekking boots. Methodologically, the project involves both objective and subjective evaluations. An objective evaluation of the thermal properties of the fabrics used in the boots was developed and applied. In addition, the thermal comfort provided when using the boots was also assessed both subjective and objectively. The evaluation of the thermal comfort during use, which was simulated in a laboratory environment, included the measurement of the temperature and moisture of the feet. The subjective assessment was performed using a questionnaire. From the results obtained, it was possible to define an optimal combination of fabrics to apply to trekking boots by considering the provided thermal insulation, air permeability and wicking. The results also revealed that the subjective perception of thermal comfort appears to be more related to the increase in temperature of the feet than to the moisture retention inside the boot. Although the evaluation of knits used in the boots indicated that a particular combination of fibres was optimal for use in the inner layer, the subjective and objective evaluation of thermal comfort revealed that the evaluation provided by users did not necessarily match the technical assessment data. No correlation was observed between the general comfort and specific thermal comfort assessments. Finally, the identification of thermal discomfort by specific foot areas would be useful in the process of designing and developing boots. Copyright © 2012 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Objects as closures - Abstract semantics of object oriented languages

NASA Technical Reports Server (NTRS)

Reddy, Uday S.

1988-01-01

The denotational semantics of object-oriented languages is discussed using the concept of closure widely used in (semi) functional programming to encapsulate side effects. It is shown that this denotational framework is adequate to explain classes, instantiation, and inheritance in the style of Simula as well as SMALLTALK-80. This framework is then compared with that of Kamin (1988), in his recent denotational definition of SMALLTALK-80, and the implications of the differences between the two approaches are discussed.

Objects as closures: Abstract semantics of object oriented languages

NASA Technical Reports Server (NTRS)

Reddy, Uday S.

1989-01-01

We discuss denotational semantics of object-oriented languages, using the concept of closure widely used in (semi) functional programming to encapsulate side effects. It is shown that this denotational framework is adequate to explain classes, instantiation, and inheritance in the style of Simula as well as SMALLTALK-80. This framework is then compared with that of Kamin, in his recent denotational definition of SMALLTALK-80, and the implications of the differences between the two approaches are discussed.

Community College Management by Objectives: Process, Progress, Problems.

ERIC Educational Resources Information Center

Deegan, William L.; And Others

The objectives of this book are: (1) to present a theoretical framework for management by objectives in community colleges, (2) to present information about alternative methods for conducting needs assessment and implementing management by objectives, (3) to present a framework for integrating academic and fiscal planning through management by…

Development and application of the RE-AIM QuEST mixed methods framework for program evaluation.

PubMed

Forman, Jane; Heisler, Michele; Damschroder, Laura J; Kaselitz, Elizabeth; Kerr, Eve A

2017-06-01

To increase the likelihood of successful implementation of interventions and promote dissemination across real-world settings, it is essential to evaluate outcomes related to dimensions other than Effectiveness alone. Glasgow and colleagues' RE-AIM framework specifies four additional types of outcomes that are important to decision-makers: Reach, Adoption, Implementation (including cost), and Maintenance. To further strengthen RE-AIM, we propose integrating qualitative assessments in an expanded framework: RE-AIM Qualitative Evaluation for Systematic Translation (RE-AIM QuEST), a mixed methods framework. RE-AIM QuEST guides formative evaluation to identify real-time implementation barriers and explain how implementation context may influence translation to additional settings. RE-AIM QuEST was used to evaluate a pharmacist-led hypertension management intervention at 3 VA facilities in 2008-2009. We systematically reviewed each of the five RE-AIM dimensions and created open-ended companion questions to quantitative measures and identified qualitative and quantitative data sources, measures, and analyses. To illustrate use of the RE-AIM QuEST framework, we provide examples of real-time, coordinated use of quantitative process measures and qualitative methods to identify site-specific issues, and retrospective use of these data sources and analyses to understand variation across sites and explain outcomes. For example, in the Reach dimension, we conducted real-time measurement of enrollment across sites and used qualitative data to better understand and address barriers at a low-enrollment site. The RE-AIM QuEST framework may be a useful tool for improving interventions in real-time, for understanding retrospectively why an intervention did or did not work, and for enhancing its sustainability and translation to other settings.

Evaluating health inequity interventions: applying a contextual (external) validity framework to programs funded by the Canadian Health Services Research Foundation.

PubMed

Phillips, Kaye; Müller-Clemm, Werner; Ysselstein, Margaretha; Sachs, Jonathan

2013-02-01

Including context in the measurement and evaluation of health in equity interventions is critical to understanding how events that occur in an intervention's environment might contribute to or impede its success. This study adapted and piloted a contextual validity assessment framework on a selection of health inequity-related programs funded by the Canadian Health Services Research Foundation (CHSRF) between 1998 and 2006. The two overarching objectives of this study were (1) to determine the relative amount and quality of attention given to conceptualizing, measuring and validating context within CHSRF funded research final reports related to health-inequity; and (2) to contribute evaluative evidence towards the incorporation of context into the assessment and measurement of health inequity interventions. The study found that of the 42/146 CHSRF programs and projects, judged to be related to health inequity 20 adequately reported on the conceptualization, measurement and validation of context. Amongst these health-inequity related project reports, greatest emphasis was placed on describing the socio-political and economical context over actually measuring and validating contextual evidence. Applying a contextual validity assessment framework was useful for distinguishing between the descriptive (conceptual) versus empirical (measurement and validation) inclusion of documented contextual evidence. Although contextual validity measurement frameworks needs further development, this study contributes insight into identifying funded research related to health inequities and preliminary criteria for assessing interventions targeted at specific populations and jurisdictions. This study also feeds a larger critical dialogue (albeit beyond the scope of this study) regarding the relevance and utility of using evaluative techniques for understanding how specific external conditions support or impede the successful implementation of health inequity interventions. Copyright �

Objective measurements to evaluate glottal space segmentation from laryngeal images.

PubMed

Gutiérrez-Arriola, J M; Osma-Ruiz, V; Sáenz-Lechón, N; Godino-Llorente, J I; Fraile, R; Arias-Londoño, J D

2012-01-01

Objective evaluation of the results of medical image segmentation is a known problem. Applied to the task of automatically detecting the glottal area from laryngeal images, this paper proposes a new objective measurement to evaluate the quality of a segmentation algorithm by comparing with the results given by a human expert. The new figure of merit is called Area Index, and its effectiveness is compared with one of the most used figures of merit found in the literature: the Pratt Index. Results over 110 laryngeal images presented high correlations between both indexes, demonstrating that the proposed measure is comparable to the Pratt Index and it is a good indicator of the segmentation quality.

Subjective and objective voice evaluation in Sjögren's syndrome.

PubMed

Saltürk, Ziya; Özdemir, Erdi; Kumral, Tolgar Lütfi; Karabacakoğlu, Zeynep; Kumral, Esra; Yildiz, Hatice Elvin; Mersinlioğlu, Gökhan; Atar, Yavuz; Berkiten, Güler; Yildirim, Güven; Uyar, Yavuz

2017-04-01

Objective The aim of this study is to assess the subjective and objective aspects of voice in Sjögren's syndrome. Methods The study enrolled 10 women with Sjögren's syndrome and 12 healthy women. Maximum phonation time, fundamental frequency, jitter, shimmer, and noise-to-harmonics ratio were determined during acoustic voice analysis. The Stroboscopy Evaluation Rating Form was used for the laryngostroboscopic evaluation. A subjective evaluation was performed using the Turkish version of Voice Handicap Index-10. Results The mean age of the Sjögren's syndrome and control groups was 46 ± 13.89 and 41.27 ± 6.99 years, respectively, and did not differ (P = 0.131). In the laryngostroboscopic evaluation, the smoothness and straightness of vocal folds, regularity, and glottal closure differed significantly. In the acoustic and aerodynamic analyses, none of the parameters differed statistically, while the Sjögren's syndrome group had significantly higher Voice Handicap Index-10 scores than the controls. Conclusion Sjögren's syndrome affects the voice and voice quality.

A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals.

PubMed

Vandenberg, Laura N; Ågerstrand, Marlene; Beronius, Anna; Beausoleil, Claire; Bergman, Åke; Bero, Lisa A; Bornehag, Carl-Gustaf; Boyer, C Scott; Cooper, Glinda S; Cotgreave, Ian; Gee, David; Grandjean, Philippe; Guyton, Kathryn Z; Hass, Ulla; Heindel, Jerrold J; Jobling, Susan; Kidd, Karen A; Kortenkamp, Andreas; Macleod, Malcolm R; Martin, Olwenn V; Norinder, Ulf; Scheringer, Martin; Thayer, Kristina A; Toppari, Jorma; Whaley, Paul; Woodruff, Tracey J; Rudén, Christina

2016-07-14

The issue of endocrine disrupting chemicals (EDCs) is receiving wide attention from both the scientific and regulatory communities. Recent analyses of the EDC literature have been criticized for failing to use transparent and objective approaches to draw conclusions about the strength of evidence linking EDC exposures to adverse health or environmental outcomes. Systematic review methodologies are ideal for addressing this issue as they provide transparent and consistent approaches to study selection and evaluation. Objective methods are needed for integrating the multiple streams of evidence (epidemiology, wildlife, laboratory animal, in vitro, and in silico data) that are relevant in assessing EDCs. We have developed a framework for the systematic review and integrated assessment (SYRINA) of EDC studies. The framework was designed for use with the International Program on Chemical Safety (IPCS) and World Health Organization (WHO) definition of an EDC, which requires appraisal of evidence regarding 1) association between exposure and an adverse effect, 2) association between exposure and endocrine disrupting activity, and 3) a plausible link between the adverse effect and the endocrine disrupting activity. Building from existing methodologies for evaluating and synthesizing evidence, the SYRINA framework includes seven steps: 1) Formulate the problem; 2) Develop the review protocol; 3) Identify relevant evidence; 4) Evaluate evidence from individual studies; 5) Summarize and evaluate each stream of evidence; 6) Integrate evidence across all streams; 7) Draw conclusions, make recommendations, and evaluate uncertainties. The proposed method is tailored to the IPCS/WHO definition of an EDC but offers flexibility for use in the context of other definitions of EDCs. When using the SYRINA framework, the overall objective is to provide the evidence base needed to support decision making, including any action to avoid/minimise potential adverse effects of exposures. This

Miniature objective lens for array digital pathology: design improvement based on clinical evaluation

NASA Astrophysics Data System (ADS)

McCall, Brian; Pierce, Mark; Graviss, Edward A.; Richards-Kortum, Rebecca R.; Tkaczyk, Tomasz S.

2016-03-01

A miniature objective designed for digital detection of Mycobacterium tuberculosis (MTB) was evaluated for diagnostic accuracy. The objective was designed for array microscopy, but fabricated and evaluated at this stage of development as a single objective. The counts and diagnoses of patient samples were directly compared for digital detection and standard microscopy. The results were found to be correlated and highly concordant. The evaluation of this lens by direct comparison to standard fluorescence sputum smear microscopy presented unique challenges and led to some new insights in the role played by the system parameters of the microscope. The design parameters and how they were developed are reviewed in light of these results. New system parameters are proposed with the goal of easing the challenges of evaluating the miniature objective and maintaining the optical performance that produced the agreeable results presented without over-optimizing. A new design is presented that meets and exceeds these criteria.

Method and Application for Dynamic Comprehensive Evaluation with Subjective and Objective Information

PubMed Central

Liu, Dinglin; Zhao, Xianglian

2013-01-01

In an effort to deal with more complicated evaluation situations, scientists have focused their efforts on dynamic comprehensive evaluation research. How to make full use of the subjective and objective information has become one of the noteworthy content. In this paper, a dynamic comprehensive evaluation method with subjective and objective information is proposed. We use the combination weighting method to determine the index weight. Analysis hierarchy process method is applied to dispose the subjective information, and criteria importance through intercriteria correlation method is used to handle the objective information. And for the time weight determination, we consider both time distance and information size to embody the principle of esteeming the present over the past. And then the linear weighted average model is constructed to make the evaluation process more practicable. Finally, an example is presented to illustrate the effectiveness of this method. Overall, the results suggest that the proposed method is reasonable and effective. PMID:24386176

Classroom Teacher's Performance-Based Evaluation Form (CTPBEF) for Public Education Schools in the State of Kuwait: A Framework

ERIC Educational Resources Information Center

Al-Shammari, Zaid; Yawkey, Thomas D.

2008-01-01

This investigation using Grounded Theory focuses on developing, designing and testing out an evaluation method used as a framework for this study. This framework evolved into the instrument entitled, "Classroom Teacher's Performance Based Evaluation Form (CTPBEF)". This study shows the processes and procedures used in CTPBEF's…

Sociotechnical systems as a framework for regulatory system design and evaluation: Using Work Domain Analysis to examine a new regulatory system.

PubMed

Carden, Tony; Goode, Natassia; Read, Gemma J M; Salmon, Paul M

2017-03-15

Like most work systems, the domain of adventure activities has seen a series of serious incidents and subsequent calls to improve regulation. Safety regulation systems aim to promote safety and reduce accidents. However, there is scant evidence they have led to improved safety outcomes. In fact there is some evidence that the poor integration of regulatory system components has led to adverse safety outcomes in some contexts. Despite this, there is an absence of methods for evaluating regulatory and compliance systems. This article argues that sociotechnical systems theory and methods provide a suitable framework for evaluating regulatory systems. This is demonstrated through an analysis of a recently introduced set of adventure activity regulations. Work Domain Analysis (WDA) was used to describe the regulatory system in terms of its functional purposes, values and priority measures, purpose-related functions, object-related processes and cognitive objects. This allowed judgement to be made on the nature of the new regulatory system and on the constraints that may impact its efficacy following implementation. Importantly, the analysis suggests that the new system's functional purpose of ensuring safe activities is not fully supported in terms of the functions and objects available to fulfil them. Potential improvements to the design of the system are discussed along with the implications for regulatory system design and evaluation across the safety critical domains generally. Copyright © 2017 Elsevier Ltd. All rights reserved.

Evaluating Intervention Effects in a Diagnostic Classification Model Framework

ERIC Educational Resources Information Center

Madison, Matthew J.; Bradshaw, Laine

2018-01-01

The evaluation of intervention effects is an important objective of educational research. One way to evaluate the effectiveness of an intervention is to conduct an experiment that assigns individuals to control and treatment groups. In the context of pretest/posttest designed studies, this is referred to as a control-group pretest/posttest design.…

Bridging informatics and implementation science: evaluating a framework to assess electronic health record implementations in community settings.

PubMed

Richardson, Joshua E; Abramson, Erika L; Pfoh, Elizabeth R; Kaushal, Rainu

2012-01-01

Effective electronic health record (EHR) implementations in community settings are critical to promoting safe and reliable EHR use as well as mitigating provider dissatisfaction that often results. The implementation challenge is compounded given the scale and scope of EHR installations that are occurring and will continue to occur over the next five years. However, when compared to EHR evaluations relatively few biomedical informatics researchers have published on evaluating EHR implementations. Fewer still have evaluated EHR implementations in community settings. We report on the methods we used to achieve a novel application of an implementation science framework in informatics to qualitatively evaluate community-based EHR implementations. We briefly provide an overview of the implementation science framework, our methods for adapting it to informatics, the effects the framework had on our qualitative methods of inquiry and analysis, and discuss its potential value for informatics research.

Linkage of additional contents to moving objects and video shots in a generic media framework for interactive television

NASA Astrophysics Data System (ADS)

Lopez, Alejandro; Noe, Miquel; Fernandez, Gabriel

2004-10-01

The GMF4iTV project (Generic Media Framework for Interactive Television) is an IST European project that consists of an end-to-end broadcasting platform providing interactivity on heterogeneous multimedia devices such as Set-Top-Boxes and PCs according to the Multimedia Home Platform (MHP) standard from DVB. This platform allows the content providers to create enhanced audiovisual contents with a degree of interactivity at moving object level or shot change from a video. The end user is then able to interact with moving objects from the video or individual shots allowing the enjoyment of additional contents associated to them (MHP applications, HTML pages, JPEG, MPEG4 files...). This paper focus the attention to the issues related to metadata and content transmission, synchronization, signaling and bitrate allocation of the GMF4iTV project.

An overview of ethical frameworks in public health: can they be supportive in the evaluation of programs to prevent overweight?

PubMed Central

2010-01-01

Background The prevention of overweight sometimes raises complex ethical questions. Ethical public health frameworks may be helpful in evaluating programs or policy for overweight prevention. We give an overview of the purpose, form and contents of such public health frameworks and investigate to which extent they are useful for evaluating programs to prevent overweight and/or obesity. Methods Our search for frameworks consisted of three steps. Firstly, we asked experts in the field of ethics and public health for the frameworks they were aware of. Secondly, we performed a search in Pubmed. Thirdly, we checked literature references in the articles on frameworks we found. In total, we thus found six ethical frameworks. We assessed the area on which the available ethical frameworks focus, the users they target at, the type of policy or intervention they propose to address, and their aim. Further, we looked at their structure and content, that is, tools for guiding the analytic process, the main ethical principles or values, possible criteria for dealing with ethical conflicts, and the concrete policy issues they are applied to. Results All frameworks aim to support public health professionals or policymakers. Most of them provide a set of values or principles that serve as a standard for evaluating policy. Most frameworks articulate both the positive ethical foundations for public health and ethical constraints or concerns. Some frameworks offer analytic tools for guiding the evaluative process. Procedural guidelines and concrete criteria for solving important ethical conflicts in the particular area of the prevention of overweight or obesity are mostly lacking. Conclusions Public health ethical frameworks may be supportive in the evaluation of overweight prevention programs or policy, but seem to lack practical guidance to address ethical conflicts in this particular area. PMID:20969761

A Conceptual Framework to Help Evaluate the Quality of Institutional Performance

ERIC Educational Resources Information Center

Kettunen, Juha

2008-01-01

Purpose: This study aims to present a general conceptual framework which can be used to evaluate quality and institutional performance in higher education. Design/methodology/approach: The quality of higher education is at the heart of the setting up of the European Higher Education Area. Strategic management is widely used in higher education…

Evaluating social outcomes of HIV/AIDS interventions: a critical assessment of contemporary indicator frameworks

PubMed Central

Mannell, Jenevieve; Cornish, Flora; Russell, Jill

2014-01-01

Introduction Contemporary HIV-related theory and policy emphasize the importance of addressing the social drivers of HIV risk and vulnerability for a long-term response. Consequently, increasing attention is being given to social and structural interventions, and to social outcomes of HIV interventions. Appropriate indicators for social outcomes are needed in order to institutionalize the commitment to addressing social outcomes. This paper critically assesses the current state of social indicators within international HIV/AIDS monitoring and evaluation frameworks. Methods We analyzed the indicator frameworks of six international organizations involved in efforts to improve and synchronize the monitoring and evaluation of the HIV/AIDS response. Our analysis classifies the 328 unique indicators according to what they measure and assesses the degree to which they offer comprehensive measurement across three dimensions: domains of the social context, levels of change and organizational capacity. Results and discussion The majority of indicators focus on individual-level (clinical and behavioural) interventions and outcomes, neglecting structural interventions, community interventions and social outcomes (e.g. stigma reduction; community capacity building; policy-maker sensitization). The main tool used to address social aspects of HIV/AIDS is the disaggregation of data by social group. This raises three main limitations. Indicator frameworks do not provide comprehensive coverage of the diverse social drivers of the epidemic, particularly neglecting criminalization, stigma, discrimination and gender norms. There is a dearth of indicators for evaluating the social impacts of HIV interventions. Indicators of organizational capacity focus on capacity to effectively deliver and manage clinical services, neglecting capacity to respond appropriately and sustainably to complex social contexts. Conclusions Current indicator frameworks cannot adequately assess the social

Framework of behavioral indicators for outcome evaluation of TB health promotion: a Delphi study of TB suspects and Tb patients

PubMed Central

2014-01-01

Background Health promotion for prevention and control of Tuberculosis (TB) is implemented worldwide because of its importance, but few reports have evaluated its impact on behavior due to a lack of standard outcome indicators. The objective of this study was to establish a framework of behavioral indicators for outcome evaluation of TB health promotion among TB suspects and patients. Methods A two-round modified Delphi method involving sixteen TB control experts was used to establish a framework of behavioral indicators for outcome evaluation of TB health promotion targeted at TB suspects and patients. Results Sixteen of seventeen invited experts in TB control (authority score of 0.91 on a 1.0 scale) participated in round 1 survey. All sixteen experts also participated in a second round survey. After two rounds of surveys and several iterations among the experts, there was consensus on a framework of indicators for measuring outcomes of TB health promotion for TB suspects and patients. For TB suspects, the experts reached consensus on 2 domains (“Healthcare seeking behavior” and “Transmission prevention”), 3 subdomains (“Seeking care after onset of TB symptoms”, “Pathways of seeking care” and “Interpersonal contact etiquette”), and 8 indicators (including among others, “Length of patient delay”). For TB patients, consensus was reached on 3 domains (“Adherence to treatment”, “Healthy lifestyle” and “Transmission prevention”), 8 subdomains (including among others, “Adherence to their medication”), and 14 indicators (including “Percentage of patients who adhered to their medication”). Operational definitions and data sources were provided for each indicator. Conclusions The findings of this study provide the basis for debate among international experts on a framework for achieving global consensus on outcome indicators for TB health promotion interventions targeted at TB patients and suspects. Such consensus will help to

Evaluating Public Spending: A Framework of Public Expenditure Reviews. World Bank Discussion Papers No. 323.

ERIC Educational Resources Information Center

Pradhan, Sanjay

This paper presents a framework for evaluating the level and composition of public expenditures, illustrated by sectoral and country examples. The paper illustrates how this framework can be applied to analyzing broad allocations of spending within and across sectors, drawing upon some key findings and country examples from major sectors (health,…

Evaluating an Objective Structured Clinical Examination (OSCE) Adapted for Social Work

ERIC Educational Resources Information Center

Bogo, Marion; Regehr, Cheryl; Katz, Ellen; Logie, Carmen; Tufford, Lea; Litvack, Andrea

2012-01-01

Objectives: To evaluate an objective structured clinical examination (OSCE) adapted for social work in a lab course and examine the degree to which it predicts competence in the practicum. Method: 125 Masters students participated in a one-scenario OSCE and wrote responses to standardized reflection questions. OSCE performance and reflections were…

Using the RE-AIM framework to evaluate physical activity public health programs in Mexico

USDA-ARS?s Scientific Manuscript database

Physical activity (PA) public health programming has been widely used in Mexico; however, few studies have documented individual and organizational factors that might be used to evaluate their public health impact. The RE-AIM framework is an evaluation tool that examines individual and organizationa...

The Social Outcomes of Older Adult Learning in Taiwan: Evaluation Framework and Indicators

ERIC Educational Resources Information Center

Lin, Li-Hui

2015-01-01

The purpose of this study is to explore the social outcomes of older adult learning in Taiwan. In light of our society's aging population structure, the task of establishing evaluation framework and indicators for the social outcomes of learning (SOL) as applied to older adults is urgent. In order to construct evaluation indicators for older adult…

Investigating Music Information Objects

ERIC Educational Resources Information Center

Weissenberger, Lynnsey K.

2016-01-01

This dissertation, titled "Investigating Music Information Objects," is a study of the nature, description, representations, and ideas related to music information objects (MIOs). This research study investigates how music practitioners from various traditions describe and conceptualize MIOs, using a theoretical framework to classify…

Eliciting Values of Patients with Multiple Chronic Conditions: Evaluation of a Patient-centered Framework.

PubMed

Berry, Andrew B L; Lim, Catherine; Hartzler, Andrea L; Hirsch, Tad; Ludman, Evette; Wagner, Edward H; Ralston, James D

2017-01-01

Patients with multiple chronic conditions often face competing demands for care, and they often do not agree with physicians on priorities for care. Patients ' values shape their healthcare priorities, but existing methods for eliciting values do not necessarily meet patients ' care planning needs. We developed a patient-centered values framework based on a field study with patients and caregivers. In this paper we report on a survey to evaluate how the framework generalizes beyond field study participants, and how well the framework supports values elicitation. We found that respondents frame values in a way that is consistent with the framework, and that domains of the framework can be used to elicit a breadth of potential values individuals with MCC express. These findings demonstrate how a patient-centered perspective on values can expand on the domains considered in values clarification methods andfacilitate patient-provider communication in establishing shared care priorities.

A Framework for Evaluating and Enhancing Alignment in Self-Regulated Learning Research

PubMed Central

Dent, Amy L.; Hoyle, Rick H.

2015-01-01

We discuss the articles of this special issue with reference to an important yet previously only implicit dimension of study quality: alignment across the theoretical and methodological decisions that collectively define an approach to self-regulated learning. Integrating and extending work by leaders in the field, we propose a framework for evaluating alignment in the way self-regulated learning research is both conducted and reported. Within this framework, the special issue articles provide a springboard for discussing methodological promises and pitfalls of increasingly sophisticated research on the dynamic, contingent, and contextualized features of self-regulated learning. PMID:25825589

An Evaluation of Database Solutions to Spatial Object Association

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kumar, V S; Kurc, T; Saltz, J

2008-06-24

Object association is a common problem encountered in many applications. Spatial object association, also referred to as crossmatch of spatial datasets, is the problem of identifying and comparing objects in two datasets based on their positions in a common spatial coordinate system--one of the datasets may correspond to a catalog of objects observed over time in a multi-dimensional domain; the other dataset may consist of objects observed in a snapshot of the domain at a time point. The use of database management systems to the solve the object association problem provides portability across different platforms and also greater flexibility. Increasingmore » dataset sizes in today's applications, however, have made object association a data/compute-intensive problem that requires targeted optimizations for efficient execution. In this work, we investigate how database-based crossmatch algorithms can be deployed on different database system architectures and evaluate the deployments to understand the impact of architectural choices on crossmatch performance and associated trade-offs. We investigate the execution of two crossmatch algorithms on (1) a parallel database system with active disk style processing capabilities, (2) a high-throughput network database (MySQL Cluster), and (3) shared-nothing databases with replication. We have conducted our study in the context of a large-scale astronomy application with real use-case scenarios.« less

A novel framework for objective detection and tracking of TC center from noisy satellite imagery

NASA Astrophysics Data System (ADS)

Johnson, Bibin; Thomas, Sachin; Rani, J. Sheeba

2018-07-01

This paper proposes a novel framework for automatically determining and tracking the center of a tropical cyclone (TC) during its entire life-cycle from the Thermal infrared (TIR) channel data of the geostationary satellite. The proposed method handles meteorological images with noise, missing or partial information due to the seasonal variability and lack of significant spatial or vortex features. To retrieve the cyclone center from these circumstances, a synergistic approach based on objective measures and Numerical Weather Prediction (NWP) model is being proposed. This method employs a spatial gradient scheme to process missing and noisy frames or a spatio-temporal gradient scheme for image sequences that are continuous and contain less noise. The initial estimate of the TC center from the missing imagery is corrected by exploiting a NWP model based post-processing scheme. The validity of the framework is tested on Infrared images of different cyclones obtained from various Geostationary satellites such as the Meteosat-7, INSAT- 3 D , Kalpana-1 etc. The computed track is compared with the actual track data obtained from Joint Typhoon Warning Center (JTWC), and it shows a reduction of mean track error by 11 % as compared to the other state of the art methods in the presence of missing and noisy frames. The proposed method is also successfully tested for simultaneous retrieval of the TC center from images containing multiple non-overlapping cyclones.

An Approach to a Comprehensive Test Framework for Analysis and Evaluation of Text Line Segmentation Algorithms

PubMed Central

Brodic, Darko; Milivojevic, Dragan R.; Milivojevic, Zoran N.

2011-01-01

The paper introduces a testing framework for the evaluation and validation of text line segmentation algorithms. Text line segmentation represents the key action for correct optical character recognition. Many of the tests for the evaluation of text line segmentation algorithms deal with text databases as reference templates. Because of the mismatch, the reliable testing framework is required. Hence, a new approach to a comprehensive experimental framework for the evaluation of text line segmentation algorithms is proposed. It consists of synthetic multi-like text samples and real handwritten text as well. Although the tests are mutually independent, the results are cross-linked. The proposed method can be used for different types of scripts and languages. Furthermore, two different procedures for the evaluation of algorithm efficiency based on the obtained error type classification are proposed. The first is based on the segmentation line error description, while the second one incorporates well-known signal detection theory. Each of them has different capabilities and convenience, but they can be used as supplements to make the evaluation process efficient. Overall the proposed procedure based on the segmentation line error description has some advantages, characterized by five measures that describe measurement procedures. PMID:22164106

An approach to a comprehensive test framework for analysis and evaluation of text line segmentation algorithms.

PubMed

Brodic, Darko; Milivojevic, Dragan R; Milivojevic, Zoran N

2011-01-01

The paper introduces a testing framework for the evaluation and validation of text line segmentation algorithms. Text line segmentation represents the key action for correct optical character recognition. Many of the tests for the evaluation of text line segmentation algorithms deal with text databases as reference templates. Because of the mismatch, the reliable testing framework is required. Hence, a new approach to a comprehensive experimental framework for the evaluation of text line segmentation algorithms is proposed. It consists of synthetic multi-like text samples and real handwritten text as well. Although the tests are mutually independent, the results are cross-linked. The proposed method can be used for different types of scripts and languages. Furthermore, two different procedures for the evaluation of algorithm efficiency based on the obtained error type classification are proposed. The first is based on the segmentation line error description, while the second one incorporates well-known signal detection theory. Each of them has different capabilities and convenience, but they can be used as supplements to make the evaluation process efficient. Overall the proposed procedure based on the segmentation line error description has some advantages, characterized by five measures that describe measurement procedures.

Evaluate, assess, treat: development and evaluation of the EAT framework to increase effective communication regarding sensitive oral-systemic health issues.

PubMed

DeBate, R D; Cragun, D; Gallentine, A A; Severson, H H; Shaw, T; Cantwell, C; Christiansen, S; Koerber, A; Hendricson, W; Tomar, S L; McCormack Brown, K; Tedesco, L A

2012-11-01

Oral healthcare providers are likely to encounter a number of sensitive oral/systemic health issues whilst interacting with patients. The purpose of the current study was to develop and evaluate a framework aimed at oral healthcare providers to engage in active secondary prevention of eating disorders (i.e. early detection of oral manifestations of disordered eating behaviours, patient approach and communication, patient-specific oral treatment, and referral to care) for patients presenting with signs of disordered eating behaviours. The EAT Framework was developed based on the Brief Motivational Interviewing (B-MI) conceptual framework and comprises three continuous steps: Evaluating, Assessing, and Treating. Using a group-randomized control design, 11 dental hygiene (DH) and seven dental (D) classes from eight institutions were randomized to either the intervention or control conditions. Both groups completed pre- and post-intervention assessments. Hierarchical linear models were conducted to measure the effects of the intervention whilst controlling for baseline levels. Statistically significant improvements from pre- to post-intervention were observed in the Intervention group compared with the Control group on knowledge of eating disorders and oral findings, skills-based knowledge, and self-efficacy (all P < 0.01). Effect sizes ranged from 0.57 to 0.95. No statistically significant differences in outcomes were observed by type of student. Although the EAT Framework was developed as part of a larger study on secondary prevention of eating disorders, the procedures and skills presented can be applied to other sensitive oral/systemic health issues. Because the EAT Framework was developed by translating B-MI principles and procedures, the framework can be easily adopted as a non-confrontational method for patient communication. © 2012 John Wiley & Sons A/S.

Evaluate, Assess, Treat: Development and evaluation of the EAT framework to increase effective communication regarding sensitive oral-systemic health issues

PubMed Central

DeBate, Rita D.; Cragun, Deborah; Gallentine, Ashley A.; Severson, Herbert H.; Shaw, Tracey; Cantwell, Carley; Christiansen, Steve; Koerber, Anne; Hendricson, William; Tomar, Scott L.; Brown, Kelli McCormack; Tedesco, Lisa A.

2012-01-01

Oral healthcare providers are likely to encounter a number of sensitive oral/systemic health issues while interacting with patients. The purpose of the current study was to develop and evaluate a framework aimed at oral healthcare providers to engage in active secondary prevention of eating disorders (i.e., early detection of oral manifestations of disordered eating behaviors, patient approach and communication, patient-specific oral treatment, and referral to care) for patients presenting with signs of disordered eating behaviors. The EAT Framework was developed based on the Brief Motivational Interviewing (B-MI) conceptual framework and comprises three continuous steps: Evaluating, Assessing, and Treating. Using a group-randomized control design, 11 dental hygiene (DH) and 7 dental (D) classes from 8 institutions were randomized to either the intervention or control conditions. Both groups completed preand post-intervention assessments. Hierarchical linear models were conducted to measure the effects of the intervention while controlling for baseline levels. Statistically significant improvements from pre-to post-intervention were observed in the Intervention group compared with the Control group on knowledge of eating disorders and oral findings, skills-based knowledge, and self-efficacy (all p < .01). Effect sizes ranged from .57–.95. No statistically significant differences in outcomes were observed by type of student. Although the EAT Framework was developed as part of a larger study on secondary prevention of eating disorders, the procedures and skills presented can be applied to other sensitive oral/systemic health issues. Because the EAT Framework was developed by translating B-MI principles and procedures, the framework can be easily adopted as a non-confrontational method for patient communication. PMID:23050505

Australia's personally controlled electronic health record and primary healthcare: generating a framework for implementation and evaluation.

PubMed

Almond, H; Cummings, E; Turner, P

2013-01-01

The Australian Government launched a personally controlled electronic health record (PCEHR) system in July 2012 committing $466.7m. Currently Australia lacks a clearly articulated implementation and evaluation framework and there remains limited detail on how this system's success will be determined. These problems are especially visible in primary healthcare. The UK and US, have been advocated as models, however they have started to report points of failure arising from their approaches. Evidence suggests that alternatives need to be considered, if mistakes are not to be replicated. Insights from e-health record implementation and evaluation approaches in Denmark and the Netherlands provide Australia with other approaches. The PCEHR requires different and radical thinking around the delivery of health services. Drawing on a range of English language articles identified between 1996 and 2012, the paper generates a conceptual framework for implementation and evaluation of the PCEHR. The generation of a grounded implementation and evaluation framework in primary healthcare will reduce provider scepticism and facilitate complex changes associated with PCEHR uptake.

Evaluating handoffs in the context of a communication framework.

PubMed

Hasan, Hani; Ali, Fadwa; Barker, Paul; Treat, Robert; Peschman, Jacob; Mohorek, Matthew; Redlich, Philip; Webb, Travis

2017-03-01

The implementation of mandated restrictions in resident duty hours has led to increased handoffs for patient care and thus more opportunities for errors during transitions of care. Much of the current handoff literature is empiric, with experts recommending the study of handoffs within an established framework. A prospective, single-institution study was conducted evaluating the process of handoffs for the care of surgical patients in the context of a published communication framework. Evaluation tools for the source, receiver, and observer were developed to identify factors impacting the handoff process, and inter-rater correlations were assessed. Data analysis was generated with Pearson/Spearman correlations and multivariate linear regressions. Rater consistency was assessed with intraclass correlations. A total of 126 handoffs were observed. Evaluations were completed by 1 observer (N = 126), 2 observers (N = 23), 2 receivers (N = 39), 1 receiver (N = 82), and 1 source (N = 78). An average (±standard deviation) service handoff included 9.2 (±4.6) patients, lasted 9.1 (±5.4) minutes, and had 4.7 (±3.4) distractions recorded by the observer. The source and receiver(s) recognized distractions in >67% of handoffs, with the most common internal and external distractions being fatigue (60% of handoffs) and extraneous staff entering/exiting the room (31%), respectively. Teams with more patients spent less time per individual patient handoff (r = -0.298; P = .001). Statistically significant intraclass correlations (P ≤ .05) were moderate between observers (r ≥ 0.4) but not receivers (r < 0.4). Intraclass correlation values between different types of raters were inconsistent (P > .05). The quality of the handoff process was affected negatively by presence of active electronic devices (β = -0.565; P = .005), number of teaching discussions (β = -0.417; P = .048), and a sense of hierarchy between source and receiver (β = -0.309; P�

Evaluating the inverse reasoning account of object discovery.

PubMed

Carroll, Christopher D; Kemp, Charles

2015-06-01

People routinely make inferences about unobserved objects. A hotel guest with welts on his arms, for example, will often worry about bed bugs. The discovery of unobserved objects almost always involves a backward inference from some observed effects (e.g., welts) to unobserved causes (e.g., bed bugs). The inverse reasoning account, which is typically formalized as Bayesian inference, posits that the strength of a backward inference is closely connected to the strength of the corresponding forward inference from the unobserved causes to the observed effects. We evaluated the inverse reasoning account of object discovery in three experiments where participants were asked to discover the unobserved "attractors" and "repellers" that controlled a "particle" moving within an arena. Experiments 1 and 2 showed that participants often failed to provide the best explanations for various particle motions, even when the best explanations were simple and when participants enthusiastically endorsed these explanations when presented with them. This failure demonstrates that object discovery is critically dependent on the processes that support hypothesis generation-processes that the inverse reasoning account does not explain. Experiment 3 demonstrated that people sometimes generate explanations that are invalid even according to their own forward inferences, suggesting that the psychological processes that support forward and backward inference are less intertwined than the inverse reasoning account suggests. The experimental findings support an alternative account of object discovery in which people rely on heuristics to generate possible explanations. Copyright © 2015 Elsevier B.V. All rights reserved.

Basic test framework for the evaluation of text line segmentation and text parameter extraction.

PubMed

Brodić, Darko; Milivojević, Dragan R; Milivojević, Zoran

2010-01-01

Text line segmentation is an essential stage in off-line optical character recognition (OCR) systems. It is a key because inaccurately segmented text lines will lead to OCR failure. Text line segmentation of handwritten documents is a complex and diverse problem, complicated by the nature of handwriting. Hence, text line segmentation is a leading challenge in handwritten document image processing. Due to inconsistencies in measurement and evaluation of text segmentation algorithm quality, some basic set of measurement methods is required. Currently, there is no commonly accepted one and all algorithm evaluation is custom oriented. In this paper, a basic test framework for the evaluation of text feature extraction algorithms is proposed. This test framework consists of a few experiments primarily linked to text line segmentation, skew rate and reference text line evaluation. Although they are mutually independent, the results obtained are strongly cross linked. In the end, its suitability for different types of letters and languages as well as its adaptability are its main advantages. Thus, the paper presents an efficient evaluation method for text analysis algorithms.

Basic Test Framework for the Evaluation of Text Line Segmentation and Text Parameter Extraction

PubMed Central

Brodić, Darko; Milivojević, Dragan R.; Milivojević, Zoran

2010-01-01

Text line segmentation is an essential stage in off-line optical character recognition (OCR) systems. It is a key because inaccurately segmented text lines will lead to OCR failure. Text line segmentation of handwritten documents is a complex and diverse problem, complicated by the nature of handwriting. Hence, text line segmentation is a leading challenge in handwritten document image processing. Due to inconsistencies in measurement and evaluation of text segmentation algorithm quality, some basic set of measurement methods is required. Currently, there is no commonly accepted one and all algorithm evaluation is custom oriented. In this paper, a basic test framework for the evaluation of text feature extraction algorithms is proposed. This test framework consists of a few experiments primarily linked to text line segmentation, skew rate and reference text line evaluation. Although they are mutually independent, the results obtained are strongly cross linked. In the end, its suitability for different types of letters and languages as well as its adaptability are its main advantages. Thus, the paper presents an efficient evaluation method for text analysis algorithms. PMID:22399932

Research and Evaluations of the Health Aspects of Disasters, Part VII: The Relief/Recovery Framework.

PubMed

Birnbaum, Marvin L; Daily, Elaine K; O'Rourke, Ann P

2016-04-01

The principal goal of research relative to disasters is to decrease the risk that a hazard will result in a disaster. Disaster studies pursue two distinct directions: (1) epidemiological (non-interventional); and (2) interventional. Both interventional and non-interventional studies require data/information obtained from assessments of function. Non-interventional studies examine the epidemiology of disasters. Interventional studies evaluate specific interventions/responses in terms of their effectiveness in meeting their respective objectives, their contribution to the overarching goal, other effects created, their respective costs, and the efficiency with which they achieved their objectives. The results of interventional studies should contribute to evidence that will be used to inform the decisions used to define standards of care and best practices for a given setting based on these standards. Interventional studies are based on the Disaster Logic Model (DLM) and are used to change or maintain levels of function (LOFs). Relief and Recovery interventional studies seek to determine the effects, outcomes, impacts, costs, and value of the intervention provided after the onset of a damaging event. The Relief/Recovery Framework provides the structure needed to systematically study the processes involved in providing relief or recovery interventions that result in a new LOF for a given Societal System and/or its component functions. It consists of the following transformational processes (steps): (1) identification of the functional state prior to the onset of the event (pre-event); (2) assessments of the current functional state; (3) comparison of the current functional state with the pre-event state and with the results of the last assessment; (4) needs identification; (5) strategic planning, including establishing the overall strategic goal(s), objectives, and priorities for interventions; (6) identification of options for interventions; (7) selection of the most

Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK

ERIC Educational Resources Information Center

Rienties, Bart; Boroowa, Avinash; Cross, Simon; Kubiak, Chris; Mayles, Kevin; Murphy, Sam

2016-01-01

There is an urgent need to develop an evidence-based framework for learning analytics whereby stakeholders can manage, evaluate, and make decisions about which types of interventions work well and under which conditions. In this article, we will work towards developing a foundation of an Analytics4Action Evaluation Framework (A4AEF) that is…

A framework for evaluating mixture analysis algorithms

NASA Astrophysics Data System (ADS)

Dasaratha, Sridhar; Vignesh, T. S.; Shanmukh, Sarat; Yarra, Malathi; Botonjic-Sehic, Edita; Grassi, James; Boudries, Hacene; Freeman, Ivan; Lee, Young K.; Sutherland, Scott

2010-04-01

In recent years, several sensing devices capable of identifying unknown chemical and biological substances have been commercialized. The success of these devices in analyzing real world samples is dependent on the ability of the on-board identification algorithm to de-convolve spectra of substances that are mixtures. To develop effective de-convolution algorithms, it is critical to characterize the relationship between the spectral features of a substance and its probability of detection within a mixture, as these features may be similar to or overlap with other substances in the mixture and in the library. While it has been recognized that these aspects pose challenges to mixture analysis, a systematic effort to quantify spectral characteristics and their impact, is generally lacking. In this paper, we propose metrics that can be used to quantify these spectral features. Some of these metrics, such as a modification of variance inflation factor, are derived from classical statistical measures used in regression diagnostics. We demonstrate that these metrics can be correlated to the accuracy of the substance's identification in a mixture. We also develop a framework for characterizing mixture analysis algorithms, using these metrics. Experimental results are then provided to show the application of this framework to the evaluation of various algorithms, including one that has been developed for a commercial device. The illustration is based on synthetic mixtures that are created from pure component Raman spectra measured on a portable device.

Integration of electro-anatomical and imaging data of the left ventricle: An evaluation framework.

PubMed

Soto-Iglesias, David; Butakoff, Constantine; Andreu, David; Fernández-Armenta, Juan; Berruezo, Antonio; Camara, Oscar

2016-08-01

Integration of electrical and structural information for scar characterization in the left ventricle (LV) is a crucial step to better guide radio-frequency ablation therapies, which are usually performed in complex ventricular tachycardia (VT) cases. This integration requires finding a common representation where to map the electrical information from the electro-anatomical map (EAM) surfaces and tissue viability information from delay-enhancement magnetic resonance images (DE-MRI). However, the development of a consistent integration method is still an open problem due to the lack of a proper evaluation framework to assess its accuracy. In this paper we present both: (i) an evaluation framework to assess the accuracy of EAM and imaging integration strategies with simulated EAM data and a set of global and local measures; and (ii) a new integration methodology based on a planar disk representation where the LV surface meshes are quasi-conformally mapped (QCM) by flattening, allowing for simultaneous visualization and joint analysis of the multi-modal data. The developed evaluation framework was applied to estimate the accuracy of the QCM-based integration strategy on a benchmark dataset of 128 synthetically generated ground-truth cases presenting different scar configurations and EAM characteristics. The obtained results demonstrate a significant reduction in global overlap errors (50-100%) with respect to state-of-the-art integration techniques, also better preserving the local topology of small structures such as conduction channels in scars. Data from seventeen VT patients were also used to study the feasibility of the QCM technique in a clinical setting, consistently outperforming the alternative integration techniques in the presence of sparse and noisy clinical data. The proposed evaluation framework has allowed a rigorous comparison of different EAM and imaging data integration strategies, providing useful information to better guide clinical practice in

Objective evaluation of the visual acuity in human eyes

NASA Astrophysics Data System (ADS)

Rosales, M. A.; López-Olazagasti, E.; Ramírez-Zavaleta, G.; Varillas, G.; Tepichín, E.

2009-08-01

Traditionally, the quality of the human vision is evaluated by a subjective test in which the examiner asks the patient to read a series of characters of different sizes, located at a certain distance of the patient. Typically, we need to ensure a subtended angle of vision of 5 minutes, which implies an object of 8.8 mm high located at 6 meters (normal or 20/20 visual acuity). These characters constitute what is known as the Snellen chart, universally used to evaluate the spatial resolution of the human eyes. The mentioned process of identification of characters is carried out by means of the eye - brain system, giving an evaluation of the subjective visual performance. In this work we consider the eye as an isolated image-forming system, and show that it is possible to isolate the function of the eye from that of the brain in this process. By knowing the impulse response of the eye´s system we can obtain, in advance, the image of the Snellen chart simultaneously. From this information, we obtain the objective performance of the eye as the optical system under test. This type of results might help to detect anomalous situations of the human vision, like the so called "cerebral myopia".

Assessing Students' Understandings of Biological Models and Their Use in Science to Evaluate a Theoretical Framework

ERIC Educational Resources Information Center

Grünkorn, Juliane; Upmeier zu Belzen, Annette; Krüger, Dirk

2014-01-01

Research in the field of students' understandings of models and their use in science describes different frameworks concerning these understandings. Currently, there is no conjoint framework that combines these structures and so far, no investigation has focused on whether it reflects students' understandings sufficiently (empirical evaluation).…

Eliciting Values of Patients with Multiple Chronic Conditions: Evaluation of a Patient-centered Framework

PubMed Central

Berry, Andrew B.L.; Lim, Catherine; Hartzler, Andrea L.; Hirsch, Tad; Ludman, Evette; Wagner, Edward H.; Ralston, James D.

2017-01-01

Patients with multiple chronic conditions often face competing demands for care, and they often do not agree with physicians on priorities for care. Patients ’ values shape their healthcare priorities, but existing methods for eliciting values do not necessarily meet patients ’ care planning needs. We developed a patient-centered values framework based on a field study with patients and caregivers. In this paper we report on a survey to evaluate how the framework generalizes beyond field study participants, and how well the framework supports values elicitation. We found that respondents frame values in a way that is consistent with the framework, and that domains of the framework can be used to elicit a breadth of potential values individuals with MCC express. These findings demonstrate how a patient-centered perspective on values can expand on the domains considered in values clarification methods andfacilitate patient-provider communication in establishing shared care priorities. PMID:29854107

R-IDEAL: A Framework for Systematic Clinical Evaluation of Technical Innovations in Radiation Oncology.

PubMed

Verkooijen, Helena M; Kerkmeijer, Linda G W; Fuller, Clifton D; Huddart, Robbert; Faivre-Finn, Corinne; Verheij, Marcel; Mook, Stella; Sahgal, Arjun; Hall, Emma; Schultz, Chris

2017-01-01

The pace of innovation in radiation oncology is high and the window of opportunity for evaluation narrow. Financial incentives, industry pressure, and patients' demand for high-tech treatments have led to widespread implementation of innovations before, or even without, robust evidence of improved outcomes has been generated. The standard phase I-IV framework for drug evaluation is not the most efficient and desirable framework for assessment of technological innovations. In order to provide a standard assessment methodology for clinical evaluation of innovations in radiotherapy, we adapted the surgical IDEAL framework to fit the radiation oncology setting. Like surgery, clinical evaluation of innovations in radiation oncology is complicated by continuous technical development, team and operator dependence, and differences in quality control. Contrary to surgery, radiotherapy innovations may be used in various ways, e.g., at different tumor sites and with different aims, such as radiation volume reduction and dose escalation. Also, the effect of radiation treatment can be modeled, allowing better prediction of potential benefits and improved patient selection. Key distinctive features of R-IDEAL include the important role of predicate and modeling studies (Stage 0), randomization at an early stage in the development of the technology, and long-term follow-up for late toxicity. We implemented R-IDEAL for clinical evaluation of a recent innovation in radiation oncology, the MRI-guided linear accelerator (MR-Linac). MR-Linac combines a radiotherapy linear accelerator with a 1.5-T MRI, aiming for improved targeting, dose escalation, and margin reduction, and is expected to increase the use of hypofractionation, improve tumor control, leading to higher cure rates and less toxicity. An international consortium, with participants from seven large cancer institutes from Europe and North America, has adopted the R-IDEAL framework to work toward coordinated, evidence

Advancing a Framework to Enable Characterization and Evaluation of Data Streams Useful for Biosurveillance

PubMed Central

Margevicius, Kristen J.; Generous, Nicholas; Taylor-McCabe, Kirsten J.; Brown, Mac; Daniel, W. Brent; Castro, Lauren; Hengartner, Andrea; Deshpande, Alina

2014-01-01

In recent years, biosurveillance has become the buzzword under which a diverse set of ideas and activities regarding detecting and mitigating biological threats are incorporated depending on context and perspective. Increasingly, biosurveillance practice has become global and interdisciplinary, requiring information and resources across public health, One Health, and biothreat domains. Even within the scope of infectious disease surveillance, multiple systems, data sources, and tools are used with varying and often unknown effectiveness. Evaluating the impact and utility of state-of-the-art biosurveillance is, in part, confounded by the complexity of the systems and the information derived from them. We present a novel approach conceptualizing biosurveillance from the perspective of the fundamental data streams that have been or could be used for biosurveillance and to systematically structure a framework that can be universally applicable for use in evaluating and understanding a wide range of biosurveillance activities. Moreover, the Biosurveillance Data Stream Framework and associated definitions are proposed as a starting point to facilitate the development of a standardized lexicon for biosurveillance and characterization of currently used and newly emerging data streams. Criteria for building the data stream framework were developed from an examination of the literature, analysis of information on operational infectious disease biosurveillance systems, and consultation with experts in the area of biosurveillance. To demonstrate utility, the framework and definitions were used as the basis for a schema of a relational database for biosurveillance resources and in the development and use of a decision support tool for data stream evaluation. PMID:24392093

Expert reasoning within an object-oriented framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bohn, S.J.; Pennock, K.A.

1991-10-01

A large number of contaminated waste sites across the United States await site remediation efforts. These sites can be physically complex, composed of multiple, possibly interacting, contaminants distributed throughout one or more media. The Remedial Action Assessment System (RAAS) is being designed and developed to support decisions concerning the selection of remediation alternatives. The goal of this system is to broaden the consideration of remediation alternatives, while reducing the time and cost of making these considerations. The Remedial Action Assessment System was designed and constructed using object-oriented techniques. It is a hybrid system which uses a combination of quantitative andmore » qualitative reasoning to consider and suggest remediation alternatives. the reasoning process that drives this application is centered around an object-oriented organization of remediation technology information. This paper briefly describes the waste remediation problem and then discusses the information structure and organization RAAS utilizes to address it. 4 refs., 4 figs.« less

A framework to observe and evaluate the sustainability of human-natural systems in a complex dynamic context.

PubMed

Satanarachchi, Niranji; Mino, Takashi

2014-01-01

This paper aims to explore the prominent implications of the process of observing complex dynamics linked to sustainability in human-natural systems and to propose a framework for sustainability evaluation by introducing the concept of sustainability boundaries. Arguing that both observing and evaluating sustainability should engage awareness of complex dynamics from the outset, we try to embody this idea in the framework by two complementary methods, namely, the layer view- and dimensional view-based methods, which support the understanding of a reflexive and iterative sustainability process. The framework enables the observation of complex dynamic sustainability contexts, which we call observation metastructures, and enable us to map the contexts to sustainability boundaries.

Patterns for Effectively Documenting Frameworks

NASA Astrophysics Data System (ADS)

Aguiar, Ademar; David, Gabriel

Good design and implementation are necessary but not sufficient pre-requisites for successfully reusing object-oriented frameworks. Although not always recognized, good documentation is crucial for effective framework reuse, and often hard, costly, and tiresome, coming with many issues, especially when we are not aware of the key problems and respective ways of addressing them. Based on existing literature, case studies and lessons learned, the authors have been mining proven solutions to recurrent problems of documenting object-oriented frameworks, and writing them in pattern form, as patterns are a very effective way of communicating expertise and best practices. This paper presents a small set of patterns addressing problems related to the framework documentation itself, here seen as an autonomous and tangible product independent of the process used to create it. The patterns aim at helping non-experts on cost-effectively documenting object-oriented frameworks. In concrete, these patterns provide guidance on choosing the kinds of documents to produce, how to relate them, and which contents to include. Although the focus is more on the documents themselves, rather than on the process and tools to produce them, some guidelines are also presented in the paper to help on applying the patterns to a specific framework.

Building a framework for ergonomic research on laparoscopic instrument handles.

PubMed

Li, Zheng; Wang, Guohui; Tan, Juan; Sun, Xulong; Lin, Hao; Zhu, Shaihong

2016-06-01

Laparoscopic surgery carries the advantage of minimal invasiveness, but ergonomic design of the instruments used has progressed slowly. Previous studies have demonstrated that the handle of laparoscopic instruments is vital for both surgical performance and surgeon's health. This review provides an overview of the sub-discipline of handle ergonomics, including an evaluation framework, objective and subjective assessment systems, data collection and statistical analyses. Furthermore, a framework for ergonomic research on laparoscopic instrument handles is proposed to standardize work on instrument design. Copyright © 2016 IJS Publishing Group Ltd. Published by Elsevier Ltd. All rights reserved.

An Evaluation of the Generalized Intelligent Framework for Tutoring (GIFT) from a Researcher’s or Analyst’s Perspective

DTIC Science & Technology

2014-12-01

An Evaluation of the Generalized Intelligent Framework for Tutoring (GIFT) from a Researcher’s or Analyst’s Perspective by Robert A...Generalized Intelligent Framework for Tutoring (GIFT) from a Researcher’s or Analyst’s Perspective Robert A Sottilare and Anne M Sinatra Human...2014 4. TITLE AND SUBTITLE An Evaluation of the Generalized Intelligent Framework for Tutoring (GIFT) from a Researcher’s or Analyst’s Perspective

The Legacy Ecosystem Management Framework: From Theory to Application in the Detention Pond Case Study

DOE Office of Scientific and Technical Information (OSTI.GOV)

Coty, J; Stevenson, M; Vogt, K A

The Detention Pond is a constructed and lined storm water treatment basin at Lawrence Livermore National Laboratory that serves multiple stakeholder objectives and programmatic goals. This paper examines the process and outcome involved in the development of a new management plan for the Detention Pond. The plan was created using a new ecosystem management tool, the Legacy Framework. This stakeholder-driven conceptual framework provides an interdisciplinary methodology for determining ecosystem health, appropriate management strategies, and sensitive indicators. The conceptual framework, the Detention Ponds project, and the use of the framework in the context of the project, are described and evaluated, andmore » evaluative criteria for this and other ecosystem management frameworks are offered. The project benefited in several ways from use of the Legacy Framework, although refinements to the framework are suggested. The stakeholder process created a context and environment in which team members became receptive to using an ecosystem management approach to evaluate and support management alternatives previously not considered. This allowed for the unanimous agreement to pursue support from upper management and organizational funding to implement a progressive management strategy. The greatly improved stakeholder relations resulted in upper management support for the project.« less

A Framework of Multi Objectives Negotiation for Dynamic Supply Chain Model

NASA Astrophysics Data System (ADS)

Chai, Jia Yee; Sakaguchi, Tatsuhiko; Shirase, Keiichi

Trends of globalization and advances in Information Technology (IT) have created opportunity in collaborative manufacturing across national borders. A dynamic supply chain utilizes these advances to enable more flexibility in business cooperation. This research proposes a concurrent decision making framework for a three echelons dynamic supply chain model. The dynamic supply chain is formed by autonomous negotiation among agents based on multi agents approach. Instead of generating negotiation aspects (such as amount, price and due date) arbitrary, this framework proposes to utilize the information available at operational level of an organization in order to generate realistic negotiation aspect. The effectiveness of the proposed model is demonstrated by various case studies.

The scholar role in the National Competence Based Catalogues of Learning Objectives for Undergraduate Medical Education (NKLM) compared to other international frameworks.

PubMed

Hautz, Stefanie C; Hautz, Wolf E; Keller, Niklas; Feufel, Markus A; Spies, Claudia

2015-01-01

In Germany, a national competence based catalogue of learning objectives in medicine (NKLM) was developed by the Society for Medical Education and the Council of Medical Faculties. As many of its international counterparts the NKLM describes the qualifications of medical school graduates. The definition of such outcome frameworks indents to make medical education transparent to students, teachers and society. The NKLM aims to amend existing lists of medical topics for assessment with learnable competencies. All outcome frameworks are structured into chapters, domains or physician roles. The definition of the scholar-role poses a number of questions such as: What distinguishes necessary qualifications of a scientifically qualified physician from those of a medical scientist? 13 outcome frameworks were identified through a systematic three-step literature review and their content compared to the scholar role in the NKLM by means of a qualitative text analysis. The three steps consist of (1) search for outcome frameworks, (2) in- and exclusion, and (3) data extraction, categorization, and validation. The results were afterwards matched with the scholar role of the NKLM. Extracted contents of all frameworks may be summarized into the components Common Basics, Clinical Application, Research, Teaching and Education, and Lifelong Learning. Compared to the included frameworks the NKLM emphasises competencies necessary for research and teaching while clinical application is less prominently mentioned. The scholar role of the NKLM differs from other international outcome frameworks. Discussing these results shall increase propagation and understanding of the NKLM and thus contribute to the qualification of future medical graduates in Germany.

Methodology for Evaluating Quality and Reusability of Learning Objects

ERIC Educational Resources Information Center

Kurilovas, Eugenijus; Bireniene, Virginija; Serikoviene, Silvija

2011-01-01

The aim of the paper is to present the scientific model and several methods for the expert evaluation of quality of learning objects (LOs) paying especial attention to LOs reusability level. The activities of eQNet Quality Network for a European Learning Resource Exchange (LRE) aimed to improve reusability of LOs of European Schoolnet's LRE…

Providing a Science Base for the Evaluation of Tobacco Products

PubMed Central

Berman, Micah L.; Connolly, Greg; Cummings, K. Michael; Djordjevic, Mirjana V.; Hatsukami, Dorothy K.; Henningfield, Jack E.; Myers, Matthew; O'Connor, Richard J.; Parascandola, Mark; Rees, Vaughan; Rice, Jerry M.

2015-01-01

Objective Evidence-based tobacco regulation requires a comprehensive scientific framework to guide the evaluation of new tobacco products and health-related claims made by product manufacturers. Methods The Tobacco Product Assessment Consortium (TobPRAC) employed an iterative process involving consortia investigators, consultants, a workshop of independent scientists and public health experts, and written reviews in order to develop a conceptual framework for evaluating tobacco products. Results The consortium developed a four-phased framework for the scientific evaluation of tobacco products. The four phases addressed by the framework are: (1) pre-market evaluation, (2) pre-claims evaluation, (3) post-market activities, and (4) monitoring and re-evaluation. For each phase, the framework proposes the use of validated testing procedures that will evaluate potential harms at both the individual and population level. Conclusions While the validation of methods for evaluating tobacco products is an ongoing and necessary process, the proposed framework need not wait for fully validated methods to be used in guiding tobacco product regulation today. PMID:26665160

An Evaluation Model for a Multidisciplinary Chronic Pelvic Pain Clinic: Application of the RE-AIM Framework.

PubMed

Chen, Innie; Money, Deborah; Yong, Paul; Williams, Christina; Allaire, Catherine

2015-09-01

Chronic pelvic pain (CPP) is a prevalent, debilitating, and costly condition. Although national guidelines and empiric evidence support the use of a multidisciplinary model of care for such patients, such clinics are uncommon in Canada. The BC Women's Centre for Pelvic Pain and Endometriosis was created to respond to this need, and there is interest in this model of care's impact on the burden of disease in British Columbia. We sought to create an approach to its evaluation using the RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) evaluation framework to assess the impact of the care model and to guide clinical decision-making and policy. The RE-AIM evaluation framework was applied to consider the different dimensions of impact of the BC Centre. The proposed measures, data sources, and data management strategies for this mixed-methods approach were identified. The five dimensions of impact were considered at individual and organizational levels, and corresponding indicators were proposed to enable integration into existing data infrastructure to facilitate collection and early program evaluation. The RE-AIM framework can be applied to the evaluation of a multidisciplinary chronic pelvic pain clinic. This will allow better assessment of the impact of innovative models of care for women with chronic pelvic pain.

a Novel Framework for Remote Sensing Image Scene Classification

NASA Astrophysics Data System (ADS)

Jiang, S.; Zhao, H.; Wu, W.; Tan, Q.

2018-04-01

High resolution remote sensing (HRRS) images scene classification aims to label an image with a specific semantic category. HRRS images contain more details of the ground objects and their spatial distribution patterns than low spatial resolution images. Scene classification can bridge the gap between low-level features and high-level semantics. It can be applied in urban planning, target detection and other fields. This paper proposes a novel framework for HRRS images scene classification. This framework combines the convolutional neural network (CNN) and XGBoost, which utilizes CNN as feature extractor and XGBoost as a classifier. Then, this framework is evaluated on two different HRRS images datasets: UC-Merced dataset and NWPU-RESISC45 dataset. Our framework achieved satisfying accuracies on two datasets, which is 95.57 % and 83.35 % respectively. From the experiments result, our framework has been proven to be effective for remote sensing images classification. Furthermore, we believe this framework will be more practical for further HRRS scene classification, since it costs less time on training stage.

Development of hospital disaster resilience: conceptual framework and potential measurement.

PubMed

Zhong, Shuang; Clark, Michele; Hou, Xiang-Yu; Zang, Yu-Li; Fitzgerald, Gerard

2014-11-01

Despite 'hospital resilience' gaining prominence in recent years, it remains poorly defined. This article aims to define hospital resilience, build a preliminary conceptual framework and highlight possible approaches to measurement. Searches were conducted of the commonly used health databases to identify relevant literature and reports. Search terms included 'resilience and framework or model' or 'evaluation or assess or measure and hospital and disaster or emergency or mass casualty and resilience or capacity or preparedness or response or safety'. Articles were retrieved that focussed on disaster resilience frameworks and the evaluation of various hospital capacities. A total of 1480 potentially eligible publications were retrieved initially but the final analysis was conducted on 47 articles, which appeared to contribute to the study objectives. Four disaster resilience frameworks and 11 evaluation instruments of hospital disaster capacity were included. Hospital resilience is a comprehensive concept derived from existing disaster resilience frameworks. It has four key domains: hospital safety; disaster preparedness and resources; continuity of essential medical services; recovery and adaptation. These domains were categorised according to four criteria, namely, robustness, redundancy, resourcefulness and rapidity. A conceptual understanding of hospital resilience is essential for an intellectual basis for an integrated approach to system development. This article (1) defines hospital resilience; (2) constructs conceptual framework (including key domains); (3) proposes comprehensive measures for possible inclusion in an evaluation instrument; and (4) develops a matrix of critical issues to enhance hospital resilience to cope with future disasters. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Educational Evaluation: The State of the Field.

ERIC Educational Resources Information Center

Wolf, Richard M., Ed.

1987-01-01

Educational evaluation is discussed. Topics include: an evaluation framework, educational objectives and study design from a 20-year perspective, a sample study, educational evaluation for local school improvement, decision-oriented evaluation studies, reporting study results, and professional standards for assuring the quality of educational…

Evaluating and Improving Cloud Processes in the Multi-Scale Modeling Framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ackerman, Thomas P.

2015-03-01

The research performed under this grant was intended to improve the embedded cloud model in the Multi-scale Modeling Framework (MMF) for convective clouds by using a 2-moment microphysics scheme rather than the single moment scheme used in all the MMF runs to date. The technical report and associated documents describe the results of testing the cloud resolving model with fixed boundary conditions and evaluation of model results with data. The overarching conclusion is that such model evaluations are problematic because errors in the forcing fields control the results so strongly that variations in parameterization values cannot be usefully constrained

Quality Attribute Techniques Framework

NASA Astrophysics Data System (ADS)

Chiam, Yin Kia; Zhu, Liming; Staples, Mark

The quality of software is achieved during its development. Development teams use various techniques to investigate, evaluate and control potential quality problems in their systems. These “Quality Attribute Techniques” target specific product qualities such as safety or security. This paper proposes a framework to capture important characteristics of these techniques. The framework is intended to support process tailoring, by facilitating the selection of techniques for inclusion into process models that target specific product qualities. We use risk management as a theory to accommodate techniques for many product qualities and lifecycle phases. Safety techniques have motivated the framework, and safety and performance techniques have been used to evaluate the framework. The evaluation demonstrates the ability of quality risk management to cover the development lifecycle and to accommodate two different product qualities. We identify advantages and limitations of the framework, and discuss future research on the framework.

Developing a cultural context for conducting a neuropsychological evaluation with a culturally diverse client: the ECLECTIC framework.

PubMed

Fujii, Daryl E M

2018-02-20

With the increasing diversification of the American population, the discipline of neuropsychology is challenged to develop appropriate tools and conceptual models to meet its evolving client base. Thus far, the focus has been on developing appropriate tests and norms to obtain accurate testing data. By contrast, far less attention has been paid to the contextual impact of culture on an evaluation. This paper attempts to address this shortcoming. This manuscript introduces the ECLECTIC framework for conceptualizing different facets of culture pertinent for understanding a culturally diverse client when conducting a neuropsychological evaluation. Individual components of the framework (E: education and literacy; C: culture and acculturation; L: language; E: economics; C: communication; T: testing situation: comfort and motivation; I: intelligence conceptualization; and C: context of immigration) are introduced and potential biases to fairness in testing are described. In this manner, the framework specifies how individual facets of culture can impact neuropsychological test performance. Clinical implementation of the framework will be illustrated with a case sample. Strengths and weaknesses of the framework are discussed as well as recommendations for implementation.

A modeling framework for evaluating streambank stabilization practices for reach-scale sediment reduction

USDA-ARS?s Scientific Manuscript database

Streambank stabilization techniques are often implemented to reduce sediment loads from unstable streambanks. Process-based models can predict sediment yields with stabilization scenarios prior to implementation. However, a framework does not exist on how to effectively utilize these models to evalu...

Objectivity and reliability in qualitative analysis: realist, contextualist and radical constructionist epistemologies.

PubMed

Madill, A; Jordan, A; Shirley, C

2000-02-01

The effect of the individual analyst on research findings can create a credibility problem for qualitative approaches from the perspective of evaluative criteria utilized in quantitative psychology. This paper explicates the ways in which objectivity and reliability are understood in qualitative analysis conducted from within three distinct epistemological frameworks: realism, contextual constructionism, and radical constructionism. It is argued that quality criteria utilized in quantitative psychology are appropriate to the evaluation of qualitative analysis only to the extent that it is conducted within a naive or scientific realist framework. The discussion is illustrated with reference to the comparison of two independent grounded theory analyses of identical material. An implication of this illustration is to identify the potential to develop a radical constructionist strand of grounded theory.

Objective Evaluation of Visual Fatigue Using Binocular Fusion Maintenance.

PubMed

Hirota, Masakazu; Morimoto, Takeshi; Kanda, Hiroyuki; Endo, Takao; Miyoshi, Tomomitsu; Miyagawa, Suguru; Hirohara, Yoko; Yamaguchi, Tatsuo; Saika, Makoto; Fujikado, Takashi

2018-03-01

In this study, we investigated whether an individual's visual fatigue can be evaluated objectively and quantitatively from their ability to maintain binocular fusion. Binocular fusion maintenance (BFM) was measured using a custom-made binocular open-view Shack-Hartmann wavefront aberrometer equipped with liquid crystal shutters, wherein eye movements and wavefront aberrations were measured simultaneously. Transmittance in the liquid crystal shutter in front of the subject's nondominant eye was reduced linearly, and BFM was determined from the transmittance at the point when binocular fusion was broken and vergence eye movement was induced. In total, 40 healthy subjects underwent the BFM test and completed a questionnaire regarding subjective symptoms before and after a visual task lasting 30 minutes. BFM was significantly reduced after the visual task ( P < 0.001) and was negatively correlated with the total subjective eye symptom score (adjusted R 2 = 0.752, P < 0.001). Furthermore, the diagnostic accuracy for visual fatigue was significantly higher in BFM than in the conventional test results (aggregated fusional vergence range, near point of convergence, and the high-frequency component of accommodative microfluctuations; P = 0.007). These results suggest that BFM can be used as an indicator for evaluating visual fatigue. BFM can be used to evaluate the visual fatigue caused by the new visual devices, such as head-mount display, objectively.

Objective Evaluation of Visual Fatigue Using Binocular Fusion Maintenance

PubMed Central

Hirota, Masakazu; Morimoto, Takeshi; Kanda, Hiroyuki; Endo, Takao; Miyoshi, Tomomitsu; Miyagawa, Suguru; Hirohara, Yoko; Yamaguchi, Tatsuo; Saika, Makoto

2018-01-01

Purpose In this study, we investigated whether an individual's visual fatigue can be evaluated objectively and quantitatively from their ability to maintain binocular fusion. Methods Binocular fusion maintenance (BFM) was measured using a custom-made binocular open-view Shack–Hartmann wavefront aberrometer equipped with liquid crystal shutters, wherein eye movements and wavefront aberrations were measured simultaneously. Transmittance in the liquid crystal shutter in front of the subject's nondominant eye was reduced linearly, and BFM was determined from the transmittance at the point when binocular fusion was broken and vergence eye movement was induced. In total, 40 healthy subjects underwent the BFM test and completed a questionnaire regarding subjective symptoms before and after a visual task lasting 30 minutes. Results BFM was significantly reduced after the visual task (P < 0.001) and was negatively correlated with the total subjective eye symptom score (adjusted R2 = 0.752, P < 0.001). Furthermore, the diagnostic accuracy for visual fatigue was significantly higher in BFM than in the conventional test results (aggregated fusional vergence range, near point of convergence, and the high-frequency component of accommodative microfluctuations; P = 0.007). Conclusions These results suggest that BFM can be used as an indicator for evaluating visual fatigue. Translational Relevance BFM can be used to evaluate the visual fatigue caused by the new visual devices, such as head-mount display, objectively. PMID:29600117

A framework for the monitoring and evaluation of international surgical initiatives in low- and middle-income countries.

PubMed

Ibrahim, George M; Cadotte, David W; Bernstein, Mark

2015-01-01

An estimated two billion people worldwide lack adequate access to surgical care. To address this humanitarian emergency, an increasing number of international surgical partnerships are emerging between developed and low- and middle-income countries (LMICs). At present, there are no clear indicators that may be used to assess the effectiveness of such initiatives. We conducted an international qualitative study of 31 surgeons from developed and LMICs involved in international partnerships across a variety of subspecialties. Thematic analysis and grounded theory were applied in order to develop a practical framework that may be applied to monitor and evaluate global surgical initiatives. Several themes emerged from the study: (i) there is a large unmet need to establish and maintain prospective databases in LMICs to inform the monitoring and evaluation of international surgical partnerships; (ii) assessment of initiatives must occur longitudinally over the span of several years; (ii) the domains of assessment are contextual and encompass cultural, institutional and regional factors; and (iv) evaluation strategies should explore broader impact within the community and country. Based on thematic analysis within the domains of inputs, outputs and outcomes, a framework for the monitoring and evaluation of international surgical initiatives, the Framework for the Assessment of InteRNational Surgical Success (FAIRNeSS) is proposed. In response to the increasing number of surgical partnerships between developed and LMICs, we propose a framework to monitor and evaluate international surgical initiatives.

Program Fair Evaluation--Summative Appraisal of Instructional Sequences with Dissimilar Objectives.

ERIC Educational Resources Information Center

Popham, W. James

A comparative evaluation involving two instructional programs is given, although the approach can easily serve to compare more than two programs. The steps involved in conducting a program fair evaluation of two instructional programs are: (1) Identify objectives (a) common to both programs, (b) unique to one program, and (c) unique to the other…

Evaluating a Blended Degree Program through the Use of the NSSE Framework

ERIC Educational Resources Information Center

Vaughan, Norman; Cloutier, David

2017-01-01

The purpose of this student-faculty partnership research study was to evaluate the effectiveness of a blended four-year Bachelor of Education Elementary Program at a Canadian university using the National Survey of Student Engagement (NSSE) framework. Data was collected from the first graduating cohort of students from the B.Ed. program in…

Assessment of Evaluation Frameworks for Design of a Sexual Risk Prevention Game for Black Adolescent Girls.

PubMed

Sockolow, Paulina; Joppa, Meredith; Zhu, Jichen

2018-01-01

Adolescent sexual risk behavior (SRB), a major public health problem affects urban Black adolescent girls increasing their health disparities and risks for sexually transmitted infections. Collaborating with these adolescents, we designed a game for smartphones that incorporates elements of trauma-informed care and social cognitive theory to reduce SRB. Game researchers promote use of a comprehensive, multipurpose framework for development and evaluation of games for health applications. Our first game development step was framework selection and measurable health outcomes identification. Literature search identified two health game frameworks, both incorporating pedagogical theory, learning theory, and gaming requirements. Arnab used the IM + LM-GM framework to develop and implement a game in a school intervention program. Yusoff's framework was developed for use during game design. We investigated concordance and discordance between our SRB game design characteristics and each framework's components. Findings indicated Arnab's framework was sufficiently comprehensive to guide development of our game and outcome measure selection.

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework.

PubMed

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process.

Using the Theory of Planned Behavior as a Framework for the Evaluation of a Professional Development Workshop

PubMed Central

PATTERSON, ROBIN R.

2001-01-01

This purpose of this study was to use a theoretical framework based on several decades of attitudinal research to assess the intentions of Microbial Discovery Workshop participants to incorporate the inquiry activities presented at the workshop into their curricula, to evaluate the participants actual use of these activities after the workshop, and to uncover the barriers and enablers the participants faced in doing so. As a framework, the theory of planned behavior was ascertained to be an appropriate means of assessment and it was revealed that participants’ intention to use the workshop activities significantly correlated with their actual use. The participants’ attitudes toward using the activities influenced their use more than the participants’ perceptions of the social pressures that would influence their decision to use the activities or their belief as to how easy or difficult it would be to incorporate a given activity. The participants were found to be highly self-efficacious pertaining to their ability to implement the activities, but perceived self-efficacy was not a significant predictor of the participants’ intentions to incorporate the activities into their teaching-learning repertoire. The study also uncovered other behaviors the participants displayed as a result of attending the workshop consistent with the goals and objectives of the workshop organizers. PMID:23653542

SEE Action Guide for States: Evaluation, Measurement, and Verification Frameworks$-$Guidance for Energy Efficiency Portfolios Funded by Utility Customers

DOE Office of Scientific and Technical Information (OSTI.GOV)

Li, Michael; Dietsch, Niko

2018-01-01

This guide describes frameworks for evaluation, measurement, and verification (EM&V) of utility customer–funded energy efficiency programs. The authors reviewed multiple frameworks across the United States and gathered input from experts to prepare this guide. This guide provides the reader with both the contents of an EM&V framework, along with the processes used to develop and update these frameworks.

An open source framework for tracking and state estimation ('Stone Soup')

NASA Astrophysics Data System (ADS)

Thomas, Paul A.; Barr, Jordi; Balaji, Bhashyam; White, Kruger

2017-05-01

The ability to detect and unambiguously follow all moving entities in a state-space is important in multiple domains both in defence (e.g. air surveillance, maritime situational awareness, ground moving target indication) and the civil sphere (e.g. astronomy, biology, epidemiology, dispersion modelling). However, tracking and state estimation researchers and practitioners have difficulties recreating state-of-the-art algorithms in order to benchmark their own work. Furthermore, system developers need to assess which algorithms meet operational requirements objectively and exhaustively rather than intuitively or driven by personal favourites. We have therefore commenced the development of a collaborative initiative to create an open source framework for production, demonstration and evaluation of Tracking and State Estimation algorithms. The initiative will develop a (MIT-licensed) software platform for researchers and practitioners to test, verify and benchmark a variety of multi-sensor and multi-object state estimation algorithms. The initiative is supported by four defence laboratories, who will contribute to the development effort for the framework. The tracking and state estimation community will derive significant benefits from this work, including: access to repositories of verified and validated tracking and state estimation algorithms, a framework for the evaluation of multiple algorithms, standardisation of interfaces and access to challenging data sets. Keywords: Tracking,

Objective Structured Professional Assessments for Trainee Educational Psychologists: An Evaluation

ERIC Educational Resources Information Center

Dunsmuir, Sandra; Atkinson, Cathy; Lang, Jane; Warhurst, Amy; Wright, Sarah

2017-01-01

Objective Structured Professional Assessments (OSPAs) were developed and evaluated at three universities in the United Kingdom, to supplement supervisor assessments of trainee educational psychologists' placement practice. Participating second year students on three educational psychology doctoral programmes (n = 31) and tutors (n = 12) were…

A Concise and Practical Framework for the Development and Usability Evaluation of Patient Information Websites.

PubMed

Peute, L W; Knijnenburg, S L; Kremer, L C; Jaspers, M W M

2015-01-01

The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS). To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population. The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA. The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content. Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations.

A psychometric evaluation of an advanced pharmacy practice experience clinical competency framework.

PubMed

Douglas Ried, L; Doty, Randell E; Nemire, Ruth E

2015-03-25

To assess the psychometric properties of the clinical competency framework known as the System of Universal Clinical Competency Evaluation in the Sunshine State (SUCCESS), including its internal consistency and content, construct, and criterion validity. Sub-competency items within each hypothesized competency pair were subjected to principal components factor analysis to demonstrate convergent and discriminant validity. Varimax rotation was conducted for each competency pair (eg, competency 1 vs competency 2, competency 1 vs competency 3, competency 2 vs competency 3). Internal consistency was evaluated using Cronbach alpha. Of the initial 78 pairings, 44 (56%) demonstrated convergent and discriminant validity. Five pairs of competencies were unidimensional. Of the 34 pairs where at least 1 competency was multidimensional, most (91%) were from competencies 7, 11, and 12, indicating modifications were warranted in those competencies. After reconfiguring the competencies, 76 (94%) of the 81 pairs resulted in 2 factors as required. A unidimensional factor emerged when all 13 of the competencies were entered into a factor analysis. The internal consistency of all of the competencies was satisfactory. Psychometric evaluation shows the SUCCESS framework demonstrates adequate reliability and validity for most competencies. However, it also provides guidance where improvements are needed as part of a continuous quality improvement program.

Direct evaluation of fault trees using object-oriented programming techniques

NASA Technical Reports Server (NTRS)

Patterson-Hine, F. A.; Koen, B. V.

1989-01-01

Object-oriented programming techniques are used in an algorithm for the direct evaluation of fault trees. The algorithm combines a simple bottom-up procedure for trees without repeated events with a top-down recursive procedure for trees with repeated events. The object-oriented approach results in a dynamic modularization of the tree at each step in the reduction process. The algorithm reduces the number of recursive calls required to solve trees with repeated events and calculates intermediate results as well as the solution of the top event. The intermediate results can be reused if part of the tree is modified. An example is presented in which the results of the algorithm implemented with conventional techniques are compared to those of the object-oriented approach.

Method and Apparatus for Evaluating Multilayer Objects for Imperfections

NASA Technical Reports Server (NTRS)

Heyman, Joseph S. (Inventor); Abedin, Nurul (Inventor); Sun, Kuen J. (Inventor)

1999-01-01

A multilayer object having multiple layers arranged in a stacking direction is evaluated for imperfections such as voids, delaminations and microcracks. First. an acoustic wave is transmitted into the object in the stacking direction via an appropriate transducer/waveguide combination. The wave propagates through the multilayer object and is received by another transducer/waveguide combination preferably located on the same surface as the transmitting combination. The received acoustic wave is correlated with the presence or absence of imperfections by, e.g., generating pulse echo signals indicative of the received acoustic wave. wherein the successive signals form distinct groups over time. The respective peak amplitudes of each group are sampled and curve fit to an exponential curve. wherein a substantial fit of approximately 80-90% indicates an absence of imperfections and a significant deviation indicates the presence of imperfections. Alternatively, the time interval between distinct groups can be measured. wherein equal intervals indicate the absence of imperfections and unequal intervals indicate the presence of imperfections.

Method and apparatus for evaluating multilayer objects for imperfections

NASA Technical Reports Server (NTRS)

Heyman, Joseph S. (Inventor); Abedin, Nurul (Inventor); Sun, Kuen J. (Inventor)

1997-01-01

A multilayer object having multiple layers arranged in a stacking direction is evaluated for imperfections such as voids, delaminations and microcracks. First, an acoustic wave is transmitted into the object in the stacking direction via an appropriate transducer/waveguide combination. The wave propagates through the multilayer object and is received by another transducer/waveguide combination preferably located on the same surface as the transmitting combination. The received acoustic wave is correlated with the presence or absence of imperfections by, e.g., generating pulse echo signals indicative of the received acoustic wave, wherein the successive signals form distinct groups over time. The respective peak amplitudes of each group are sampled and curve fit to an exponential curve, wherein a substantial fit of approximately 80-90% indicates an absence of imperfections and a significant deviation indicates the presence of imperfections. Alternatively, the time interval between distinct groups can be measured, wherein equal intervals indicate the absence of imperfections and unequal intervals indicate the presence of imperfections.

A Decision Support Framework for Evaluation of Engineered ...

EPA Pesticide Factsheets

Engineered nanomaterials (ENM) are currently being developed and applied at rates that far exceed our ability to evaluate their potential for environmental or human health risks. The gap between material development and capacity for assessment grows wider every day. Transformative approaches are required that enhance our ability to forecast potential exposure and adverse health risks based on limited information such as the physical and chemical parameters of ENM, their proposed uses, and functional assays reflective of key ENM - environmental interactions. We are developing a framework that encompasses the potential for release of nanomaterials across a product life cycle, environmental transport, transformations and fate, exposure to sensitive species, including humans, and the potential for causing adverse effects. Each component of the framework is conceive of as a sequential segmented model depicting the movement, transformations and actions of ENM through environmental or biological compartments, and along which targeted functional assays can be developed that are indicative of projected rates of ENM movement or action. The eventual goal is to allow simple predictive models to be built that incorporate the data from key functional assays and thereby allow rapid screening of the projected margin of exposure for proposed applications of ENM enabled products. In this way, cases where a substantially safe margin of exposure is forecast can be reduced in

Developing a clinical utility framework to evaluate prediction models in radiogenomics

NASA Astrophysics Data System (ADS)

Wu, Yirong; Liu, Jie; Munoz del Rio, Alejandro; Page, David C.; Alagoz, Oguzhan; Peissig, Peggy; Onitilo, Adedayo A.; Burnside, Elizabeth S.

2015-03-01