Research on efficiency evaluation model of integrated energy system based on hybrid multi-attribute decision-making.

PubMed

Li, Yan

2017-05-25

The efficiency evaluation model of integrated energy system, involving many influencing factors, and the attribute values are heterogeneous and non-deterministic, usually cannot give specific numerical or accurate probability distribution characteristics, making the final evaluation result deviation. According to the characteristics of the integrated energy system, a hybrid multi-attribute decision-making model is constructed. The evaluation model considers the decision maker's risk preference. In the evaluation of the efficiency of the integrated energy system, the evaluation value of some evaluation indexes is linguistic value, or the evaluation value of the evaluation experts is not consistent. These reasons lead to ambiguity in the decision information, usually in the form of uncertain linguistic values and numerical interval values. In this paper, the risk preference of decision maker is considered when constructing the evaluation model. Interval-valued multiple-attribute decision-making method and fuzzy linguistic multiple-attribute decision-making model are proposed. Finally, the mathematical model of efficiency evaluation of integrated energy system is constructed.

Report of the Inter-Organizational Committee on Evaluation. Internal Evaluation Model.

ERIC Educational Resources Information Center

White, Roy; Murray, John

Based upon the premise that school divisions in Manitoba, Canada, should evaluate and improve upon themselves, this evaluation model was developed. The participating personnel and the development of the evaluation model are described. The model has 11 parts: (1) needs assessment; (2) statement of objectives; (3) definition of objectives; (4)…

The Spiral-Interactive Program Evaluation Model.

ERIC Educational Resources Information Center

Khaleel, Ibrahim Adamu

1988-01-01

Describes the spiral interactive program evaluation model, which is designed to evaluate vocational-technical education programs in secondary schools in Nigeria. Program evaluation is defined; utility oriented and process oriented models for evaluation are described; and internal and external evaluative factors and variables that define each…

Model Evaluation of Continuous Data Pharmacometric Models: Metrics and Graphics

PubMed Central

Nguyen, THT; Mouksassi, M‐S; Holford, N; Al‐Huniti, N; Freedman, I; Hooker, AC; John, J; Karlsson, MO; Mould, DR; Pérez Ruixo, JJ; Plan, EL; Savic, R; van Hasselt, JGC; Weber, B; Zhou, C; Comets, E

2017-01-01

This article represents the first in a series of tutorials on model evaluation in nonlinear mixed effect models (NLMEMs), from the International Society of Pharmacometrics (ISoP) Model Evaluation Group. Numerous tools are available for evaluation of NLMEM, with a particular emphasis on visual assessment. This first basic tutorial focuses on presenting graphical evaluation tools of NLMEM for continuous data. It illustrates graphs for correct or misspecified models, discusses their pros and cons, and recalls the definition of metrics used. PMID:27884052

Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases : a review and suggested reporting framework.

PubMed

Haji Ali Afzali, Hossein; Gray, Jodi; Karnon, Jonathan

2013-04-01

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess 'best practice' in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.

Presenting an Evaluation Model for the Cancer Registry Software.

PubMed

Moghaddasi, Hamid; Asadi, Farkhondeh; Rabiei, Reza; Rahimi, Farough; Shahbodaghi, Reihaneh

2017-12-01

As cancer is increasingly growing, cancer registry is of great importance as the main core of cancer control programs, and many different software has been designed for this purpose. Therefore, establishing a comprehensive evaluation model is essential to evaluate and compare a wide range of such software. In this study, the criteria of the cancer registry software have been determined by studying the documents and two functional software of this field. The evaluation tool was a checklist and in order to validate the model, this checklist was presented to experts in the form of a questionnaire. To analyze the results of validation, an agreed coefficient of %75 was determined in order to apply changes. Finally, when the model was approved, the final version of the evaluation model for the cancer registry software was presented. The evaluation model of this study contains tool and method of evaluation. The evaluation tool is a checklist including the general and specific criteria of the cancer registry software along with their sub-criteria. The evaluation method of this study was chosen as a criteria-based evaluation method based on the findings. The model of this study encompasses various dimensions of cancer registry software and a proper method for evaluating it. The strong point of this evaluation model is the separation between general criteria and the specific ones, while trying to fulfill the comprehensiveness of the criteria. Since this model has been validated, it can be used as a standard to evaluate the cancer registry software.

How Do You Evaluate Everyone Who Isn't a Teacher? An Adaptable Evaluation Model for Professional Support Personnel.

ERIC Educational Resources Information Center

Stronge, James H.; And Others

The evaluation of professional support personnel in the schools has been a neglected area in educational evaluation. The Center for Research on Educational Accountability and Teacher Evaluation (CREATE) has worked to develop a conceptually sound evaluation model and then to translate the model into practical evaluation procedures that facilitate…

Teacher Evaluation Models: Compliance or Growth Oriented?

ERIC Educational Resources Information Center

Clenchy, Kelly R.

2017-01-01

This research study reviewed literature specific to the evolution of teacher evaluation models and explored the effectiveness of standards-based evaluation models' potential to facilitate professional growth. The researcher employed descriptive phenomenology to conduct a study of teachers' perceptions of a standard-based evaluation model's…

Evaluation of animal models of neurobehavioral disorders

PubMed Central

van der Staay, F Josef; Arndt, Saskia S; Nordquist, Rebecca E

2009-01-01

Animal models play a central role in all areas of biomedical research. The process of animal model building, development and evaluation has rarely been addressed systematically, despite the long history of using animal models in the investigation of neuropsychiatric disorders and behavioral dysfunctions. An iterative, multi-stage trajectory for developing animal models and assessing their quality is proposed. The process starts with defining the purpose(s) of the model, preferentially based on hypotheses about brain-behavior relationships. Then, the model is developed and tested. The evaluation of the model takes scientific and ethical criteria into consideration. Model development requires a multidisciplinary approach. Preclinical and clinical experts should establish a set of scientific criteria, which a model must meet. The scientific evaluation consists of assessing the replicability/reliability, predictive, construct and external validity/generalizability, and relevance of the model. We emphasize the role of (systematic and extended) replications in the course of the validation process. One may apply a multiple-tiered 'replication battery' to estimate the reliability/replicability, validity, and generalizability of result. Compromised welfare is inherent in many deficiency models in animals. Unfortunately, 'animal welfare' is a vaguely defined concept, making it difficult to establish exact evaluation criteria. Weighing the animal's welfare and considerations as to whether action is indicated to reduce the discomfort must accompany the scientific evaluation at any stage of the model building and evaluation process. Animal model building should be discontinued if the model does not meet the preset scientific criteria, or when animal welfare is severely compromised. The application of the evaluation procedure is exemplified using the rat with neonatal hippocampal lesion as a proposed model of schizophrenia. In a manner congruent to that for improving animal models, guided by the procedure expounded upon in this paper, the developmental and evaluation procedure itself may be improved by careful definition of the purpose(s) of a model and by defining better evaluation criteria, based on the proposed use of the model. PMID:19243583

A merged model of quality improvement and evaluation: maximizing return on investment.

PubMed

Woodhouse, Lynn D; Toal, Russ; Nguyen, Trang; Keene, DeAnna; Gunn, Laura; Kellum, Andrea; Nelson, Gary; Charles, Simone; Tedders, Stuart; Williams, Natalie; Livingood, William C

2013-11-01

Quality improvement (QI) and evaluation are frequently considered to be alternative approaches for monitoring and assessing program implementation and impact. The emphasis on third-party evaluation, particularly associated with summative evaluation, and the grounding of evaluation in the social and behavioral science contrast with an emphasis on the integration of QI process within programs or organizations and its origins in management science and industrial engineering. Working with a major philanthropic organization in Georgia, we illustrate how a QI model is integrated with evaluation for five asthma prevention and control sites serving poor and underserved communities in rural and urban Georgia. A primary foundation of this merged model of QI and evaluation is a refocusing of the evaluation from an intimidating report card summative evaluation by external evaluators to an internally engaged program focus on developmental evaluation. The benefits of the merged model to both QI and evaluation are discussed. The use of evaluation based logic models can help anchor a QI program in evidence-based practice and provide linkage between process and outputs with the longer term distal outcomes. Merging the QI approach with evaluation has major advantages, particularly related to enhancing the funder's return on investment. We illustrate how a Plan-Do-Study-Act model of QI can (a) be integrated with evaluation based logic models, (b) help refocus emphasis from summative to developmental evaluation, (c) enhance program ownership and engagement in evaluation activities, and (d) increase the role of evaluators in providing technical assistance and support.

A participatory evaluation model for Healthier Communities: developing indicators for New Mexico.

PubMed Central

Wallerstein, N

2000-01-01

Participatory evaluation models that invite community coalitions to take an active role in developing evaluations of their programs are a natural fit with Healthy Communities initiatives. The author describes the development of a participatory evaluation model for New Mexico's Healthier Communities program. She describes evaluation principles, research questions, and baseline findings. The evaluation model shows the links between process, community-level system impacts, and population health changes. PMID:10968754

Agent-based modeling as a tool for program design and evaluation.

PubMed

Lawlor, Jennifer A; McGirr, Sara

2017-12-01

Recently, systems thinking and systems science approaches have gained popularity in the field of evaluation; however, there has been relatively little exploration of how evaluators could use quantitative tools to assist in the implementation of systems approaches therein. The purpose of this paper is to explore potential uses of one such quantitative tool, agent-based modeling, in evaluation practice. To this end, we define agent-based modeling and offer potential uses for it in typical evaluation activities, including: engaging stakeholders, selecting an intervention, modeling program theory, setting performance targets, and interpreting evaluation results. We provide demonstrative examples from published agent-based modeling efforts both inside and outside the field of evaluation for each of the evaluative activities discussed. We further describe potential pitfalls of this tool and offer cautions for evaluators who may chose to implement it in their practice. Finally, the article concludes with a discussion of the future of agent-based modeling in evaluation practice and a call for more formal exploration of this tool as well as other approaches to simulation modeling in the field. Copyright © 2017 Elsevier Ltd. All rights reserved.

Evaluation of Models of the Reading Process.

ERIC Educational Resources Information Center

Balajthy, Ernest

A variety of reading process models have been proposed and evaluated in reading research. Traditional approaches to model evaluation specify the workings of a system in a simplified fashion to enable organized, systematic study of the system's components. Following are several statistical methods of model evaluation: (1) empirical research on…

PREFACE SPECIAL ISSUE ON MODEL EVALUATION: EVALUATION OF URBAN AND REGIONAL EULERIAN AIR QUALITY MODELS

EPA Science Inventory

The "Preface to the Special Edition on Model Evaluation: Evaluation of Urban and Regional Eulerian Air Quality Models" is a brief introduction to the papers included in a special issue of Atmospheric Environment. The Preface provides a background for the papers, which have thei...

Empirically evaluating decision-analytic models.

PubMed

Goldhaber-Fiebert, Jeremy D; Stout, Natasha K; Goldie, Sue J

2010-08-01

Model-based cost-effectiveness analyses support decision-making. To augment model credibility, evaluation via comparison to independent, empirical studies is recommended. We developed a structured reporting format for model evaluation and conducted a structured literature review to characterize current model evaluation recommendations and practices. As an illustration, we applied the reporting format to evaluate a microsimulation of human papillomavirus and cervical cancer. The model's outputs and uncertainty ranges were compared with multiple outcomes from a study of long-term progression from high-grade precancer (cervical intraepithelial neoplasia [CIN]) to cancer. Outcomes included 5 to 30-year cumulative cancer risk among women with and without appropriate CIN treatment. Consistency was measured by model ranges overlapping study confidence intervals. The structured reporting format included: matching baseline characteristics and follow-up, reporting model and study uncertainty, and stating metrics of consistency for model and study results. Structured searches yielded 2963 articles with 67 meeting inclusion criteria and found variation in how current model evaluations are reported. Evaluation of the cervical cancer microsimulation, reported using the proposed format, showed a modeled cumulative risk of invasive cancer for inadequately treated women of 39.6% (30.9-49.7) at 30 years, compared with the study: 37.5% (28.4-48.3). For appropriately treated women, modeled risks were 1.0% (0.7-1.3) at 30 years, study: 1.5% (0.4-3.3). To support external and projective validity, cost-effectiveness models should be iteratively evaluated as new studies become available, with reporting standardized to facilitate assessment. Such evaluations are particularly relevant for models used to conduct comparative effectiveness analyses.

Evaluation Theory, Models, and Applications

ERIC Educational Resources Information Center

Stufflebeam, Daniel L.; Shinkfield, Anthony J.

2007-01-01

"Evaluation Theory, Models, and Applications" is designed for evaluators and students who need to develop a commanding knowledge of the evaluation field: its history, theory and standards, models and approaches, procedures, and inclusion of personnel as well as program evaluation. This important book shows how to choose from a growing…

A Model for the Evaluation of Educational Products.

ERIC Educational Resources Information Center

Bertram, Charles L.

A model for the evaluation of educational products based on experience with development of three such products is described. The purpose of the evaluation model is to indicate the flow of evaluation activity as products undergo development. Evaluation is given Stufflebeam's definition as the process of delineating, obtaining, and providing useful…

Comparison of the Mortality Probability Admission Model III, National Quality Forum, and Acute Physiology and Chronic Health Evaluation IV hospital mortality models: implications for national benchmarking*.

PubMed

Kramer, Andrew A; Higgins, Thomas L; Zimmerman, Jack E

2014-03-01

To examine the accuracy of the original Mortality Probability Admission Model III, ICU Outcomes Model/National Quality Forum modification of Mortality Probability Admission Model III, and Acute Physiology and Chronic Health Evaluation IVa models for comparing observed and risk-adjusted hospital mortality predictions. Retrospective paired analyses of day 1 hospital mortality predictions using three prognostic models. Fifty-five ICUs at 38 U.S. hospitals from January 2008 to December 2012. Among 174,001 intensive care admissions, 109,926 met model inclusion criteria and 55,304 had data for mortality prediction using all three models. None. We compared patient exclusions and the discrimination, calibration, and accuracy for each model. Acute Physiology and Chronic Health Evaluation IVa excluded 10.7% of all patients, ICU Outcomes Model/National Quality Forum 20.1%, and Mortality Probability Admission Model III 24.1%. Discrimination of Acute Physiology and Chronic Health Evaluation IVa was superior with area under receiver operating curve (0.88) compared with Mortality Probability Admission Model III (0.81) and ICU Outcomes Model/National Quality Forum (0.80). Acute Physiology and Chronic Health Evaluation IVa was better calibrated (lowest Hosmer-Lemeshow statistic). The accuracy of Acute Physiology and Chronic Health Evaluation IVa was superior (adjusted Brier score = 31.0%) to that for Mortality Probability Admission Model III (16.1%) and ICU Outcomes Model/National Quality Forum (17.8%). Compared with observed mortality, Acute Physiology and Chronic Health Evaluation IVa overpredicted mortality by 1.5% and Mortality Probability Admission Model III by 3.1%; ICU Outcomes Model/National Quality Forum underpredicted mortality by 1.2%. Calibration curves showed that Acute Physiology and Chronic Health Evaluation performed well over the entire risk range, unlike the Mortality Probability Admission Model and ICU Outcomes Model/National Quality Forum models. Acute Physiology and Chronic Health Evaluation IVa had better accuracy within patient subgroups and for specific admission diagnoses. Acute Physiology and Chronic Health Evaluation IVa offered the best discrimination and calibration on a large common dataset and excluded fewer patients than Mortality Probability Admission Model III or ICU Outcomes Model/National Quality Forum. The choice of ICU performance benchmarks should be based on a comparison of model accuracy using data for identical patients.

Bayesian cross-validation for model evaluation and selection, with application to the North American Breeding Bird Survey

USGS Publications Warehouse

Link, William; Sauer, John R.

2016-01-01

The analysis of ecological data has changed in two important ways over the last 15 years. The development and easy availability of Bayesian computational methods has allowed and encouraged the fitting of complex hierarchical models. At the same time, there has been increasing emphasis on acknowledging and accounting for model uncertainty. Unfortunately, the ability to fit complex models has outstripped the development of tools for model selection and model evaluation: familiar model selection tools such as Akaike's information criterion and the deviance information criterion are widely known to be inadequate for hierarchical models. In addition, little attention has been paid to the evaluation of model adequacy in context of hierarchical modeling, i.e., to the evaluation of fit for a single model. In this paper, we describe Bayesian cross-validation, which provides tools for model selection and evaluation. We describe the Bayesian predictive information criterion and a Bayesian approximation to the BPIC known as the Watanabe-Akaike information criterion. We illustrate the use of these tools for model selection, and the use of Bayesian cross-validation as a tool for model evaluation, using three large data sets from the North American Breeding Bird Survey.

a model based on crowsourcing for detecting natural hazards

NASA Astrophysics Data System (ADS)

Duan, J.; Ma, C.; Zhang, J.; Liu, S.; Liu, J.

2015-12-01

Remote Sensing Technology provides a new method for the detecting,early warning,mitigation and relief of natural hazards. Given the suddenness and the unpredictability of the location of natural hazards as well as the actual demands for hazards work, this article proposes an evaluation model for remote sensing detecting of natural hazards based on crowdsourcing. Firstly, using crowdsourcing model and with the help of the Internet and the power of hundreds of millions of Internet users, this evaluation model provides visual interpretation of high-resolution remote sensing images of hazards area and collects massive valuable disaster data; secondly, this evaluation model adopts the strategy of dynamic voting consistency to evaluate the disaster data provided by the crowdsourcing workers; thirdly, this evaluation model pre-estimates the disaster severity with the disaster pre-evaluation model based on regional buffers; lastly, the evaluation model actuates the corresponding expert system work according to the forecast results. The idea of this model breaks the boundaries between geographic information professionals and the public, makes the public participation and the citizen science eventually be realized, and improves the accuracy and timeliness of hazards assessment results.

Decisionmaking Context Model for Enhancing Evaluation Utilization.

ERIC Educational Resources Information Center

Brown, Robert D.; And Others

1984-01-01

This paper discusses two models that hold promise for helping evaluators understand and cope with different decision contexts: (1) the conflict Model (Janis and Mann, 1977) and the Social Process Model (Vroom and Yago, 1974). Implications and guidelines for using decisionmaking models in evaluation settings are presented. (BS)

An Evaluation System for the Online Training Programs in Meteorology and Hydrology

ERIC Educational Resources Information Center

Wang, Yong; Zhi, Xiefei

2009-01-01

This paper studies the current evaluation system for the online training program in meteorology and hydrology. CIPP model that includes context evaluation, input evaluation, process evaluation and product evaluation differs from Kirkpatrick model including reactions evaluation, learning evaluation, transfer evaluation and results evaluation in…

Level-Specific Evaluation of Model Fit in Multilevel Structural Equation Modeling

ERIC Educational Resources Information Center

Ryu, Ehri; West, Stephen G.

2009-01-01

In multilevel structural equation modeling, the "standard" approach to evaluating the goodness of model fit has a potential limitation in detecting the lack of fit at the higher level. Level-specific model fit evaluation can address this limitation and is more informative in locating the source of lack of model fit. We proposed level-specific test…

Evaluation of integrated assessment model hindcast experiments: a case study of the GCAM 3.0 land use module

DOE Office of Scientific and Technical Information (OSTI.GOV)

Snyder, Abigail C.; Link, Robert P.; Calvin, Katherine V.

Hindcasting experiments (conducting a model forecast for a time period in which observational data are available) are being undertaken increasingly often by the integrated assessment model (IAM) community, across many scales of models. When they are undertaken, the results are often evaluated using global aggregates or otherwise highly aggregated skill scores that mask deficiencies. We select a set of deviation-based measures that can be applied on different spatial scales (regional versus global) to make evaluating the large number of variable–region combinations in IAMs more tractable. We also identify performance benchmarks for these measures, based on the statistics of the observationalmore » dataset, that allow a model to be evaluated in absolute terms rather than relative to the performance of other models at similar tasks. An ideal evaluation method for hindcast experiments in IAMs would feature both absolute measures for evaluation of a single experiment for a single model and relative measures to compare the results of multiple experiments for a single model or the same experiment repeated across multiple models, such as in community intercomparison studies. The performance benchmarks highlight the use of this scheme for model evaluation in absolute terms, providing information about the reasons a model may perform poorly on a given measure and therefore identifying opportunities for improvement. To demonstrate the use of and types of results possible with the evaluation method, the measures are applied to the results of a past hindcast experiment focusing on land allocation in the Global Change Assessment Model (GCAM) version 3.0. The question of how to more holistically evaluate models as complex as IAMs is an area for future research. We find quantitative evidence that global aggregates alone are not sufficient for evaluating IAMs that require global supply to equal global demand at each time period, such as GCAM. The results of this work indicate it is unlikely that a single evaluation measure for all variables in an IAM exists, and therefore sector-by-sector evaluation may be necessary.« less

Evaluation of integrated assessment model hindcast experiments: a case study of the GCAM 3.0 land use module

DOE PAGES

Snyder, Abigail C.; Link, Robert P.; Calvin, Katherine V.

2017-11-29

Hindcasting experiments (conducting a model forecast for a time period in which observational data are available) are being undertaken increasingly often by the integrated assessment model (IAM) community, across many scales of models. When they are undertaken, the results are often evaluated using global aggregates or otherwise highly aggregated skill scores that mask deficiencies. We select a set of deviation-based measures that can be applied on different spatial scales (regional versus global) to make evaluating the large number of variable–region combinations in IAMs more tractable. We also identify performance benchmarks for these measures, based on the statistics of the observationalmore » dataset, that allow a model to be evaluated in absolute terms rather than relative to the performance of other models at similar tasks. An ideal evaluation method for hindcast experiments in IAMs would feature both absolute measures for evaluation of a single experiment for a single model and relative measures to compare the results of multiple experiments for a single model or the same experiment repeated across multiple models, such as in community intercomparison studies. The performance benchmarks highlight the use of this scheme for model evaluation in absolute terms, providing information about the reasons a model may perform poorly on a given measure and therefore identifying opportunities for improvement. To demonstrate the use of and types of results possible with the evaluation method, the measures are applied to the results of a past hindcast experiment focusing on land allocation in the Global Change Assessment Model (GCAM) version 3.0. The question of how to more holistically evaluate models as complex as IAMs is an area for future research. We find quantitative evidence that global aggregates alone are not sufficient for evaluating IAMs that require global supply to equal global demand at each time period, such as GCAM. The results of this work indicate it is unlikely that a single evaluation measure for all variables in an IAM exists, and therefore sector-by-sector evaluation may be necessary.« less

Evaluation of integrated assessment model hindcast experiments: a case study of the GCAM 3.0 land use module

NASA Astrophysics Data System (ADS)

Snyder, Abigail C.; Link, Robert P.; Calvin, Katherine V.

2017-11-01

Hindcasting experiments (conducting a model forecast for a time period in which observational data are available) are being undertaken increasingly often by the integrated assessment model (IAM) community, across many scales of models. When they are undertaken, the results are often evaluated using global aggregates or otherwise highly aggregated skill scores that mask deficiencies. We select a set of deviation-based measures that can be applied on different spatial scales (regional versus global) to make evaluating the large number of variable-region combinations in IAMs more tractable. We also identify performance benchmarks for these measures, based on the statistics of the observational dataset, that allow a model to be evaluated in absolute terms rather than relative to the performance of other models at similar tasks. An ideal evaluation method for hindcast experiments in IAMs would feature both absolute measures for evaluation of a single experiment for a single model and relative measures to compare the results of multiple experiments for a single model or the same experiment repeated across multiple models, such as in community intercomparison studies. The performance benchmarks highlight the use of this scheme for model evaluation in absolute terms, providing information about the reasons a model may perform poorly on a given measure and therefore identifying opportunities for improvement. To demonstrate the use of and types of results possible with the evaluation method, the measures are applied to the results of a past hindcast experiment focusing on land allocation in the Global Change Assessment Model (GCAM) version 3.0. The question of how to more holistically evaluate models as complex as IAMs is an area for future research. We find quantitative evidence that global aggregates alone are not sufficient for evaluating IAMs that require global supply to equal global demand at each time period, such as GCAM. The results of this work indicate it is unlikely that a single evaluation measure for all variables in an IAM exists, and therefore sector-by-sector evaluation may be necessary.

The bottom-up approach to integrative validity: a new perspective for program evaluation.

PubMed

Chen, Huey T

2010-08-01

The Campbellian validity model and the traditional top-down approach to validity have had a profound influence on research and evaluation. That model includes the concepts of internal and external validity and within that model, the preeminence of internal validity as demonstrated in the top-down approach. Evaluators and researchers have, however, increasingly recognized that in an evaluation, the over-emphasis on internal validity reduces that evaluation's usefulness and contributes to the gulf between academic and practical communities regarding interventions. This article examines the limitations of the Campbellian validity model and the top-down approach and provides a comprehensive, alternative model, known as the integrative validity model for program evaluation. The integrative validity model includes the concept of viable validity, which is predicated on a bottom-up approach to validity. This approach better reflects stakeholders' evaluation views and concerns, makes external validity workable, and becomes therefore a preferable alternative for evaluation of health promotion/social betterment programs. The integrative validity model and the bottom-up approach enable evaluators to meet scientific and practical requirements, facilitate in advancing external validity, and gain a new perspective on methods. The new perspective also furnishes a balanced view of credible evidence, and offers an alternative perspective for funding. Copyright (c) 2009 Elsevier Ltd. All rights reserved.

The Third Phase of AQMEII: Evaluation Strategy and Multi-Model Performance Analysis

EPA Science Inventory

AQMEII (Air Quality Model Evaluation International Initiative) is an extraordinary effort promoting policy-relevant research on regional air quality model evaluation across the European and North American atmospheric modelling communities, providing the ideal platform for advanci...

Training Module on the Evaluation of Best Modeling Practices

EPA Pesticide Factsheets

Building upon the fundamental concepts outlined in previous modules, the objectives of this module are to explore the topic of model evaluation and identify the 'best modeling practices' and strategies for the Evaluation Stage of the model life-cycle.

A Multi Criteria Group Decision-Making Model for Teacher Evaluation in Higher Education Based on Cloud Model and Decision Tree

ERIC Educational Resources Information Center

Chang, Ting-Cheng; Wang, Hui

2016-01-01

This paper proposes a cloud multi-criteria group decision-making model for teacher evaluation in higher education which is involving subjectivity, imprecision and fuzziness. First, selecting the appropriate evaluation index depending on the evaluation objectives, indicating a clear structural relationship between the evaluation index and…

Software Quality Evaluation Models Applicable in Health Information and Communications Technologies. A Review of the Literature.

PubMed

Villamor Ordozgoiti, Alberto; Delgado Hito, Pilar; Guix Comellas, Eva María; Fernandez Sanchez, Carlos Manuel; Garcia Hernandez, Milagros; Lluch Canut, Teresa

2016-01-01

Information and Communications Technologies in healthcare has increased the need to consider quality criteria through standardised processes. The aim of this study was to analyse the software quality evaluation models applicable to healthcare from the perspective of ICT-purchasers. Through a systematic literature review with the keywords software, product, quality, evaluation and health, we selected and analysed 20 original research papers published from 2005-2016 in health science and technology databases. The results showed four main topics: non-ISO models, software quality evaluation models based on ISO/IEC standards, studies analysing software quality evaluation models, and studies analysing ISO standards for software quality evaluation. The models provide cost-efficiency criteria for specific software, and improve use outcomes. The ISO/IEC25000 standard is shown as the most suitable for evaluating the quality of ICTs for healthcare use from the perspective of institutional acquisition.

CMAQ Involvement in Air Quality Model Evaluation International Initiative

EPA Pesticide Factsheets

Description of Air Quality Model Evaluation International Initiative (AQMEII). Different chemical transport models are applied by different groups over North America and Europe and evaluated against observations.

THE ATMOSPHERIC MODEL EVALUATION TOOL (AMET); AIR QUALITY MODULE

EPA Science Inventory

This presentation reviews the development of the Atmospheric Model Evaluation Tool (AMET) air quality module. The AMET tool is being developed to aid in the model evaluation. This presentation focuses on the air quality evaluation portion of AMET. Presented are examples of the...

Systematic evaluation of atmospheric chemistry-transport model CHIMERE

NASA Astrophysics Data System (ADS)

Khvorostyanov, Dmitry; Menut, Laurent; Mailler, Sylvain; Siour, Guillaume; Couvidat, Florian; Bessagnet, Bertrand; Turquety, Solene

2017-04-01

Regional-scale atmospheric chemistry-transport models (CTM) are used to develop air quality regulatory measures, to support environmentally sensitive decisions in the industry, and to address variety of scientific questions involving the atmospheric composition. Model performance evaluation with measurement data is critical to understand their limits and the degree of confidence in model results. CHIMERE CTM (http://www.lmd.polytechnique.fr/chimere/) is a French national tool for operational forecast and decision support and is widely used in the international research community in various areas of atmospheric chemistry and physics, climate, and environment (http://www.lmd.polytechnique.fr/chimere/CW-articles.php). This work presents the model evaluation framework applied systematically to the new CHIMERE CTM versions in the course of the continuous model development. The framework uses three of the four CTM evaluation types identified by the Environmental Protection Agency (EPA) and the American Meteorological Society (AMS): operational, diagnostic, and dynamic. It allows to compare the overall model performance in subsequent model versions (operational evaluation), identify specific processes and/or model inputs that could be improved (diagnostic evaluation), and test the model sensitivity to the changes in air quality, such as emission reductions and meteorological events (dynamic evaluation). The observation datasets currently used for the evaluation are: EMEP (surface concentrations), AERONET (optical depths), and WOUDC (ozone sounding profiles). The framework is implemented as an automated processing chain and allows interactive exploration of the results via a web interface.

Atmospheric Model Evaluation Tool for meteorological and air quality simulations

EPA Pesticide Factsheets

The Atmospheric Model Evaluation Tool compares model predictions to observed data from various meteorological and air quality observation networks to help evaluate meteorological and air quality simulations.

Use of program logic models in the Southern Rural Access Program evaluation.

PubMed

Pathman, Donald; Thaker, Samruddhi; Ricketts, Thomas C; Albright, Jennifer B

2003-01-01

The Southern Rural Access Program (SRAP) evaluation team used program logic models to clarify grantees' activities, objectives, and timelines. This information was used to benchmark data from grantees' progress reports to assess the program's successes. This article presents a brief background on the use of program logic models--essentially charts or diagrams specifying a program's planned activities, objectives, and goals--for evaluating and managing a program. It discusses the structure of the logic models chosen for the SRAP and how the model concept was introduced to the grantees to promote acceptance and use of the models. The article describes how the models helped clarify the program's objectives and helped lead agencies plan and manage the many program initiatives and subcontractors in their states. Models also provided a framework for grantees to report their progress to the National Program Office and evaluators and promoted the evaluators' visibility and acceptance by the grantees. Program logics, however, increased grantees' reporting requirements and demanded substantial time of the evaluators. Program logic models, on balance, proved their merit in the SRAP through their contributions to its management and evaluation and by providing a better understanding of the program's initiatives, successes, and potential impact.

EVALUATION TECHNIQUES AND TOOL DEVELOPMENT FOR FY 08 CMAQ RELEASE

EPA Science Inventory

In this task, research efforts are outlined that relate to the AMD Model Evaluation Program element and support CMAQ releases within the FY05-FY08 time period. Model evaluation serves dual purposes; evaluation is necessary to characterize the accuracy of model predictions, and e...

A Conceptual Framework Curriculum Evaluation Electrical Engineering Education

ERIC Educational Resources Information Center

Imansari, Nurulita; Sutadji, Eddy

2017-01-01

This evaluation is a conceptual framework that has been analyzed in the hope that can help research related an evaluation of the curriculum. The Model of evaluation used was CIPPO model. CIPPO Model consists of "context," "input," "process," "product," and "outcomes." On the dimension of the…

The Use of AMET and Automated Scripts for Model Evaluation

EPA Science Inventory

The Atmospheric Model Evaluation Tool (AMET) is a suite of software designed to facilitate the analysis and evaluation of meteorological and air quality models. AMET matches the model output for particular locations to the corresponding observed values from one or more networks ...

Developing Best Practices for Detecting Change at Marine Renewable Energy Sites

NASA Astrophysics Data System (ADS)

Linder, H. L.; Horne, J. K.

2016-02-01

In compliance with the National Environmental Policy Act (NEPA), an evaluation of environmental effects is mandatory for obtaining permits for any Marine Renewable Energy (MRE) project in the US. Evaluation includes an assessment of baseline conditions and on-going monitoring during operation to determine if biological conditions change relative to the baseline. Currently, there are no best practices for the analysis of MRE monitoring data. We have developed an approach to evaluate and recommend analytic models used to characterize and detect change in biological monitoring data. The approach includes six steps: review current MRE monitoring practices, identify candidate models to analyze data, fit models to a baseline dataset, develop simulated scenarios of change, evaluate model fit to simulated data, and produce recommendations on the choice of analytic model for monitoring data. An empirical data set from a proposed tidal turbine site at Admiralty Inlet, Puget Sound, Washington was used to conduct the model evaluation. Candidate models that were evaluated included: linear regression, time series, and nonparametric models. Model fit diagnostics Root-Mean-Square-Error and Mean-Absolute-Scaled-Error were used to measure accuracy of predicted values from each model. A power analysis was used to evaluate the ability of each model to measure and detect change from baseline conditions. As many of these models have yet to be applied in MRE monitoring studies, results of this evaluation will generate comprehensive guidelines on choice of model to detect change in environmental monitoring data from MRE sites. The creation of standardized guidelines for model selection enables accurate comparison of change between life stages of a MRE project, within life stages to meet real time regulatory requirements, and comparison of environmental changes among MRE sites.

An evaluation of evaluative personality terms: a comparison of the big seven and five-factor model in predicting psychopathology.

PubMed

Durrett, Christine; Trull, Timothy J

2005-09-01

Two personality models are compared regarding their relationship with personality disorder (PD) symptom counts and with lifetime Axis I diagnoses. These models share 5 similar domains, and the Big 7 model also includes 2 domains assessing self-evaluation: positive and negative valence. The Big 7 model accounted for more variance in PDs than the 5-factor model, primarily because of the association of negative valence with most PDs. Although low-positive valence was associated with most Axis I diagnoses, the 5-factor model generally accounted for more variance in Axis I diagnoses than the Big 7 model. Some predicted associations between self-evaluation and psychopathology were not found, and unanticipated associations emerged. These findings are discussed regarding the utility of evaluative terms in clinical assessment.

[Transparency as a prerequisite of innovation in health services research: deficits in the reporting of model projects concerning managed care].

PubMed

Wiethege, J; Ommen, O; Ernstmann, N; Pfaff, H

2010-10-01

Currently, elements of managed care are being implemented in the German health-care system. The legal basis for these innovations are § 140, § 73, § 137, and §§ 63 et seq. of the German Social Code - Part 5 (SGB V). For the model projects according to §§ 63 et seq. of the German Social Code a scientific evaluation and publication of the evaluation results is mandatory. The present study examines the status of evaluation of German model projects. The present study has a mixed method design: A mail and telephone survey with the German Federal Social Insurance Authority, the health insurance funds, and the regional Associations of Statutory Health Insurance Physicians has been conducted. Furthermore, an internet research on "Medpilot" and "Google" has been accomplished to search for model projects and their evaluation reports. 34 model projects met the inclusion criteria. 13 of these projects had been terminated up to 30/9/2008. 6 of them have published an evaluation report. 4 model projects have published substantial documents. One model project in progress has published a meaningful interim report. 12 model projects failed to give information concerning the evaluator or the duration of the model projects. The results show a significant deficit in the mandatory reporting of the evaluation of model projects in Germany. There is a need for action for the legislator and the health insurance funds in terms of promoting the evaluation and the publication of the results. The institutions evaluating the model projects should obligate themselves to publish the evaluation results. The publication is an essential precondition for the development of managed care structures in the health-care system and in the development of scientific evaluation methods. © Georg Thieme Verlag KG Stuttgart · New York.

Biological Modeling As A Method for Data Evaluation and ...

EPA Pesticide Factsheets

Biological Models, evaluating consistency of data and integrating diverse data, examples of pharmacokinetics and response and pharmacodynamics Biological Models, evaluating consistency of data and integrating diverse data, examples of pharmacokinetics and response and pharmacodynamics

Impact on house staff evaluation scores when changing from a Dreyfus- to a Milestone-based evaluation model: one internal medicine residency program's findings.

PubMed

Friedman, Karen A; Balwan, Sandy; Cacace, Frank; Katona, Kyle; Sunday, Suzanne; Chaudhry, Saima

2014-01-01

As graduate medical education (GME) moves into the Next Accreditation System (NAS), programs must take a critical look at their current models of evaluation and assess how well they align with reporting outcomes. Our objective was to assess the impact on house staff evaluation scores when transitioning from a Dreyfus-based model of evaluation to a Milestone-based model of evaluation. Milestones are a key component of the NAS. We analyzed all end of rotation evaluations of house staff completed by faculty for academic years 2010-2011 (pre-Dreyfus model) and 2011-2012 (post-Milestone model) in one large university-based internal medicine residency training program. Main measures included change in PGY-level average score; slope, range, and separation of average scores across all six Accreditation Council for Graduate Medical Education (ACGME) competencies. Transitioning from a Dreyfus-based model to a Milestone-based model resulted in a larger separation in the scores between our three post-graduate year classes, a steeper progression of scores in the PGY-1 class, a wider use of the 5-point scale on our global end of rotation evaluation form, and a downward shift in the PGY-1 scores and an upward shift in the PGY-3 scores. For faculty trained in both models of assessment, the Milestone-based model had greater discriminatory ability as evidenced by the larger separation in the scores for all the classes, in particular the PGY-1 class.

[Decision modeling for economic evaluation of health technologies].

PubMed

de Soárez, Patrícia Coelho; Soares, Marta Oliveira; Novaes, Hillegonda Maria Dutilh

2014-10-01

Most economic evaluations that participate in decision-making processes for incorporation and financing of technologies of health systems use decision models to assess the costs and benefits of the compared strategies. Despite the large number of economic evaluations conducted in Brazil, there is a pressing need to conduct an in-depth methodological study of the types of decision models and their applicability in our setting. The objective of this literature review is to contribute to the knowledge and use of decision models in the national context of economic evaluations of health technologies. This article presents general definitions about models and concerns with their use; it describes the main models: decision trees, Markov chains, micro-simulation, simulation of discrete and dynamic events; it discusses the elements involved in the choice of model; and exemplifies the models addressed in national economic evaluation studies of diagnostic and therapeutic preventive technologies and health programs.

Graphical approach to assess the soil fertility evaluation model validity for rice (case study: southern area of Merapi Mountain, Indonesia)

NASA Astrophysics Data System (ADS)

Julianto, E. A.; Suntoro, W. A.; Dewi, W. S.; Partoyo

2018-03-01

Climate change has been reported to exacerbate land resources degradation including soil fertility decline. The appropriate validity use on soil fertility evaluation could reduce the risk of climate change effect on plant cultivation. This study aims to assess the validity of a Soil Fertility Evaluation Model using a graphical approach. The models evaluated were the Indonesian Soil Research Center (PPT) version model, the FAO Unesco version model, and the Kyuma version model. Each model was then correlated with rice production (dry grain weight/GKP). The goodness of fit of each model can be tested to evaluate the quality and validity of a model, as well as the regression coefficient (R2). This research used the Eviews 9 programme by a graphical approach. The results obtained three curves, namely actual, fitted, and residual curves. If the actual and fitted curves are widely apart or irregular, this means that the quality of the model is not good, or there are many other factors that are still not included in the model (large residual) and conversely. Indeed, if the actual and fitted curves show exactly the same shape, it means that all factors have already been included in the model. Modification of the standard soil fertility evaluation models can improve the quality and validity of a model.

Study on process evaluation model of students' learning in practical course

NASA Astrophysics Data System (ADS)

Huang, Jie; Liang, Pei; Shen, Wei-min; Ye, Youxiang

2017-08-01

In practical course teaching based on project object method, the traditional evaluation methods include class attendance, assignments and exams fails to give incentives to undergraduate students to learn innovatively and autonomously. In this paper, the element such as creative innovation, teamwork, document and reporting were put into process evaluation methods, and a process evaluation model was set up. Educational practice shows that the evaluation model makes process evaluation of students' learning more comprehensive, accurate, and fairly.

Multilevel Evaluation Alignment: An Explication of a Four-Step Model

ERIC Educational Resources Information Center

Yang, Huilan; Shen, Jianping; Cao, Honggao; Warfield, Charles

2004-01-01

Using the evaluation work on the W.K. Kellogg Foundation's Unleashing Resources Initiative as an example, in this article we explicate a general four-step model appropriate for multilevel evaluation alignment. We review the relevant literature, argue for the need for evaluation alignment in a multilevel context, explain the four-step model,…

A Course Evaluation System in an Open University.

ERIC Educational Resources Information Center

Chacon, Fabio J.

A model is presented for response to evaluating instruction in a university based on the teaching-at-a-distance concept. Technically appropriate and operationally viable, this model is applied to the National Open University of Venezuela (UNA). The model is based on two principles of educational evaluation: (1) the concept of evaluation as a…

Models and Mechanisms for Evaluating Government-Funded Research: An International Comparison

ERIC Educational Resources Information Center

Coryn, Chris L. S.; Hattie, John A.; Scriven, Michael; Hartmann, David J.

2007-01-01

This research describes, classifies, and comparatively evaluates national models and mechanisms used to evaluate research and allocate research funding in 16 countries. Although these models and mechanisms vary widely in terms of how research is evaluated and financed, nearly all share the common characteristic of relating funding to some measure…

Models and techniques for evaluating the effectiveness of aircraft computing systems

NASA Technical Reports Server (NTRS)

Meyer, J. F.

1978-01-01

Progress in the development of system models and techniques for the formulation and evaluation of aircraft computer system effectiveness is reported. Topics covered include: analysis of functional dependence: a prototype software package, METAPHOR, developed to aid the evaluation of performability; and a comprehensive performability modeling and evaluation exercise involving the SIFT computer.

The Model Life-cycle: Training Module

EPA Pesticide Factsheets

Model Life-Cycle includes identification of problems & the subsequent development, evaluation, & application of the model. Objectives: define ‘model life-cycle’, explore stages of model life-cycle, & strategies for development, evaluation, & applications.

Models for evaluating the performability of degradable computing systems

NASA Technical Reports Server (NTRS)

Wu, L. T.

1982-01-01

Recent advances in multiprocessor technology established the need for unified methods to evaluate computing systems performance and reliability. In response to this modeling need, a general modeling framework that permits the modeling, analysis and evaluation of degradable computing systems is considered. Within this framework, several user oriented performance variables are identified and shown to be proper generalizations of the traditional notions of system performance and reliability. Furthermore, a time varying version of the model is developed to generalize the traditional fault tree reliability evaluation methods of phased missions.

Evaluation of Liquid Fuel Spray Models for Hybrid RANS/LES and DLES Prediction of Turbulent Reactive Flows

NASA Astrophysics Data System (ADS)

Afshar, Ali

An evaluation of Lagrangian-based, discrete-phase models for multi-component liquid sprays encountered in the combustors of gas turbine engines is considered. In particular, the spray modeling capabilities of the commercial software, ANSYS Fluent, was evaluated. Spray modeling was performed for various cold flow validation cases. These validation cases include a liquid jet in a cross-flow, an airblast atomizer, and a high shear fuel nozzle. Droplet properties including velocity and diameter were investigated and compared with previous experimental and numerical results. Different primary and secondary breakup models were evaluated in this thesis. The secondary breakup models investigated include the Taylor analogy breakup (TAB) model, the wave model, the Kelvin-Helmholtz Rayleigh-Taylor model (KHRT), and the Stochastic secondary droplet (SSD) approach. The modeling of fuel sprays requires a proper treatment for the turbulence. Reynolds-averaged Navier-Stokes (RANS), large eddy simulation (LES), hybrid RANS/LES, and dynamic LES (DLES) were also considered for the turbulent flows involving sprays. The spray and turbulence models were evaluated using the available benchmark experimental data.

Effects of Model Characterization on Preschool Children's Evaluative, Imitative and Nonimitative Responses.

ERIC Educational Resources Information Center

Whitman, Thomas L.; Taub, Susan Ilene

This study examined the effects of differentially characterizing a model as "good", "bad", or "neutral" on preschool children's subsequent evaluation and imitation of the model. The model's aggressive and motor behaviors were more frequently imitated than were his non-aggressive and verbal behaviors. Instructions influenced the Ss' evaluation of…

Rhode Island Model Evaluation & Support System: Teacher. Edition III

ERIC Educational Resources Information Center

Rhode Island Department of Education, 2015

2015-01-01

Rhode Island educators believe that implementing a fair, accurate, and meaningful educator evaluation and support system will help improve teaching and learning. The primary purpose of the Rhode Island Model Teacher Evaluation and Support System (Rhode Island Model) is to help all teachers improve. Through the Model, the goal is to help create a…

Two Decades of WRF/CMAQ simulations over the continental United States: New approaches for performing dynamic model evaluation and determining confidence limits for ozone exceedances

EPA Science Inventory

Confidence in the application of models for forecasting and regulatory assessments is furthered by conducting four types of model evaluation: operational, dynamic, diagnostic, and probabilistic. Operational model evaluation alone does not reveal the confidence limits that can be ...

The performance evaluation model of mining project founded on the weight optimization entropy value method

NASA Astrophysics Data System (ADS)

Mao, Chao; Chen, Shou

2017-01-01

According to the traditional entropy value method still have low evaluation accuracy when evaluating the performance of mining projects, a performance evaluation model of mineral project founded on improved entropy is proposed. First establish a new weight assignment model founded on compatible matrix analysis of analytic hierarchy process (AHP) and entropy value method, when the compatibility matrix analysis to achieve consistency requirements, if it has differences between subjective weights and objective weights, moderately adjust both proportions, then on this basis, the fuzzy evaluation matrix for performance evaluation. The simulation experiments show that, compared with traditional entropy and compatible matrix analysis method, the proposed performance evaluation model of mining project based on improved entropy value method has higher accuracy assessment.

Land Surface Verification Toolkit (LVT) - A Generalized Framework for Land Surface Model Evaluation

NASA Technical Reports Server (NTRS)

Kumar, Sujay V.; Peters-Lidard, Christa D.; Santanello, Joseph; Harrison, Ken; Liu, Yuqiong; Shaw, Michael

2011-01-01

Model evaluation and verification are key in improving the usage and applicability of simulation models for real-world applications. In this article, the development and capabilities of a formal system for land surface model evaluation called the Land surface Verification Toolkit (LVT) is described. LVT is designed to provide an integrated environment for systematic land model evaluation and facilitates a range of verification approaches and analysis capabilities. LVT operates across multiple temporal and spatial scales and employs a large suite of in-situ, remotely sensed and other model and reanalysis datasets in their native formats. In addition to the traditional accuracy-based measures, LVT also includes uncertainty and ensemble diagnostics, information theory measures, spatial similarity metrics and scale decomposition techniques that provide novel ways for performing diagnostic model evaluations. Though LVT was originally designed to support the land surface modeling and data assimilation framework known as the Land Information System (LIS), it also supports hydrological data products from other, non-LIS environments. In addition, the analysis of diagnostics from various computational subsystems of LIS including data assimilation, optimization and uncertainty estimation are supported within LVT. Together, LIS and LVT provide a robust end-to-end environment for enabling the concepts of model data fusion for hydrological applications. The evolving capabilities of LVT framework are expected to facilitate rapid model evaluation efforts and aid the definition and refinement of formal evaluation procedures for the land surface modeling community.

EVALUATION OF THE REAL-TIME AIR-QUALITY MODEL USING THE RAPS (REGIONAL AIR POLLUTION STUDY) DATA BASE. VOLUME 4. EVALUATION GUIDE

EPA Science Inventory

The theory and programming of statistical tests for evaluating the Real-Time Air-Quality Model (RAM) using the Regional Air Pollution Study (RAPS) data base are fully documented in four volumes. Moreover, the tests are generally applicable to other model evaluation problems. Volu...

Models and techniques for evaluating the effectiveness of aircraft computing systems

NASA Technical Reports Server (NTRS)

Meyer, J. F.

1978-01-01

The development of system models that can provide a basis for the formulation and evaluation of aircraft computer system effectiveness, the formulation of quantitative measures of system effectiveness, and the development of analytic and simulation techniques for evaluating the effectiveness of a proposed or existing aircraft computer are described. Specific topics covered include: system models; performability evaluation; capability and functional dependence; computation of trajectory set probabilities; and hierarchical modeling of an air transport mission.

Impact on house staff evaluation scores when changing from a Dreyfus- to a Milestone-based evaluation model: one internal medicine residency program's findings.

PubMed

Friedman, Karen A; Balwan, Sandy; Cacace, Frank; Katona, Kyle; Sunday, Suzanne; Chaudhry, Saima

2014-01-01

Purpose As graduate medical education (GME) moves into the Next Accreditation System (NAS), programs must take a critical look at their current models of evaluation and assess how well they align with reporting outcomes. Our objective was to assess the impact on house staff evaluation scores when transitioning from a Dreyfus-based model of evaluation to a Milestone-based model of evaluation. Milestones are a key component of the NAS. Method We analyzed all end of rotation evaluations of house staff completed by faculty for academic years 2010-2011 (pre-Dreyfus model) and 2011-2012 (post-Milestone model) in one large university-based internal medicine residency training program. Main measures included change in PGY-level average score; slope, range, and separation of average scores across all six Accreditation Council for Graduate Medical Education (ACGME) competencies. Results Transitioning from a Dreyfus-based model to a Milestone-based model resulted in a larger separation in the scores between our three post-graduate year classes, a steeper progression of scores in the PGY-1 class, a wider use of the 5-point scale on our global end of rotation evaluation form, and a downward shift in the PGY-1 scores and an upward shift in the PGY-3 scores. Conclusions For faculty trained in both models of assessment, the Milestone-based model had greater discriminatory ability as evidenced by the larger separation in the scores for all the classes, in particular the PGY-1 class.

External Evaluation of Two Fluconazole Infant Population Pharmacokinetic Models

PubMed Central

Hwang, Michael F.; Beechinor, Ryan J.; Wade, Kelly C.; Benjamin, Daniel K.; Smith, P. Brian; Hornik, Christoph P.; Capparelli, Edmund V.; Duara, Shahnaz; Kennedy, Kathleen A.; Cohen-Wolkowiez, Michael

2017-01-01

ABSTRACT Fluconazole is an antifungal agent used for the treatment of invasive candidiasis, a leading cause of morbidity and mortality in premature infants. Population pharmacokinetic (PK) models of fluconazole in infants have been previously published by Wade et al. (Antimicrob Agents Chemother 52:4043–4049, 2008, https://doi.org/10.1128/AAC.00569-08) and Momper et al. (Antimicrob Agents Chemother 60:5539–5545, 2016, https://doi.org/10.1128/AAC.00963-16). Here we report the results of the first external evaluation of the predictive performance of both models. We used patient-level data from both studies to externally evaluate both PK models. The predictive performance of each model was evaluated using the model prediction error (PE), mean prediction error (MPE), mean absolute prediction error (MAPE), prediction-corrected visual predictive check (pcVPC), and normalized prediction distribution errors (NPDE). The values of the parameters of each model were reestimated using both the external and merged data sets. When evaluated with the external data set, the model proposed by Wade et al. showed lower median PE, MPE, and MAPE (0.429 μg/ml, 41.9%, and 57.6%, respectively) than the model proposed by Momper et al. (2.45 μg/ml, 188%, and 195%, respectively). The values of the majority of reestimated parameters were within 20% of their respective original parameter values for all model evaluations. Our analysis determined that though both models are robust, the model proposed by Wade et al. had greater accuracy and precision than the model proposed by Momper et al., likely because it was derived from a patient population with a wider age range. This study highlights the importance of the external evaluation of infant population PK models. PMID:28893774

Evaluation of the Navys Sea/Shore Flow Policy

DTIC Science & Technology

2016-06-01

Std. Z39.18 i Abstract CNA developed an independent Discrete -Event Simulation model to evaluate and assess the effect of...a more steady manning level, but the variability remains, even if the system is optimized. In building a Discrete -Event Simulation model, we...steady-state model. In FY 2014, CNA developed a Discrete -Event Simulation model to evaluate the impact of sea/shore flow policy (the DES-SSF model

Intelligent Evaluation Method of Tank Bottom Corrosion Status Based on Improved BP Artificial Neural Network

NASA Astrophysics Data System (ADS)

Qiu, Feng; Dai, Guang; Zhang, Ying

According to the acoustic emission information and the appearance inspection information of tank bottom online testing, the external factors associated with tank bottom corrosion status are confirmed. Applying artificial neural network intelligent evaluation method, three tank bottom corrosion status evaluation models based on appearance inspection information, acoustic emission information, and online testing information are established. Comparing with the result of acoustic emission online testing through the evaluation of test sample, the accuracy of the evaluation model based on online testing information is 94 %. The evaluation model can evaluate tank bottom corrosion accurately and realize acoustic emission online testing intelligent evaluation of tank bottom.

Evaluation of the ²³⁹Pu prompt fission neutron spectrum induced by neutrons of 500 keV and associated covariances

DOE PAGES

Neudecker, D.; Talou, P.; Kawano, T.; ...

2015-08-01

We present evaluations of the prompt fission neutron spectrum (PFNS) of ²³⁹Pu induced by 500 keV neutrons, and associated covariances. In a previous evaluation by Talou et al. 2010, surprisingly low evaluated uncertainties were obtained, partly due to simplifying assumptions in the quantification of uncertainties from experiment and model. Therefore, special emphasis is placed here on a thorough uncertainty quantification of experimental data and of the Los Alamos model predicted values entering the evaluation. In addition, the Los Alamos model was extended and an evaluation technique was employed that takes into account the qualitative differences between normalized model predicted valuesmore » and experimental shape data. These improvements lead to changes in the evaluated PFNS and overall larger evaluated uncertainties than in the previous work. However, these evaluated uncertainties are still smaller than those obtained in a statistical analysis using experimental information only, due to strong model correlations. Hence, suggestions to estimate model defect uncertainties are presented, which lead to more reasonable evaluated uncertainties. The calculated k eff of selected criticality benchmarks obtained with these new evaluations agree with each other within their uncertainties despite the different approaches to estimate model defect uncertainties. The k eff one standard deviations overlap with some of those obtained using ENDF/B-VII.1, albeit their mean values are further away from unity. Spectral indexes for the Jezebel critical assembly calculated with the newly evaluated PFNS agree with the experimental data for selected (n,γ) and (n,f) reactions, and show improvements for high-energy threshold (n,2n) reactions compared to ENDF/B-VII.1.« less

Evaluation of the 239 Pu prompt fission neutron spectrum induced by neutrons of 500 keV and associated covariances

DOE Office of Scientific and Technical Information (OSTI.GOV)

Neudecker, D.; Talou, P.; Kawano, T.

2015-08-01

We present evaluations of the prompt fission neutron spectrum (PFNS) of (PU)-P-239 induced by 500 keV neutrons, and associated covariances. In a previous evaluation by Talon et al. (2010), surprisingly low evaluated uncertainties were obtained, partly due to simplifying assumptions in the quantification of uncertainties from experiment and model. Therefore, special emphasis is placed here on a thorough uncertainty quantification of experimental data and of the Los Alamos model predicted values entering the evaluation. In addition, the Los Alamos model was extended and an evaluation technique was employed that takes into account the qualitative differences between normalized model predicted valuesmore » and experimental shape data These improvements lead to changes in the evaluated PENS and overall larger evaluated uncertainties than in the previous work. However, these evaluated uncertainties are still smaller than those obtained in a statistical analysis using experimental information only, due to strong model correlations. Hence, suggestions to estimate model defect uncertainties are presented. which lead to more reasonable evaluated uncertainties. The calculated k(eff) of selected criticality benchmarks obtained with these new evaluations agree with each other within their uncertainties despite the different approaches to estimate model defect uncertainties. The k(eff) one standard deviations overlap with some of those obtained using ENDF/B-VILl, albeit their mean values are further away from unity. Spectral indexes for the Jezebel critical assembly calculated with the newly evaluated PFNS agree with the experimental data for selected (n,) and (n,f) reactions, and show improvements for highenergy threshold (n,2n) reactions compared to ENDF/B-VII.l. (C) 2015 Elsevier B.V. All rights reserved.« less

Small area population forecasting: some experience with British models.

PubMed

Openshaw, S; Van Der Knaap, G A

1983-01-01

This study is concerned with the evaluation of the various models including time-series forecasts, extrapolation, and projection procedures, that have been developed to prepare population forecasts for planning purposes. These models are evaluated using data for the Netherlands. "As part of a research project at the Erasmus University, space-time population data has been assembled in a geographically consistent way for the period 1950-1979. These population time series are of sufficient length for the first 20 years to be used to build models and then evaluate the performance of the model for the next 10 years. Some 154 different forecasting models for 832 municipalities have been evaluated. It would appear that the best forecasts are likely to be provided by either a Holt-Winters model, or a ratio-correction model, or a low order exponential-smoothing model." excerpt

[Application of entropy-weight TOPSIS model in synthetical quality evaluation of Angelica sinensis growing in Gansu Province].

PubMed

Gu, Zhi-rong; Wang, Ya-li; Sun, Yu-jing; Dind, Jun-xia

2014-09-01

To investigate the establishment and application methods of entropy-weight TOPSIS model in synthetical quality evaluation of traditional Chinese medicine with Angelica sinensis growing in Gansu Province as an example. The contents of ferulic acid, 3-butylphthalide, Z-butylidenephthalide, Z-ligustilide, linolic acid, volatile oil, and ethanol soluble extractive were used as an evaluation index set. The weights of each evaluation index were determined by information entropy method. The entropyweight TOPSIS model was established to synthetically evaluate the quality of Angelica sinensis growing in Gansu Province by Euclid closeness degree. The results based on established model were in line with the daodi meaning and the knowledge of clinical experience. The established model was simple in calculation, objective, reliable, and can be applied to synthetical quality evaluation of traditional Chinese medicine.

BioVapor Model Evaluation

EPA Science Inventory

General background on modeling and specifics of modeling vapor intrusion are given. Three classical model applications are described and related to the problem of petroleum vapor intrusion. These indicate the need for model calibration and uncertainty analysis. Evaluation of Bi...

Quality Evaluation of Raw Moutan Cortex Using the AHP and Gray Correlation-TOPSIS Method

PubMed Central

Zhou, Sujuan; Liu, Bo; Meng, Jiang

2017-01-01

Background: Raw Moutan cortex (RMC) is an important Chinese herbal medicine. Comprehensive and objective quality evaluation of Chinese herbal medicine has been one of the most important issues in the modern herbs development. Objective: To evaluate and compare the quality of RMC using the weighted gray correlation- Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method. Materials and Methods: The percentage composition of gallic acid, catechin, oxypaeoniflorin, paeoniflorin, quercetin, benzoylpaeoniflorin, paeonol in different batches of RMC was determined, and then adopting MATLAB programming to construct the gray correlation-TOPSIS assessment model for quality evaluation of RMC. Results: The quality evaluation results of model evaluation and objective evaluation were consistent, reliable, and stable. Conclusion: The model of gray correlation-TOPSIS can be well applied to the quality evaluation of traditional Chinese medicine with multiple components and has broad prospect in application. SUMMARY The experiment tries to construct a model to evaluate the quality of RMC using the weighted gray correlation- Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method. Results show the model is reliable and provide a feasible way in evaluating quality of traditional Chinese medicine with multiple components. PMID:28839384

dETECT: A Model for the Evaluation of Instructional Units for Teaching Computing in Middle School

ERIC Educational Resources Information Center

von Wangenheim, Christiane G.; Petri, Giani; Zibertti, André W.; Borgatto, Adriano F.; Hauck, Jean C. R.; Pacheco, Fernando S.; Filho, Raul Missfeldt

2017-01-01

The objective of this article is to present the development and evaluation of dETECT (Evaluating TEaching CompuTing), a model for the evaluation of the quality of instructional units for teaching computing in middle school based on the students' perception collected through a measurement instrument. The dETECT model was systematically developed…

The Use of AMET & Automated Scripts for Model Evaluation

EPA Science Inventory

Brief overview of EPA’s new CMAQ website to be launched publically in June, 2017. Details on the upcoming release of the Atmospheric Model Evaluation Tool (AMET) and the creation of automated scripts for post-processing and evaluating air quality model data.

Four-dimensional evaluation of regional air quality models

EPA Science Inventory

We present highlights of the results obtained in the third phase of the Air Quality Model Evaluation International Initiative (AQMEII3). Activities in AQMEII3 were focused on evaluating the performance of global, hemispheric and regional modeling systems over Europe and North Ame...

Diagnosing Alzheimer's disease: a systematic review of economic evaluations.

PubMed

Handels, Ron L H; Wolfs, Claire A G; Aalten, Pauline; Joore, Manuela A; Verhey, Frans R J; Severens, Johan L

2014-03-01

The objective of this study is to systematically review the literature on economic evaluations of interventions for the early diagnosis of Alzheimer's disease (AD) and related disorders and to describe their general and methodological characteristics. We focused on the diagnostic aspects of the decision models to assess the applicability of existing decision models for the evaluation of the recently revised diagnostic research criteria for AD. PubMed and the National Institute for Health Research Economic Evaluation database were searched for English-language publications related to economic evaluations on diagnostic technologies. Trial-based economic evaluations were assessed using the Consensus on Health Economic Criteria list. Modeling studies were assessed using the framework for quality assessment of decision-analytic models. The search retrieved 2109 items, from which eight decision-analytic modeling studies and one trial-based economic evaluation met all eligibility criteria. Diversity among the study objective and characteristics was considerable and, despite considerable methodological quality, several flaws were indicated. Recommendations were focused on diagnostic aspects and the applicability of existing models for the evaluation of recently revised diagnostic research criteria for AD. Copyright © 2014 The Alzheimer's Association. Published by Elsevier Inc. All rights reserved.

The Context, Process, and Outcome Evaluation Model for Organisational Health Interventions

PubMed Central

Fridrich, Annemarie; Jenny, Gregor J.; Bauer, Georg F.

2015-01-01

To facilitate evaluation of complex, organisational health interventions (OHIs), this paper aims at developing a context, process, and outcome (CPO) evaluation model. It builds on previous model developments in the field and advances them by clearly defining and relating generic evaluation categories for OHIs. Context is defined as the underlying frame that influences and is influenced by an OHI. It is further differentiated into the omnibus and discrete contexts. Process is differentiated into the implementation process, as the time-limited enactment of the original intervention plan, and the change process of individual and collective dynamics triggered by the implementation process. These processes lead to proximate, intermediate, and distal outcomes, as all results of the change process that are meaningful for various stakeholders. Research questions that might guide the evaluation of an OHI according to the CPO categories and a list of concrete themes/indicators and methods/sources applied within the evaluation of an OHI project at a hospital in Switzerland illustrate the model's applicability in structuring evaluations of complex OHIs. In conclusion, the model supplies a common language and a shared mental model for improving communication between researchers and company members and will improve the comparability and aggregation of evaluation study results. PMID:26557665

The Context, Process, and Outcome Evaluation Model for Organisational Health Interventions.

PubMed

Fridrich, Annemarie; Jenny, Gregor J; Bauer, Georg F

2015-01-01

To facilitate evaluation of complex, organisational health interventions (OHIs), this paper aims at developing a context, process, and outcome (CPO) evaluation model. It builds on previous model developments in the field and advances them by clearly defining and relating generic evaluation categories for OHIs. Context is defined as the underlying frame that influences and is influenced by an OHI. It is further differentiated into the omnibus and discrete contexts. Process is differentiated into the implementation process, as the time-limited enactment of the original intervention plan, and the change process of individual and collective dynamics triggered by the implementation process. These processes lead to proximate, intermediate, and distal outcomes, as all results of the change process that are meaningful for various stakeholders. Research questions that might guide the evaluation of an OHI according to the CPO categories and a list of concrete themes/indicators and methods/sources applied within the evaluation of an OHI project at a hospital in Switzerland illustrate the model's applicability in structuring evaluations of complex OHIs. In conclusion, the model supplies a common language and a shared mental model for improving communication between researchers and company members and will improve the comparability and aggregation of evaluation study results.

Using RAND’s Military Career Model To Evaluate The Impact Of Institutional Requirements On The Air Force Space Officer Career Field

DTIC Science & Technology

2017-01-01

Using RAND’s Military Career Model to Evaluate the Impact of Institutional Requirements on the Air Force Space Officer Career Field...Military Career Model (MCM), a detailed personnel simulation model, to evaluate the impact of changes to IRs on the space officer (13S) career field. The...as well. We recommend that future work evaluate the impact of IRs on multiple career fields to determine which career fields have the most to gain

Comparisons and Evaluation of Hall Thruster Models

DTIC Science & Technology

2002-03-20

COVERED (FROM - TO) 20-04-2001 to 20-04-2002 4. TITLE AND SUBTITLE comparisons and Evaluation of Hall Thruster Models Unclassified 5a. CONTRACT NUMBER...TITLE AND SUBTITLE Comparisons and Evaluation of Hall Thruster Models 5c. PROGRAM ELEMENT NUMBER 5d. PROJECT NUMBER 5d. TASK NUMBER 6. AUTHOR(S...evaluation of Hall thruster models G. J. M. Hagelaar, J. Bareilles, L. Garrigues, and J.-P. Boeuf CPAT, Bâtiment 3R2, Université Paul Sabatier 118 Route

[Evaluation on a fast weight reduction model in vitro].

PubMed

Li, Songtao; Li, Ying; Wen, Ying; Sun, Changhao

2010-03-01

To establish a fast and effective model in vitro for screening weight-reducing drugs and taking preliminary evaluation of the model. Mature adipocytes of SD rat induced by oleic acid were used to establish a obesity model in vitro. Isoprel, genistein, caffeine were selected as positive agents and curcumine as negative agent to evaluate the obesity model. Lipolysis of adipocytes was stimulated significantly by isoprel, genistein and caffeine rather than curcumine. This model could be used efficiently for screening weight-losing drugs.

New model framework and structure and the commonality evaluation model. [concerning unmanned spacecraft projects

NASA Technical Reports Server (NTRS)

1977-01-01

The development of a framework and structure for shuttle era unmanned spacecraft projects and the development of a commonality evaluation model is documented. The methodology developed for model utilization in performing cost trades and comparative evaluations for commonality studies is discussed. The model framework consists of categories of activities associated with the spacecraft system's development process. The model structure describes the physical elements to be treated as separate identifiable entities. Cost estimating relationships for subsystem and program-level components were calculated.

Initial draft of CSE-UCLA evaluation model based on weighted product in order to optimize digital library services in computer college in Bali

NASA Astrophysics Data System (ADS)

Divayana, D. G. H.; Adiarta, A.; Abadi, I. B. G. S.

2018-01-01

The aim of this research was to create initial design of CSE-UCLA evaluation model modified with Weighted Product in evaluating digital library service at Computer College in Bali. The method used in this research was developmental research method and developed by Borg and Gall model design. The results obtained from the research that conducted earlier this month was a rough sketch of Weighted Product based CSE-UCLA evaluation model that the design had been able to provide a general overview of the stages of weighted product based CSE-UCLA evaluation model used in order to optimize the digital library services at the Computer Colleges in Bali.

Evaluation of Biogenic and Fire Emissions in a Global Chemistry Model with NOMADSS, DC3 and SEAC4RS observations

NASA Astrophysics Data System (ADS)

Emmons, L. K.; Wiedinmyer, C.; Park, M.; Kaser, L.; Apel, E. C.; Guenther, A. B.

2014-12-01

Numerous measurements of compounds produced by biogenic and fire emissions were made during several recent field campaigns in the southeast United States, providing a unique data set for emissions and chemical model evaluation. The NCAR Community Atmosphere Model with Chemistry (CAM-chem) is coupled to the Community Land Model (CLM), which includes the biogenic emissions model MEGAN-v2.1, allowing for online calculation of emissions from vegetation for 150 compounds. Simulations of CAM-chem for summers 2012 and 2013 are evaluated with the aircraft and ground-based observations from DC3, NOMADSS and SEAC4RS. Comparison of directly emitted biogenic species, such as isoprene, terpenes, methanol and acetone, are used to evaluate the MEGAN emissions. Evaluation of oxidation products, including methyl vinyl ketone (MVK), methacrolein, formaldehyde, and other oxygenated VOCs are used to test the model chemistry mechanism. In addition, several biomass burning inventories are used in the model, including FINN, QFED, and FLAMBE, and are compared for their impact on atmospheric composition and ozone production, and evaluated with the aircraft observations.

EVALUATION OF ALTERNATIVE GAUSSIAN PLUME DISPERSION MODELING TECHNIQUES IN ESTIMATING SHORT-TERM SULFUR DIOXIDE CONCENTRATIONS

EPA Science Inventory

A routinely applied atmospheric dispersion model was modified to evaluate alternative modeling techniques which allowed for more detailed source data, onsite meteorological data, and several dispersion methodologies. These were evaluated with hourly SO2 concentrations measured at...

Model for the evaluation of drug-dispensing services in primary health care

PubMed Central

Sartor, Vanessa de Bona; de Freitas, Sergio Fernando Torres

2014-01-01

OBJECTIVE To develop a model for evaluating the efficacy of drug-dispensing service in primary health care. METHODS An efficacy criterion was adopted to determine the level of achievement of the service objectives. The evaluation model was developed on the basis of a literature search and discussions with experts. The applicability test of the model was conducted in 15 primary health care units in the city of Florianópolis, state of Santa Catarina, in 2010, and data were recorded in structured and pretested questionnaires. RESULTS The model developed was evaluated using five dimensions of analysis for analysis. The model was suitable for evaluating service efficacy and helped to identify the critical points of each service dimension. CONCLUSIONS Adaptations to the data collection technique may be required to adjust for the reality and needs of each situation. The evaluation of the drug-dispensing service should promote adequate access to medications supplied through the public health system. PMID:25372174

Evaluating models of healthcare delivery using the Model of Care Evaluation Tool (MCET).

PubMed

Hudspeth, Randall S; Vogt, Marjorie; Wysocki, Ken; Pittman, Oralea; Smith, Susan; Cooke, Cindy; Dello Stritto, Rita; Hoyt, Karen Sue; Merritt, T Jeanne

2016-08-01

Our aim was to provide the outcome of a structured Model of Care (MoC) Evaluation Tool (MCET), developed by an FAANP Best-practices Workgroup, that can be used to guide the evaluation of existing MoCs being considered for use in clinical practice. Multiple MoCs are available, but deciding which model of health care delivery to use can be confusing. This five-component tool provides a structured assessment approach to model selection and has universal application. A literature review using CINAHL, PubMed, Ovid, and EBSCO was conducted. The MCET evaluation process includes five sequential components with a feedback loop from component 5 back to component 3 for reevaluation of any refinements. The components are as follows: (1) Background, (2) Selection of an MoC, (3) Implementation, (4) Evaluation, and (5) Sustainability and Future Refinement. This practical resource considers an evidence-based approach to use in determining the best model to implement based on need, stakeholder considerations, and feasibility. ©2015 American Association of Nurse Practitioners.

Evaluation of Fast-Time Wake Vortex Prediction Models

NASA Technical Reports Server (NTRS)

Proctor, Fred H.; Hamilton, David W.

2009-01-01

Current fast-time wake models are reviewed and three basic types are defined. Predictions from several of the fast-time models are compared. Previous statistical evaluations of the APA-Sarpkaya and D2P fast-time models are discussed. Root Mean Square errors between fast-time model predictions and Lidar wake measurements are examined for a 24 hr period at Denver International Airport. Shortcomings in current methodology for evaluating wake errors are also discussed.

A Comprehensive Model for Developing and Evaluating Study Abroad Programs in Counselor Education

ERIC Educational Resources Information Center

Santos, Syntia Dinora

2014-01-01

This paper introduces a model to guide the process of designing and evaluating study abroad programs, addressing particular stages and influential factors. The main purpose of the model is to serve as a basic structure for those who want to develop their own program or evaluate previous cultural immersion experiences. The model is based on the…

Evaluating a Computational Model of Social Causality and Responsibility

DTIC Science & Technology

2006-01-01

Evaluating a Computational Model of Social Causality and Responsibility Wenji Mao University of Southern California Institute for Creative...empirically evaluate a computa- tional model of social causality and responsibility against human social judgments. Results from our experimental...developed a general computational model of social cau- sality and responsibility [10, 11] that formalizes the factors people use in reasoning about

Exploring Secondary Students' Epistemological Features Depending on the Evaluation Levels of the Group Model on Blood Circulation

ERIC Educational Resources Information Center

Lee, Shinyoung; Kim, Heui-Baik

2014-01-01

The purpose of this study is to identify the epistemological features and model qualities depending on model evaluation levels and to explore the reasoning process behind high-level evaluation through small group interaction about blood circulation. Nine groups of three to four students in the eighth grade participated in the modeling practice.…

Statistical modeling for visualization evaluation through data fusion.

PubMed

Chen, Xiaoyu; Jin, Ran

2017-11-01

There is a high demand of data visualization providing insights to users in various applications. However, a consistent, online visualization evaluation method to quantify mental workload or user preference is lacking, which leads to an inefficient visualization and user interface design process. Recently, the advancement of interactive and sensing technologies makes the electroencephalogram (EEG) signals, eye movements as well as visualization logs available in user-centered evaluation. This paper proposes a data fusion model and the application procedure for quantitative and online visualization evaluation. 15 participants joined the study based on three different visualization designs. The results provide a regularized regression model which can accurately predict the user's evaluation of task complexity, and indicate the significance of all three types of sensing data sets for visualization evaluation. This model can be widely applied to data visualization evaluation, and other user-centered designs evaluation and data analysis in human factors and ergonomics. Copyright © 2016 Elsevier Ltd. All rights reserved.

Model Performance Evaluation and Scenario Analysis (MPESA)

EPA Pesticide Factsheets

Model Performance Evaluation and Scenario Analysis (MPESA) assesses the performance with which models predict time series data. The tool was developed Hydrological Simulation Program-Fortran (HSPF) and the Stormwater Management Model (SWMM)

Evaluation of the Williams-type spring wheat model in North Dakota and Minnesota

NASA Technical Reports Server (NTRS)

Leduc, S. (Principal Investigator)

1982-01-01

The Williams type model, developed similarly to previous models of C.V.D. Williams, uses monthly temperature and precipitation data as well as soil and topological variables to predict the yield of the spring wheat crop. The models are statistically developed using the regression technique. Eight model characteristics are examined in the evaluation of the model. Evaluation is at the crop reporting district level, the state level and for the entire region. A ten year bootstrap test was the basis of the statistical evaluation. The accuracy and current indication of modeled yield reliability could show improvement. There is great variability in the bias measured over the districts, but there is a slight overall positive bias. The model estimates for the east central crop reporting district in Minnesota are not accurate. The estimate of yield for 1974 were inaccurate for all of the models.

Practical Findings from Applying the PSD Model for Evaluating Software Design Specifications

NASA Astrophysics Data System (ADS)

Räisänen, Teppo; Lehto, Tuomas; Oinas-Kukkonen, Harri

This paper presents practical findings from applying the PSD model to evaluating the support for persuasive features in software design specifications for a mobile Internet device. On the one hand, our experiences suggest that the PSD model fits relatively well for evaluating design specifications. On the other hand, the model would benefit from more specific heuristics for evaluating each technique to avoid unnecessary subjectivity. Better distinction between the design principles in the social support category would also make the model easier to use. Practitioners who have no theoretical background can apply the PSD model to increase the persuasiveness of the systems they design. The greatest benefit of the PSD model for researchers designing new systems may be achieved when it is applied together with a sound theory, such as the Elaboration Likelihood Model. Using the ELM together with the PSD model, one may increase the chances for attitude change.

Model Performance Evaluation and Scenario Analysis (MPESA) Tutorial

EPA Pesticide Factsheets

The model performance evaluation consists of metrics and model diagnostics. These metrics provides modelers with statistical goodness-of-fit measures that capture magnitude only, sequence only, and combined magnitude and sequence errors.

Design and Establishment of Quality Model of Fundamental Geographic Information Database

NASA Astrophysics Data System (ADS)

Ma, W.; Zhang, J.; Zhao, Y.; Zhang, P.; Dang, Y.; Zhao, T.

2018-04-01

In order to make the quality evaluation for the Fundamental Geographic Information Databases(FGIDB) more comprehensive, objective and accurate, this paper studies and establishes a quality model of FGIDB, which formed by the standardization of database construction and quality control, the conformity of data set quality and the functionality of database management system, and also designs the overall principles, contents and methods of the quality evaluation for FGIDB, providing the basis and reference for carry out quality control and quality evaluation for FGIDB. This paper designs the quality elements, evaluation items and properties of the Fundamental Geographic Information Database gradually based on the quality model framework. Connected organically, these quality elements and evaluation items constitute the quality model of the Fundamental Geographic Information Database. This model is the foundation for the quality demand stipulation and quality evaluation of the Fundamental Geographic Information Database, and is of great significance on the quality assurance in the design and development stage, the demand formulation in the testing evaluation stage, and the standard system construction for quality evaluation technology of the Fundamental Geographic Information Database.

Fuzzy Risk Evaluation in Failure Mode and Effects Analysis Using a D Numbers Based Multi-Sensor Information Fusion Method.

PubMed

Deng, Xinyang; Jiang, Wen

2017-09-12

Failure mode and effect analysis (FMEA) is a useful tool to define, identify, and eliminate potential failures or errors so as to improve the reliability of systems, designs, and products. Risk evaluation is an important issue in FMEA to determine the risk priorities of failure modes. There are some shortcomings in the traditional risk priority number (RPN) approach for risk evaluation in FMEA, and fuzzy risk evaluation has become an important research direction that attracts increasing attention. In this paper, the fuzzy risk evaluation in FMEA is studied from a perspective of multi-sensor information fusion. By considering the non-exclusiveness between the evaluations of fuzzy linguistic variables to failure modes, a novel model called D numbers is used to model the non-exclusive fuzzy evaluations. A D numbers based multi-sensor information fusion method is proposed to establish a new model for fuzzy risk evaluation in FMEA. An illustrative example is provided and examined using the proposed model and other existing method to show the effectiveness of the proposed model.

Fuzzy Risk Evaluation in Failure Mode and Effects Analysis Using a D Numbers Based Multi-Sensor Information Fusion Method

PubMed Central

Deng, Xinyang

2017-01-01

Failure mode and effect analysis (FMEA) is a useful tool to define, identify, and eliminate potential failures or errors so as to improve the reliability of systems, designs, and products. Risk evaluation is an important issue in FMEA to determine the risk priorities of failure modes. There are some shortcomings in the traditional risk priority number (RPN) approach for risk evaluation in FMEA, and fuzzy risk evaluation has become an important research direction that attracts increasing attention. In this paper, the fuzzy risk evaluation in FMEA is studied from a perspective of multi-sensor information fusion. By considering the non-exclusiveness between the evaluations of fuzzy linguistic variables to failure modes, a novel model called D numbers is used to model the non-exclusive fuzzy evaluations. A D numbers based multi-sensor information fusion method is proposed to establish a new model for fuzzy risk evaluation in FMEA. An illustrative example is provided and examined using the proposed model and other existing method to show the effectiveness of the proposed model. PMID:28895905

Modeling the dynamics of evaluation: a multilevel neural network implementation of the iterative reprocessing model.

PubMed

Ehret, Phillip J; Monroe, Brian M; Read, Stephen J

2015-05-01

We present a neural network implementation of central components of the iterative reprocessing (IR) model. The IR model argues that the evaluation of social stimuli (attitudes, stereotypes) is the result of the IR of stimuli in a hierarchy of neural systems: The evaluation of social stimuli develops and changes over processing. The network has a multilevel, bidirectional feedback evaluation system that integrates initial perceptual processing and later developing semantic processing. The network processes stimuli (e.g., an individual's appearance) over repeated iterations, with increasingly higher levels of semantic processing over time. As a result, the network's evaluations of stimuli evolve. We discuss the implications of the network for a number of different issues involved in attitudes and social evaluation. The success of the network supports the IR model framework and provides new insights into attitude theory. © 2014 by the Society for Personality and Social Psychology, Inc.

Computational Evaluation of the Traceback Method

ERIC Educational Resources Information Center

Kol, Sheli; Nir, Bracha; Wintner, Shuly

2014-01-01

Several models of language acquisition have emerged in recent years that rely on computational algorithms for simulation and evaluation. Computational models are formal and precise, and can thus provide mathematically well-motivated insights into the process of language acquisition. Such models are amenable to robust computational evaluation,…

77 FR 27814 - Model Safety Evaluation for Plant-Specific Adoption of Technical Specifications Task Force...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-05-11

... NUCLEAR REGULATORY COMMISSION [Project No. 753; NRC-2012-0019] Model Safety Evaluation for Plant... Regulatory Commission (NRC) is announcing the availability of the model safety evaluation (SE) for plant... the Improved Standard Technical Specification (ISTS), NUREG-1431, ``Standard Technical Specifications...

Dynamic Evaluation of Long-Term Air Quality Model Simulations Over the Northeastern U.S.

EPA Science Inventory

Dynamic model evaluation assesses a modeling system's ability to reproduce changes in air quality induced by changes in meteorology and/or emissions. In this paper, we illustrate various approaches to dynamic mode evaluation utilizing 18 years of air quality simulations perform...

“Overview and Evaluation of AQMEII Phase 2 Coupled Simulations over North America”

EPA Science Inventory

This presentation provides an overview of the second phase of the Air Quality Model Evaluation International Initative (AQMEII). Activities in this phase are focused on the application and evaluation of coupled meteorology-chemistry models to assess how well these models can simu...

Principal Evaluation in Indiana: Practitioners' Perceptions of a New Statewide Model

ERIC Educational Resources Information Center

Andrews, Kelly A.; Boyland, Lori G.; Quick, Marilynn M.

2016-01-01

This study examines administrators' perspectives of a state-developed principal evaluation model adopted by a majority of Indiana school districts after legislation mandated policy reform in educator evaluation. Feedback was gathered from public school superintendents (the evaluators) and principals (those being evaluated), with 364 participants.…

Beyond Evaluation: A Model for Cooperative Evaluation of Internet Resources.

ERIC Educational Resources Information Center

Kirkwood, Hal P., Jr.

1998-01-01

Presents a status report on Web site evaluation efforts, listing dead, merged, new review, Yahoo! wannabes, subject-specific review, former librarian-managed, and librarian-managed review sites; discusses how sites are evaluated; describes and demonstrates (reviewing company directories) the Marr/Kirkwood evaluation model; and provides an…

Multi-objective optimization for generating a weighted multi-model ensemble

NASA Astrophysics Data System (ADS)

Lee, H.

2017-12-01

Many studies have demonstrated that multi-model ensembles generally show better skill than each ensemble member. When generating weighted multi-model ensembles, the first step is measuring the performance of individual model simulations using observations. There is a consensus on the assignment of weighting factors based on a single evaluation metric. When considering only one evaluation metric, the weighting factor for each model is proportional to a performance score or inversely proportional to an error for the model. While this conventional approach can provide appropriate combinations of multiple models, the approach confronts a big challenge when there are multiple metrics under consideration. When considering multiple evaluation metrics, it is obvious that a simple averaging of multiple performance scores or model ranks does not address the trade-off problem between conflicting metrics. So far, there seems to be no best method to generate weighted multi-model ensembles based on multiple performance metrics. The current study applies the multi-objective optimization, a mathematical process that provides a set of optimal trade-off solutions based on a range of evaluation metrics, to combining multiple performance metrics for the global climate models and their dynamically downscaled regional climate simulations over North America and generating a weighted multi-model ensemble. NASA satellite data and the Regional Climate Model Evaluation System (RCMES) software toolkit are used for assessment of the climate simulations. Overall, the performance of each model differs markedly with strong seasonal dependence. Because of the considerable variability across the climate simulations, it is important to evaluate models systematically and make future projections by assigning optimized weighting factors to the models with relatively good performance. Our results indicate that the optimally weighted multi-model ensemble always shows better performance than an arithmetic ensemble mean and may provide reliable future projections.

What Counts is not Falling … but Landing: Strategic Analysis: An Adapted Model for Implementation Evaluation.

PubMed

Brousselle, Astrid

2004-04-01

Implementation evaluations, also called process evaluations, involve studying the development of programmes, and identifying and understanding their strengths and weaknesses. Undertaking an implementation evaluation offers insights into evaluation objectives, but does not help the researcher develop a research strategy. During the implementation analysis of the UNAIDS drug access initiative in Chile, the strategic analysis model developed by Crozier and Friedberg was used. However, a major incompatibility was noted between the procedure put forward by Crozier and Friedberg and the specific characteristics of the programme being evaluated. In this article, an adapted strategic analysis model for programme evaluation is proposed.

Evaluating Air-Quality Models: Review and Outlook.

NASA Astrophysics Data System (ADS)

Weil, J. C.; Sykes, R. I.; Venkatram, A.

1992-10-01

Over the past decade, much attention has been devoted to the evaluation of air-quality models with emphasis on model performance in predicting the high concentrations that are important in air-quality regulations. This paper stems from our belief that this practice needs to be expanded to 1) evaluate model physics and 2) deal with the large natural or stochastic variability in concentration. The variability is represented by the root-mean- square fluctuating concentration (c about the mean concentration (C) over an ensemble-a given set of meteorological, source, etc. conditions. Most air-quality models used in applications predict C, whereas observations are individual realizations drawn from an ensemble. For cC large residuals exist between predicted and observed concentrations, which confuse model evaluations.This paper addresses ways of evaluating model physics in light of the large c the focus is on elevated point-source models. Evaluation of model physics requires the separation of the mean model error-the difference between the predicted and observed C-from the natural variability. A residual analysis is shown to be an elective way of doing this. Several examples demonstrate the usefulness of residuals as well as correlation analyses and laboratory data in judging model physics.In general, c models and predictions of the probability distribution of the fluctuating concentration (c), (c, are in the developmental stage, with laboratory data playing an important role. Laboratory data from point-source plumes in a convection tank show that (c approximates a self-similar distribution along the plume center plane, a useful result in a residual analysis. At pmsent,there is one model-ARAP-that predicts C, c, and (c for point-source plumes. This model is more computationally demanding than other dispersion models (for C only) and must be demonstrated as a practical tool. However, it predicts an important quantity for applications- the uncertainty in the very high and infrequent concentrations. The uncertainty is large and is needed in evaluating operational performance and in predicting the attainment of air-quality standards.

Development of the evaluation instrument use CIPP on the implementation of project assessment topic optik

NASA Astrophysics Data System (ADS)

Asfaroh, Jati Aurum; Rosana, Dadan; Supahar

2017-08-01

This research aims to develop an evaluation instrument models CIPP valid and reliable as well as determine the feasibility and practicality of an evaluation instrument models CIPP. An evaluation instrument models CIPP to evaluate the implementation of the project assessment topic optik to measure problem-solving skills of junior high school class VIII in the Yogyakarta region. This research is a model of development that uses 4-D. Subject of product trials are students in class VIII SMP N 1 Galur and SMP N 1 Sleman. Data collection techniques in this research using non-test techniques include interviews, questionnaires and observations. Validity in this research was analyzed using V'Aikens. Reliability analyzed using ICC. This research uses 7 raters are derived from two lecturers expert (expert judgment), two practitioners (science teacher) and three colleagues. The results of this research is the evaluation's instrument model of CIPP is used to evaluate the implementation of the implementation of the project assessment instruments. The validity result of evaluation instrument have V'Aikens values between 0.86 to 1, which means a valid and 0.836 reliability values into categories so well that it has been worth used as an evaluation instrument.

Milestone-specific, Observed data points for evaluating levels of performance (MODEL) assessment strategy for anesthesiology residency programs.

PubMed

Nagy, Christopher J; Fitzgerald, Brian M; Kraus, Gregory P

2014-01-01

Anesthesiology residency programs will be expected to have Milestones-based evaluation systems in place by July 2014 as part of the Next Accreditation System. The San Antonio Uniformed Services Health Education Consortium (SAUSHEC) anesthesiology residency program developed and implemented a Milestones-based feedback and evaluation system a year ahead of schedule. It has been named the Milestone-specific, Observed Data points for Evaluating Levels of performance (MODEL) assessment strategy. The "MODEL Menu" and the "MODEL Blueprint" are tools that other anesthesiology residency programs can use in developing their own Milestones-based feedback and evaluation systems prior to ACGME-required implementation. Data from our early experience with the streamlined MODEL blueprint assessment strategy showed substantially improved faculty compliance with reporting requirements. The MODEL assessment strategy provides programs with a workable assessment method for residents, and important Milestones data points to programs for ACGME reporting.

Evaluation of Model Recognition for Grammar-Based Automatic 3d Building Model Reconstruction

NASA Astrophysics Data System (ADS)

Yu, Qian; Helmholz, Petra; Belton, David

2016-06-01

In recent years, 3D city models are in high demand by many public and private organisations, and the steadily growing capacity in both quality and quantity are increasing demand. The quality evaluation of these 3D models is a relevant issue both from the scientific and practical points of view. In this paper, we present a method for the quality evaluation of 3D building models which are reconstructed automatically from terrestrial laser scanning (TLS) data based on an attributed building grammar. The entire evaluation process has been performed in all the three dimensions in terms of completeness and correctness of the reconstruction. Six quality measures are introduced to apply on four datasets of reconstructed building models in order to describe the quality of the automatic reconstruction, and also are assessed on their validity from the evaluation point of view.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mundaca, Luis; Neij, Lena; Worrell, Ernst

The growing complexities of energy systems, environmental problems and technology markets are driving and testing most energy-economy models to their limits. To further advance bottom-up models from a multidisciplinary energy efficiency policy evaluation perspective, we review and critically analyse bottom-up energy-economy models and corresponding evaluation studies on energy efficiency policies to induce technological change. We use the household sector as a case study. Our analysis focuses on decision frameworks for technology choice, type of evaluation being carried out, treatment of market and behavioural failures, evaluated policy instruments, and key determinants used to mimic policy instruments. Although the review confirms criticismmore » related to energy-economy models (e.g. unrealistic representation of decision-making by consumers when choosing technologies), they provide valuable guidance for policy evaluation related to energy efficiency. Different areas to further advance models remain open, particularly related to modelling issues, techno-economic and environmental aspects, behavioural determinants, and policy considerations.« less

Evaluating Models of Human Performance: Safety-Critical Systems Applications

NASA Technical Reports Server (NTRS)

Feary, Michael S.

2012-01-01

This presentation is part of panel discussion on Evaluating Models of Human Performance. The purpose of this panel is to discuss the increasing use of models in the world today and specifically focus on how to describe and evaluate models of human performance. My presentation will focus on discussions of generating distributions of performance, and the evaluation of different strategies for humans performing tasks with mixed initiative (Human-Automation) systems. I will also discuss issues with how to provide Human Performance modeling data to support decisions on acceptability and tradeoffs in the design of safety critical systems. I will conclude with challenges for the future.

Neutral models as a way to evaluate the Sea Level Affecting Marshes Model (SLAMM)

EPA Science Inventory

A commonly used landscape model to simulate wetland change – the Sea Level Affecting Marshes Model(SLAMM) – has rarely been explicitly assessed for its prediction accuracy. Here, we evaluated this model using recently proposed neutral models – including the random constraint matc...

Comprehensive system models: Strategies for evaluation

NASA Technical Reports Server (NTRS)

Field, Christopher; Kutzbach, John E.; Ramanathan, V.; Maccracken, Michael C.

1992-01-01

The task of evaluating comprehensive earth system models is vast involving validations of every model component at every scale of organization, as well as tests of all the individual linkages. Even the most detailed evaluation of each of the component processes and the individual links among them should not, however, engender confidence in the performance of the whole. The integrated earth system is so rich with complex feedback loops, often involving components of the atmosphere, oceans, biosphere, and cryosphere, that it is certain to exhibit emergent properties very difficult to predict from the perspective of a narrow focus on any individual component of the system. Therefore, a substantial share of the task of evaluating comprehensive earth system models must reside at the level of whole system evaluations. Since complete, integrated atmosphere/ ocean/ biosphere/ hydrology models are not yet operational, questions of evaluation must be addressed at the level of the kinds of earth system processes that the models should be competent to simulate, rather than at the level of specific performance criteria. Here, we have tried to identify examples of earth system processes that are difficult to simulate with existing models and that involve a rich enough suite of feedbacks that they are unlikely to be satisfactorily described by highly simplified or toy models. Our purpose is not to specify a checklist of evaluation criteria but to introduce characteristics of the earth system that may present useful opportunities for model testing and, of course, improvement.

Evaluation of Stratospheric Transport in New 3D Models Using the Global Modeling Initiative Grading Criteria

NASA Technical Reports Server (NTRS)

Strahan, Susan E.; Douglass, Anne R.; Einaudi, Franco (Technical Monitor)

2001-01-01

The Global Modeling Initiative (GMI) Team developed objective criteria for model evaluation in order to identify the best representation of the stratosphere. This work created a method to quantitatively and objectively discriminate between different models. In the original GMI study, 3 different meteorological data sets were used to run an offline chemistry and transport model (CTM). Observationally-based grading criteria were derived and applied to these simulations and various aspects of stratospheric transport were evaluated; grades were assigned. Here we report on the application of the GMI evaluation criteria to CTM simulations integrated with a new assimilated wind data set and a new general circulation model (GCM) wind data set. The Finite Volume Community Climate Model (FV-CCM) is a new GCM developed at Goddard which uses the NCAR CCM physics and the Lin and Rood advection scheme. The FV-Data Assimilation System (FV-DAS) is a new data assimilation system which uses the FV-CCM as its core model. One year CTM simulations of 2.5 degrees longitude by 2 degrees latitude resolution were run for each wind data set. We present the evaluation of temperature and annual transport cycles in the lower and middle stratosphere in the two new CTM simulations. We include an evaluation of high latitude transport which was not part of the original GMI criteria. Grades for the new simulations will be compared with those assigned during the original GMT evaluations and areas of improvement will be identified.

A new harvest operation cost model to evaluate forest harvest layout alternatives

Treesearch

Mark M. Clark; Russell D. Meller; Timothy P. McDonald; Chao Chi Ting

1997-01-01

The authors develop a new model for harvest operation costs that can be used to evaluate stands for potential harvest. The model is based on felling, extraction, and access costs, and is unique in its consideration of the interaction between harvest area shapes and access roads. The scientists illustrate the model and evaluate the impact of stand size, volume, and road...

Performance of the SEAPROG prognosis variant of the forest vegetation simulator.

Treesearch

Michael H. McClellan; Frances E. Biles

2003-01-01

This paper reports the first phase of a recent effort to evaluate the performance and use of the FVS-SEAPROG vegetation growth model. In this paper, we present our evaluation of SEAPROG’s performance in modeling the growth of even-aged stands regenerated by clearcutting, windthrow, or fire. We evaluated the model by comparing model predictions to observed values from...

Field Evaluation of the Pedostructure-Based Model (Kamel®)

USDA-ARS?s Scientific Manuscript database

This study involves a field evaluation of the pedostructure-based model Kamel and comparisons between Kamel and the Hydrus-1D model for predicting profile soil moisture. This paper also presents a sensitivity analysis of Kamel with an evaluation field site used as the base scenario. The field site u...

A Critique of Kirkpatrick's Evaluation Model

ERIC Educational Resources Information Center

Reio, Thomas G., Jr.; Rocco, Tonette S.; Smith, Douglas H.; Chang, Elegance

2017-01-01

Donald Kirkpatrick published a series of articles originating from his doctoral dissertation in the late 1950s describing a four-level training evaluation model. From its beginning, it was easily understood and became one of the most influential evaluation models impacting the field of HRD. While well received and popular, the Kirkpatrick model…

Rhode Island Model Evaluation & Support System: Building Administrator. Edition III

ERIC Educational Resources Information Center

Rhode Island Department of Education, 2015

2015-01-01

Rhode Island educators believe that implementing a fair, accurate, and meaningful educator evaluation and support system will help improve teaching, learning, and school leadership. The primary purpose of the Rhode Island Model Building Administrator Evaluation and Support System (Rhode Island Model) is to help all building administrators improve.…

A Model for Evaluating Development Programs. Miscellaneous Report.

ERIC Educational Resources Information Center

Burton, John E., Jr.; Rogers, David L.

Taking the position that the Classical Experimental Evaluation (CEE) Model does not do justice to the process of acquiring information necessary for decision making re planning, programming, implementing, and recycling program activities, this paper presents the Inductive, System-Process (ISP) evaluation model as an alternative to be used in…

Conceptual design and feasibility evaluation model of a 10 to the 8th power bit oligatomic mass memory. Volume 2: Feasibility evaluation model

NASA Technical Reports Server (NTRS)

Horst, R. L.; Nordstrom, M. J.

1972-01-01

The partially populated oligatomic mass memory feasibility model is described and evaluated. A system was desired to verify the feasibility of the oligatomic (mirror) memory approach as applicable to large scale solid state mass memories.

A Survey of Model Evaluation Approaches with a Tutorial on Hierarchical Bayesian Methods

ERIC Educational Resources Information Center

Shiffrin, Richard M.; Lee, Michael D.; Kim, Woojae; Wagenmakers, Eric-Jan

2008-01-01

This article reviews current methods for evaluating models in the cognitive sciences, including theoretically based approaches, such as Bayes factors and minimum description length measures; simulation approaches, including model mimicry evaluations; and practical approaches, such as validation and generalization measures. This article argues…

Annual Application and Evaluation of the Online Coupled WRF‐CMAQ System over North America under AQMEII Phase 2

EPA Science Inventory

We present an application of the online coupled WRF-CMAQ modeling system to two annual simulations over North America performed under Phase 2 of the Air Quality Model Evaluation International Initiative (AQMEII). Operational evaluation shows that model performance is comparable t...

A model of evaluation planning, implementation and management: Toward a ?culture of information? within organizations

NASA Astrophysics Data System (ADS)

Bhola, H. S.

1992-03-01

The argument underlying the ongoing "paradigm shift" from logical positivism to constructionism is briefly laid out. A model of evaluation planning, implementation and management (called the P-I-M Model, for short) is then presented that assumes a complementarity between the two paradigms. The model further implies that for effective decision-making within human organizations, both "evaluative data" and "descriptive data" are needed. "Evaluative data" generated by evaluation studies must, therefore, be undergirded by an appropriate management information system (MIS) that can generate "descriptive data", concurrently with the process of program implementation. The P-I-M Model, if fully actualized, will enable human organizations to become vibrant "cultures of information" where "informed" decision-making becomes a shared norm among all stakeholders.

The Strengths and Limitations of Satellite Data for Evaluating Tropospheric Processes in Chemistry-Climate Models

NASA Technical Reports Server (NTRS)

Duncan, Bryan

2012-01-01

There is now a wealth of satellite data products available with which to evaluate a model fs simulation of tropospheric composition and other model processes. All of these data products have their strengths and limitations that need to be considered for this purpose. For example, uncertainties are introduced into a data product when 1) converting a slant column to a vertical column and 2) estimating the amount of a total column of a trace gas (e.g., ozone, nitrogen dioxide) that resides in the troposphere. Oftentimes, these uncertainties are not well quantified and the satellite data products are not well evaluated against in situ observations. However, these limitations do not preclude us from using these data products to evaluate our model processes if we understand these strengths and limitations when developing diagnostics. I will show several examples of how satellite data products are being used to evaluate particular model processes with a focus on the strengths and limitations of these data products. In addition, I will introduce the goals of a newly formed team to address issues on the topic of "satellite data for improved model evaluation and process studies" that is established in support of the IGAC/SPARC Global Chemistry ]Climate Modeling and Evaluation Workshop.

Development of evaluation models of manpower needs for dismantling the dry conversion process-related equipment in uranium refining and conversion plant (URCP)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sari Izumo; Hideo Usui; Mitsuo Tachibana

Evaluation models for determining the manpower needs for dismantling various types of equipment in uranium refining and conversion plant (URCP) have been developed. The models are widely applicable to other uranium handling facilities. Additionally, a simplified model was developed for easily and accurately calculating the manpower needs for dismantling dry conversion process-related equipment (DP equipment). It is important to evaluate beforehand project management data such as manpower needs to prepare an optimized decommissioning plan and implement effective dismantling activity. The Japan Atomic Energy Agency (JAEA) has developed the project management data evaluation system for dismantling activities (PRODIA code), which canmore » generate project management data using evaluation models. For preparing an optimized decommissioning plan, these evaluation models should be established based on the type of nuclear facility and actual dismantling data. In URCP, the dry conversion process of reprocessed uranium and others was operated until 1999, and the equipment related to the main process was dismantled from 2008 to 2011. Actual data such as manpower for dismantling were collected during the dismantling activities, and evaluation models were developed using the collected actual data on the basis of equipment classification considering the characteristics of uranium handling facility. (authors)« less

The role of self-improvement and self-evaluation motives in social comparisons with idealised female bodies in the media.

PubMed

Halliwell, Emma; Dittmar, Helga

2005-09-01

This study investigates the effect of social comparisons with media models on women's body image based on either self-evaluation or self-improvement motives. Ninety-eight women, for whom appearance was a relevant comparison dimension, viewed advertisements that did, or did not, feature idealised models, after being prompted to engage in self-evaluation or self-improvement comparisons. The results indicate that, when focusing on self-evaluation, comparisons with thin models are associated with higher body-focused anxiety than viewing no model advertisements. In contrast, when focusing on self-improvement, comparisons with thin models are not associated with higher body-focused anxiety than viewing no models. Furthermore, women's general tendency to engage in social comparisons moderated the effects of self-evaluative comparisons with models, so that women who did not habitually engage in social comparisons were most strongly affected. It is suggested that motive for social comparison may explain previous inconsistencies in the experimental exposure literature and warrants more careful attention in future research.

Evaluation of a hydrological model based on Bidirectional Reach (BReach)

NASA Astrophysics Data System (ADS)

Van Eerdenbrugh, Katrien; Van Hoey, Stijn; Verhoest, Niko E. C.

2016-04-01

Evaluation and discrimination of model structures is crucial to ensure an appropriate use of hydrological models. When evaluating model results by aggregating their quality in (a subset of) individual observations, overall results of this analysis sometimes conceal important detailed information about model structural deficiencies. Analyzing model results within their local (time) context can uncover this detailed information. In this research, a methodology called Bidirectional Reach (BReach) is proposed to evaluate and analyze results of a hydrological model by assessing the maximum left and right reach in each observation point that is used for model evaluation. These maximum reaches express the capability of the model to describe a subset of the evaluation data both in the direction of the previous (left) and of the following data (right). This capability is evaluated on two levels. First, on the level of individual observations, the combination of a parameter set and an observation is classified as non-acceptable if the deviation between the accompanying model result and the measurement exceeds observational uncertainty. Second, the behavior in a sequence of observations is evaluated by means of a tolerance degree. This tolerance degree expresses the condition for satisfactory model behavior in a data series and is defined by the percentage of observations within this series that can have non-acceptable model results. Based on both criteria, the maximum left and right reaches of a model in an observation represent the data points in the direction of the previous respectively the following observations beyond which none of the sampled parameter sets both are satisfactory and result in an acceptable deviation. After assessing these reaches for a variety of tolerance degrees, results can be plotted in a combined BReach plot that show temporal changes in the behavior of model results. The methodology is applied on a Probability Distributed Model (PDM) of the river Grote Nete upstream of Geel-Zammel with 1 106 randomly sampled parameter sets for three separate years. Acceptable model results must fit in the 95 % uncertainty bounds of observed discharges and tolerance degrees of 0 %, 5 %, 10 %, 20 % and 40 % are applied. An evaluation of BReach results with regard to other variables, such as the magnitude and the rate of change of the observed discharges enables to detect recurring patterns in model errors. This results in an augmented understanding of the model's structural deficiencies, revealing the incapability of the PDM model to simulate both high and low flow simulations with a single parameter set for this catchment. As the methodology can be applied for different hydrological model structures, it is a useful tool to gain understanding of the difference in behavior of competing models.

A Regional Climate Model Evaluation System based on Satellite and other Observations

NASA Astrophysics Data System (ADS)

Lean, P.; Kim, J.; Waliser, D. E.; Hall, A. D.; Mattmann, C. A.; Granger, S. L.; Case, K.; Goodale, C.; Hart, A.; Zimdars, P.; Guan, B.; Molotch, N. P.; Kaki, S.

2010-12-01

Regional climate models are a fundamental tool needed for downscaling global climate simulations and projections, such as those contributing to the Coupled Model Intercomparison Projects (CMIPs) that form the basis of the IPCC Assessment Reports. The regional modeling process provides the means to accommodate higher resolution and a greater complexity of Earth System processes. Evaluation of both the global and regional climate models against observations is essential to identify model weaknesses and to direct future model development efforts focused on reducing the uncertainty associated with climate projections. However, the lack of reliable observational data and the lack of formal tools are among the serious limitations to addressing these objectives. Recent satellite observations are particularly useful as they provide a wealth of information on many different aspects of the climate system, but due to their large volume and the difficulties associated with accessing and using the data, these datasets have been generally underutilized in model evaluation studies. Recognizing this problem, NASA JPL / UCLA is developing a model evaluation system to help make satellite observations, in conjunction with in-situ, assimilated, and reanalysis datasets, more readily accessible to the modeling community. The system includes a central database to store multiple datasets in a common format and codes for calculating predefined statistical metrics to assess model performance. This allows the time taken to compare model simulations with satellite observations to be reduced from weeks to days. Early results from the use this new model evaluation system for evaluating regional climate simulations over California/western US regions will be presented.

The Air Quality Model Evaluation International Initiative ...

EPA Pesticide Factsheets

This presentation provides an overview of the Air Quality Model Evaluation International Initiative (AQMEII). It contains a synopsis of the three phases of AQMEII, including objectives, logistics, and timelines. It also provides a number of examples of analyses conducted through AQMEII with a particular focus on past and future analyses of deposition. The National Exposure Research Laboratory (NERL) Computational Exposure Division (CED) develops and evaluates data, decision-support tools, and models to be applied to media-specific or receptor-specific problem areas. CED uses modeling-based approaches to characterize exposures, evaluate fate and transport, and support environmental diagnostics/forensics with input from multiple data sources. It also develops media- and receptor-specific models, process models, and decision support tools for use both within and outside of EPA.

A Regional Climate Model Evaluation System based on contemporary Satellite and other Observations for Assessing Regional Climate Model Fidelity

NASA Astrophysics Data System (ADS)

Waliser, D. E.; Kim, J.; Mattman, C.; Goodale, C.; Hart, A.; Zimdars, P.; Lean, P.

2011-12-01

Evaluation of climate models against observations is an essential part of assessing the impact of climate variations and change on regionally important sectors and improving climate models. Regional climate models (RCMs) are of a particular concern. RCMs provide fine-scale climate needed by the assessment community via downscaling global climate model projections such as those contributing to the Coupled Model Intercomparison Project (CMIP) that form one aspect of the quantitative basis of the IPCC Assessment Reports. The lack of reliable fine-resolution observational data and formal tools and metrics has represented a challenge in evaluating RCMs. Recent satellite observations are particularly useful as they provide a wealth of information and constraints on many different processes within the climate system. Due to their large volume and the difficulties associated with accessing and using contemporary observations, however, these datasets have been generally underutilized in model evaluation studies. Recognizing this problem, NASA JPL and UCLA have developed the Regional Climate Model Evaluation System (RCMES) to help make satellite observations, in conjunction with in-situ and reanalysis datasets, more readily accessible to the regional modeling community. The system includes a central database (Regional Climate Model Evaluation Database: RCMED) to store multiple datasets in a common format and codes for calculating and plotting statistical metrics to assess model performance (Regional Climate Model Evaluation Tool: RCMET). This allows the time taken to compare model data with satellite observations to be reduced from weeks to days. RCMES is a component of the recent ExArch project, an international effort for facilitating the archive and access of massive amounts data for users using cloud-based infrastructure, in this case as applied to the study of climate and climate change. This presentation will describe RCMES and demonstrate its utility using examples from RCMs applied to the southwest US as well as to Africa based on output from the CORDEX activity. Application of RCMES to the evaluation of multi-RCM hindcast for CORDEX-Africa will be presented in a companion paper in A41.

Decision-relevant evaluation of climate models: A case study of chill hours in California

NASA Astrophysics Data System (ADS)

Jagannathan, K. A.; Jones, A. D.; Kerr, A. C.

2017-12-01

The past decade has seen a proliferation of different climate datasets with over 60 climate models currently in use. Comparative evaluation and validation of models can assist practitioners chose the most appropriate models for adaptation planning. However, such assessments are usually conducted for `climate metrics' such as seasonal temperature, while sectoral decisions are often based on `decision-relevant outcome metrics' such as growing degree days or chill hours. Since climate models predict different metrics with varying skill, the goal of this research is to conduct a bottom-up evaluation of model skill for `outcome-based' metrics. Using chill hours (number of hours in winter months where temperature is lesser than 45 deg F) in Fresno, CA as a case, we assess how well different GCMs predict the historical mean and slope of chill hours, and whether and to what extent projections differ based on model selection. We then compare our results with other climate-based evaluations of the region, to identify similarities and differences. For the model skill evaluation, historically observed chill hours were compared with simulations from 27 GCMs (and multiple ensembles). Model skill scores were generated based on a statistical hypothesis test of the comparative assessment. Future projections from RCP 8.5 runs were evaluated, and a simple bias correction was also conducted. Our analysis indicates that model skill in predicting chill hour slope is dependent on its skill in predicting mean chill hours, which results from the non-linear nature of the chill metric. However, there was no clear relationship between the models that performed well for the chill hour metric and those that performed well in other temperature-based evaluations (such winter minimum temperature or diurnal temperature range). Further, contrary to conclusions from other studies, we also found that the multi-model mean or large ensemble mean results may not always be most appropriate for this outcome metric. Our assessment sheds light on key differences between global versus local skill, and broad versus specific skill of climate models, highlighting that decision-relevant model evaluation may be crucial for providing practitioners with the best available climate information for their specific needs.

Constructing service-oriented architecture adoption maturity matrix using Kano model

NASA Astrophysics Data System (ADS)

Hamzah, Mohd Hamdi Irwan; Baharom, Fauziah; Mohd, Haslina

2017-10-01

Commonly, organizations adopted Service-Oriented Architecture (SOA) because it can provide a flexible reconfiguration and can reduce the development time and cost. In order to guide the SOA adoption, previous industry and academia have constructed SOA maturity model. However, there is a limited number of works on how to construct the matrix in the previous SOA maturity model. Therefore, this study is going to provide a method that can be used in order to construct the matrix in the SOA maturity model. This study adapts Kano Model to construct the cross evaluation matrix focused on SOA adoption IT and business benefits. This study found that Kano Model can provide a suitable and appropriate method for constructing the cross evaluation matrix in SOA maturity model. Kano model also can be used to plot, organize and better represent the evaluation dimension for evaluating the SOA adoption.

Three new models for evaluation of standard involute spur gear mesh stiffness

NASA Astrophysics Data System (ADS)

Liang, Xihui; Zhang, Hongsheng; Zuo, Ming J.; Qin, Yong

2018-02-01

Time-varying mesh stiffness is one of the main internal excitation sources of gear dynamics. Accurate evaluation of gear mesh stiffness is crucial for gear dynamic analysis. This study is devoted to developing new models for spur gear mesh stiffness evaluation. Three models are proposed. The proposed model 1 can give very accurate mesh stiffness result but the gear bore surface must be assumed to be rigid. Enlighted by the proposed model 1, our research discovers that the angular deflection pattern of the gear bore surface of a pair of meshing gears under a constant torque basically follows a cosine curve. Based on this finding, two other models are proposed. The proposed model 2 evaluates gear mesh stiffness by using angular deflections at different circumferential angles of an end surface circle of the gear bore. The proposed model 3 requires using only the angular deflection at an arbitrary circumferential angle of an end surface circle of the gear bore but this model can only be used for a gear with the same tooth profile among all teeth. The proposed models are accurate in gear mesh stiffness evaluation and easy to use. Finite element analysis is used to validate the accuracy of the proposed models.

Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods

DOE PAGES

Liu, Peigui; Elshall, Ahmed S.; Ye, Ming; ...

2016-02-05

Evaluating marginal likelihood is the most critical and computationally expensive task, when conducting Bayesian model averaging to quantify parametric and model uncertainties. The evaluation is commonly done by using Laplace approximations to evaluate semianalytical expressions of the marginal likelihood or by using Monte Carlo (MC) methods to evaluate arithmetic or harmonic mean of a joint likelihood function. This study introduces a new MC method, i.e., thermodynamic integration, which has not been attempted in environmental modeling. Instead of using samples only from prior parameter space (as in arithmetic mean evaluation) or posterior parameter space (as in harmonic mean evaluation), the thermodynamicmore » integration method uses samples generated gradually from the prior to posterior parameter space. This is done through a path sampling that conducts Markov chain Monte Carlo simulation with different power coefficient values applied to the joint likelihood function. The thermodynamic integration method is evaluated using three analytical functions by comparing the method with two variants of the Laplace approximation method and three MC methods, including the nested sampling method that is recently introduced into environmental modeling. The thermodynamic integration method outperforms the other methods in terms of their accuracy, convergence, and consistency. The thermodynamic integration method is also applied to a synthetic case of groundwater modeling with four alternative models. The application shows that model probabilities obtained using the thermodynamic integration method improves predictive performance of Bayesian model averaging. As a result, the thermodynamic integration method is mathematically rigorous, and its MC implementation is computationally general for a wide range of environmental problems.« less

The Regional Climate Model Evaluation System: A Systematic Evaluation Of CORDEX Simulations Using Obs4MIPs

NASA Astrophysics Data System (ADS)

Goodman, A.; Lee, H.; Waliser, D. E.; Guttowski, W.

2017-12-01

Observation-based evaluations of global climate models (GCMs) have been a key element for identifying systematic model biases that can be targeted for model improvements and for establishing uncertainty associated with projections of global climate change. However, GCMs are limited in their ability to represent physical phenomena which occur on smaller, regional scales, including many types of extreme weather events. In order to help facilitate projections in changes of such phenomena, simulations from regional climate models (RCMs) for 14 different domains around the world are being provided by the Coordinated Regional Climate Downscaling Experiment (CORDEX; www.cordex.org). However, although CORDEX specifies standard simulation and archiving protocols, these simulations are conducted independently by individual research and modeling groups representing each of these domains often with different output requirements and data archiving and exchange capabilities. Thus, with respect to similar efforts using GCMs (e.g., the Coupled Model Intercomparison Project, CMIP), it is more difficult to achieve a standardized, systematic evaluation of the RCMs for each domain and across all the CORDEX domains. Using the Regional Climate Model Evaluation System (RCMES; rcmes.jpl.nasa.gov) developed at JPL, we are developing easy to use templates for performing systematic evaluations of CORDEX simulations. Results from the application of a number of evaluation metrics (e.g., biases, centered RMS, and pattern correlations) will be shown for a variety of physical quantities and CORDEX domains. These evaluations are performed using products from obs4MIPs, an activity initiated by DOE and NASA, and now shepherded by the World Climate Research Program's Data Advisory Council.

Toward a model of school inspections in a polycentric system.

PubMed

Janssens, Frans J G; Ehren, Melanie C M

2016-06-01

Many education systems are developing towards more lateral structures where schools collaborate in networks to improve and provide (inclusive) education. These structures call for bottom-up models of network evaluation and accountability instead of the current hierarchical arrangements where single schools are evaluated by a central agency. This paper builds on available research about network effectiveness to present evolving models of network evaluation. Network effectiveness can be defined as the achievement of positive network level outcomes that cannot be attained by individual organizational participants acting alone. Models of network evaluation need to take into account the relations between network members, the structure of the network, its processes and its internal mechanism to enforce norms in order to understand the achievement and outcomes of the network and how these may evolve over time. A range of suitable evaluation models are presented in this paper, as well as a tentative school inspection framework which is inspired by these models. The final section will present examples from Inspectorates of Education in Northern Ireland and Scotland who have developed newer inspection models to evaluate the effectiveness of a range of different networks. Copyright © 2016 Elsevier Ltd. All rights reserved.

Evaluating significance in linear mixed-effects models in R.

PubMed

Luke, Steven G

2017-08-01

Mixed-effects models are being used ever more frequently in the analysis of experimental data. However, in the lme4 package in R the standards for evaluating significance of fixed effects in these models (i.e., obtaining p-values) are somewhat vague. There are good reasons for this, but as researchers who are using these models are required in many cases to report p-values, some method for evaluating the significance of the model output is needed. This paper reports the results of simulations showing that the two most common methods for evaluating significance, using likelihood ratio tests and applying the z distribution to the Wald t values from the model output (t-as-z), are somewhat anti-conservative, especially for smaller sample sizes. Other methods for evaluating significance, including parametric bootstrapping and the Kenward-Roger and Satterthwaite approximations for degrees of freedom, were also evaluated. The results of these simulations suggest that Type 1 error rates are closest to .05 when models are fitted using REML and p-values are derived using the Kenward-Roger or Satterthwaite approximations, as these approximations both produced acceptable Type 1 error rates even for smaller samples.

Brief Lags in Interrupted Sequential Performance: Evaluating a Model and Model Evaluation Method

DTIC Science & Technology

2015-01-05

rehearsal mechanism in the model. To evaluate the model we developed a simple new goodness-of-fit test based on analysis of variance that offers an...repeated step). Sequen- tial constraints are common in medicine, equipment maintenance, computer programming and technical support, data analysis ...legal analysis , accounting, and many other home and workplace environ- ments. Sequential constraints also play a role in such basic cognitive processes

A Multi-Model Assessment for the 2006 and 2010 Simulations under the Air Quality Model Evaluation International Initiative (AQMEII) Phase 2 over North America: Part II. Evaluation of Column Variable Predictions Using Satellite Data

EPA Science Inventory

Within the context of the Air Quality Model Evaluation International Initiative phase 2 (AQMEII2) project, this part II paper performs a multi-model assessment of major column abundances of gases, radiation, aerosol, and cloud variables for 2006 and 2010 simulations with three on...

An Evaluation of Three Approximate Item Response Theory Models for Equating Test Scores.

ERIC Educational Resources Information Center

Marco, Gary L.; And Others

Three item response models were evaluated for estimating item parameters and equating test scores. The models, which approximated the traditional three-parameter model, included: (1) the Rasch one-parameter model, operationalized in the BICAL computer program; (2) an approximate three-parameter logistic model based on coarse group data divided…

A condition metric for Eucalyptus woodland derived from expert evaluations.

PubMed

Sinclair, Steve J; Bruce, Matthew J; Griffioen, Peter; Dodd, Amanda; White, Matthew D

2018-02-01

The evaluation of ecosystem quality is important for land-management and land-use planning. Evaluation is unavoidably subjective, and robust metrics must be based on consensus and the structured use of observations. We devised a transparent and repeatable process for building and testing ecosystem metrics based on expert data. We gathered quantitative evaluation data on the quality of hypothetical grassy woodland sites from experts. We used these data to train a model (an ensemble of 30 bagged regression trees) capable of predicting the perceived quality of similar hypothetical woodlands based on a set of 13 site variables as inputs (e.g., cover of shrubs, richness of native forbs). These variables can be measured at any site and the model implemented in a spreadsheet as a metric of woodland quality. We also investigated the number of experts required to produce an opinion data set sufficient for the construction of a metric. The model produced evaluations similar to those provided by experts, as shown by assessing the model's quality scores of expert-evaluated test sites not used to train the model. We applied the metric to 13 woodland conservation reserves and asked managers of these sites to independently evaluate their quality. To assess metric performance, we compared the model's evaluation of site quality with the managers' evaluations through multidimensional scaling. The metric performed relatively well, plotting close to the center of the space defined by the evaluators. Given the method provides data-driven consensus and repeatability, which no single human evaluator can provide, we suggest it is a valuable tool for evaluating ecosystem quality in real-world contexts. We believe our approach is applicable to any ecosystem. © 2017 State of Victoria.

Semi-Markov adjunction to the Computer-Aided Markov Evaluator (CAME)

NASA Technical Reports Server (NTRS)

Rosch, Gene; Hutchins, Monica A.; Leong, Frank J.; Babcock, Philip S., IV

1988-01-01

The rule-based Computer-Aided Markov Evaluator (CAME) program was expanded in its ability to incorporate the effect of fault-handling processes into the construction of a reliability model. The fault-handling processes are modeled as semi-Markov events and CAME constructs and appropriate semi-Markov model. To solve the model, the program outputs it in a form which can be directly solved with the Semi-Markov Unreliability Range Evaluator (SURE) program. As a means of evaluating the alterations made to the CAME program, the program is used to model the reliability of portions of the Integrated Airframe/Propulsion Control System Architecture (IAPSA 2) reference configuration. The reliability predictions are compared with a previous analysis. The results bear out the feasibility of utilizing CAME to generate appropriate semi-Markov models to model fault-handling processes.

On Lack of Robustness in Hydrological Model Development Due to Absence of Guidelines for Selecting Calibration and Evaluation Data: Demonstration for Data-Driven Models

NASA Astrophysics Data System (ADS)

Zheng, Feifei; Maier, Holger R.; Wu, Wenyan; Dandy, Graeme C.; Gupta, Hoshin V.; Zhang, Tuqiao

2018-02-01

Hydrological models are used for a wide variety of engineering purposes, including streamflow forecasting and flood-risk estimation. To develop such models, it is common to allocate the available data to calibration and evaluation data subsets. Surprisingly, the issue of how this allocation can affect model evaluation performance has been largely ignored in the research literature. This paper discusses the evaluation performance bias that can arise from how available data are allocated to calibration and evaluation subsets. As a first step to assessing this issue in a statistically rigorous fashion, we present a comprehensive investigation of the influence of data allocation on the development of data-driven artificial neural network (ANN) models of streamflow. Four well-known formal data splitting methods are applied to 754 catchments from Australia and the U.S. to develop 902,483 ANN models. Results clearly show that the choice of the method used for data allocation has a significant impact on model performance, particularly for runoff data that are more highly skewed, highlighting the importance of considering the impact of data splitting when developing hydrological models. The statistical behavior of the data splitting methods investigated is discussed and guidance is offered on the selection of the most appropriate data splitting methods to achieve representative evaluation performance for streamflow data with different statistical properties. Although our results are obtained for data-driven models, they highlight the fact that this issue is likely to have a significant impact on all types of hydrological models, especially conceptual rainfall-runoff models.

A Model for Evaluating Programs for the Gifted under Non-Experimental Conditions.

ERIC Educational Resources Information Center

Carter, Kyle R.

1992-01-01

The article presents and illustrates use of an evaluation model for assessing programs for the gifted where tight experimental control is not possible. The model consists of four components: ex post factor designs including intact groups; comparative evaluation; strength of treatment; and multiple outcome assessment from flexible data sources. (DB)

Evaluating habitat suitability models for nesting white-headed woodpeckers in unburned forest

Treesearch

Quresh S. Latif; Victoria A. Saab; Kim Mellen-Mclean; Jonathan G. Dudley

2015-01-01

Habitat suitability models can provide guidelines for species conservation by predicting where species of interest are likely to occur. Presence-only models are widely used but typically provide only relative indices of habitat suitability (HSIs), necessitating rigorous evaluation often using independently collected presence-absence data. We refined and evaluated...

Kentucky Migrant Technology Project: External Evaluation Report, 1997-98.

ERIC Educational Resources Information Center

Popp, Robert J.

During its first year of operation (1997-98), the Kentucky Migrant Technology Project successfully implemented its model, used internal and external evaluations to inform improvement of the model, and began plans for expansion into new service areas. This evaluation report is organized around five questions that focus on the project model and its…

Evaluating the capability of regional-scale air quality models to capture the vertical distribution of pollutants

EPA Science Inventory

This study is conducted in the framework of the Air Quality Modelling Evaluation International Initiative (AQMEII) and aims at the operational evaluation of an ensemble of 12 regional-scale chemical transport models used to predict air quality over the North American (NA) and Eur...

Testing of a Program Evaluation Model: Final Report.

ERIC Educational Resources Information Center

Nagler, Phyllis J.; Marson, Arthur A.

A program evaluation model developed by Moraine Park Technical Institute (MPTI) is described in this report. Following background material, the four main evaluation criteria employed in the model are identified as program quality, program relevance to community needs, program impact on MPTI, and the transition and growth of MPTI graduates in the…

Properties of the Multiple Measures in Arizona's Teacher Evaluation Model. REL 2015-050

ERIC Educational Resources Information Center

Lazarev, Valeriy; Newman, Denis; Sharp, Alyssa

2014-01-01

This study explored the relationships among the components of the Arizona Department of Education's new teacher evaluation model, with a particular focus on the extent to which ratings from the state model's teacher observation instrument differentiated higher and lower performance. The study used teacher-level evaluation data collected by the…

EVALUATION OF THE REAL-TIME AIR-QUALITY MODEL USING THE RAPS (REGIONAL AIR POLLUTION STUDY) DATA BASE. VOLUME 1. OVERVIEW

EPA Science Inventory

The theory and programming of statistical tests for evaluating the Real-Time Air-Quality Model (RAM) using the Regional Air Pollution Study (RAPS) data base are fully documented in four report volumes. Moreover, the tests are generally applicable to other model evaluation problem...

Toward a More Pragmatic Approach to Morality: A Critical Evaluation of Kohlberg's Model

ERIC Educational Resources Information Center

Krebs, Dennis L.; Denton, Kathy

2005-01-01

In this article, the authors evaluate L. Kohlberg's (1984) cognitive-developmental approach to morality, find it wanting, and introduce a more pragmatic approach. They review research designed to evaluate Kohlberg's model, describe how they revised the model to accommodate discrepant findings, and explain why they concluded that it is poorly…

Revitalizing Adversary Evaluation: Deep Dark Deficits or Muddled Mistaken Musings

ERIC Educational Resources Information Center

Thurston, Paul

1978-01-01

The adversary evaluation model consists of utilizing the judicial process as a metaphor for educational evaluation. In this article, previous criticism of the model is addressed and its fundamental problems are detailed. It is speculated that the model could be improved by borrowing ideas from other legal forms of inquiry. (Author/GC)

Stochastic performance modeling and evaluation of obstacle detectability with imaging range sensors

NASA Technical Reports Server (NTRS)

Matthies, Larry; Grandjean, Pierrick

1993-01-01

Statistical modeling and evaluation of the performance of obstacle detection systems for Unmanned Ground Vehicles (UGVs) is essential for the design, evaluation, and comparison of sensor systems. In this report, we address this issue for imaging range sensors by dividing the evaluation problem into two levels: quality of the range data itself and quality of the obstacle detection algorithms applied to the range data. We review existing models of the quality of range data from stereo vision and AM-CW LADAR, then use these to derive a new model for the quality of a simple obstacle detection algorithm. This model predicts the probability of detecting obstacles and the probability of false alarms, as a function of the size and distance of the obstacle, the resolution of the sensor, and the level of noise in the range data. We evaluate these models experimentally using range data from stereo image pairs of a gravel road with known obstacles at several distances. The results show that the approach is a promising tool for predicting and evaluating the performance of obstacle detection with imaging range sensors.

EPA EXPOSURE MODELS LIBRARY AND INTEGRATED MODEL EVALUATION SYSTEM

EPA Science Inventory

The third edition of the U.S. Environmental Protection Agencys (EPA) EML/IMES (Exposure Models Library and Integrated Model Evaluation System) on CD-ROM is now available. The purpose of the disc is to provide a compact and efficient means to distribute exposure models, documentat...

Evaluating Emulation-based Models of Distributed Computing Systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Jones, Stephen T.; Gabert, Kasimir G.; Tarman, Thomas D.

Emulation-based models of distributed computing systems are collections of virtual ma- chines, virtual networks, and other emulation components configured to stand in for oper- ational systems when performing experimental science, training, analysis of design alterna- tives, test and evaluation, or idea generation. As with any tool, we should carefully evaluate whether our uses of emulation-based models are appropriate and justified. Otherwise, we run the risk of using a model incorrectly and creating meaningless results. The variety of uses of emulation-based models each have their own goals and deserve thoughtful evaluation. In this paper, we enumerate some of these uses andmore » describe approaches that one can take to build an evidence-based case that a use of an emulation-based model is credible. Predictive uses of emulation-based models, where we expect a model to tell us something true about the real world, set the bar especially high and the principal evaluation method, called validation , is comensurately rigorous. We spend the majority of our time describing and demonstrating the validation of a simple predictive model using a well-established methodology inherited from decades of development in the compuational science and engineering community.« less

Evaluation of Thoracic Injury in Swine Model with a Noise Immune Stethoscope

DTIC Science & Technology

2011-04-01

USAARL Report No. 2011-16 Evaluation of Thoracic Injury in Swine Model with a Noise Immune Stethoscope By Alaistair Bushby Eric J. Ansorge Keith...Include area code) 22-04-2011 Final Evaluation of Thoracic Injury in Swine Model with a Noise Immune Stethoscope Alaistair Bushby Eric J. Ansorge...and to provide life saving interventions. This study evaluated the feasibility and sensitivity of a newly developed electronic stethoscope concept

Predicting the difficulty of pure, strict, epistatic models: metrics for simulated model selection.

PubMed

Urbanowicz, Ryan J; Kiralis, Jeff; Fisher, Jonathan M; Moore, Jason H

2012-09-26

Algorithms designed to detect complex genetic disease associations are initially evaluated using simulated datasets. Typical evaluations vary constraints that influence the correct detection of underlying models (i.e. number of loci, heritability, and minor allele frequency). Such studies neglect to account for model architecture (i.e. the unique specification and arrangement of penetrance values comprising the genetic model), which alone can influence the detectability of a model. In order to design a simulation study which efficiently takes architecture into account, a reliable metric is needed for model selection. We evaluate three metrics as predictors of relative model detection difficulty derived from previous works: (1) Penetrance table variance (PTV), (2) customized odds ratio (COR), and (3) our own Ease of Detection Measure (EDM), calculated from the penetrance values and respective genotype frequencies of each simulated genetic model. We evaluate the reliability of these metrics across three very different data search algorithms, each with the capacity to detect epistatic interactions. We find that a model's EDM and COR are each stronger predictors of model detection success than heritability. This study formally identifies and evaluates metrics which quantify model detection difficulty. We utilize these metrics to intelligently select models from a population of potential architectures. This allows for an improved simulation study design which accounts for differences in detection difficulty attributed to model architecture. We implement the calculation and utilization of EDM and COR into GAMETES, an algorithm which rapidly and precisely generates pure, strict, n-locus epistatic models.

Promoting Excellence in Nursing Education (PENE): Pross evaluation model.

PubMed

Pross, Elizabeth A

2010-08-01

The purpose of this article is to examine the Promoting Excellence in Nursing Education (PENE) Pross evaluation model. A conceptual evaluation model, such as the one described here, may be useful to nurse academicians in the ongoing evaluation of educational programs, especially those with goals of excellence. Frameworks for evaluating nursing programs are necessary because they offer a way to systematically assess the educational effectiveness of complex nursing programs. This article describes the conceptual framework and its tenets of excellence. Copyright 2009 Elsevier Ltd. All rights reserved.

Integrated Model for E-Learning Acceptance

NASA Astrophysics Data System (ADS)

Ramadiani; Rodziah, A.; Hasan, S. M.; Rusli, A.; Noraini, C.

2016-01-01

E-learning is not going to work if the system is not used in accordance with user needs. User Interface is very important to encourage using the application. Many theories had discuss about user interface usability evaluation and technology acceptance separately, actually why we do not make it correlation between interface usability evaluation and user acceptance to enhance e-learning process. Therefore, the evaluation model for e-learning interface acceptance is considered important to investigate. The aim of this study is to propose the integrated e-learning user interface acceptance evaluation model. This model was combined some theories of e-learning interface measurement such as, user learning style, usability evaluation, and the user benefit. We formulated in constructive questionnaires which were shared at 125 English Language School (ELS) students. This research statistics used Structural Equation Model using LISREL v8.80 and MANOVA analysis.

Evaluating AIDS Prevention: Contributions of Multiple Disciplines.

ERIC Educational Resources Information Center

Leviton, Laura C., Ed.; And Others

1990-01-01

Seven essays on efforts of evaluate prevention programs aimed at the acquired immune deficiency syndrome (AIDS) are presented. Topics include public health psychology, mathematical models of epidemiology, estimates of incubation periods, ethnographic evaluations of AIDS prevention programs, an AIDS education model, theory-based evaluation, and…

Evaluation systems for clinical governance development: a comparative study.

PubMed

Hooshmand, Elaheh; Tourani, Sogand; Ravaghi, Hamid; Ebrahimipour, Hossein

2014-01-01

Lack of scientific and confirmed researches and expert knowledge about evaluation systems for clinical governance development in Iran have made studies on different evaluation systems for clinical governance development a necessity. These studies must provide applied strategies to design criteria of implementing clinical governance for hospital's accreditation. This is a descriptive and comparative study on development of clinical governance models all over the world. Data have been gathered by reviewing related articles. Models have been studied in comprehensive review method. The evaluated models of clinical governance development were Australian, NHS, SPOCK and OPTIGOV. The final aspects extracted from these models were Responsiveness, Policies and Strategies, Organizational Structure, Allocating Resources, Education and Occupational Development, Performance Evaluation, External Evaluation, Patient Oriented Approach, Risk Management, Personnel's Participation, Information Technology, Human Resources, Research and Development, Evidence Based Medicine, Clinical Audit, Health Technology Assessment and Quality. These results are applicable for completing the present criteria which evaluating clinical governance application and provide practical framework to evaluate country's hospital on the basis of clinical governance elements.

[Application of Markov model in post-marketing pharmacoeconomic evaluation of traditional Chinese medicine].

PubMed

Wang, Xin; Su, Xia; Sun, Wentao; Xie, Yanming; Wang, Yongyan

2011-10-01

In post-marketing study of traditional Chinese medicine (TCM), pharmacoeconomic evaluation has an important applied significance. However, the economic literatures of TCM have been unable to fully and accurately reflect the unique overall outcomes of treatment with TCM. For the special nature of TCM itself, we recommend that Markov model could be introduced into post-marketing pharmacoeconomic evaluation of TCM, and also explore the feasibility of model application. Markov model can extrapolate the study time horizon, suit with effectiveness indicators of TCM, and provide measurable comprehensive outcome. In addition, Markov model can promote the development of TCM quality of life scale and the methodology of post-marketing pharmacoeconomic evaluation.

Undergraduate medical education programme renewal: a longitudinal context, input, process and product evaluation study.

PubMed

Mirzazadeh, Azim; Gandomkar, Roghayeh; Hejri, Sara Mortaz; Hassanzadeh, Gholamreza; Koochak, Hamid Emadi; Golestani, Abolfazl; Jafarian, Ali; Jalili, Mohammad; Nayeri, Fatemeh; Saleh, Narges; Shahi, Farhad; Razavi, Seyed Hasan Emami

2016-02-01

The purpose of this study was to utilize the Context, Input, Process and Product (CIPP) evaluation model as a comprehensive framework to guide initiating, planning, implementing and evaluating a revised undergraduate medical education programme. The eight-year longitudinal evaluation study consisted of four phases compatible with the four components of the CIPP model. In the first phase, we explored the strengths and weaknesses of the traditional programme as well as contextual needs, assets, and resources. For the second phase, we proposed a model for the programme considering contextual features. During the process phase, we provided formative information for revisions and adjustments. Finally, in the fourth phase, we evaluated the outcomes of the new undergraduate medical education programme in the basic sciences phase. Information was collected from different sources such as medical students, faculty members, administrators, and graduates, using various qualitative and quantitative methods including focus groups, questionnaires, and performance measures. The CIPP model has the potential to guide policy makers to systematically collect evaluation data and to manage stakeholders' reactions at each stage of the reform in order to make informed decisions. However, the model may result in evaluation burden and fail to address some unplanned evaluation questions.

The functional basis of face evaluation

PubMed Central

Oosterhof, Nikolaas N.; Todorov, Alexander

2008-01-01

People automatically evaluate faces on multiple trait dimensions, and these evaluations predict important social outcomes, ranging from electoral success to sentencing decisions. Based on behavioral studies and computer modeling, we develop a 2D model of face evaluation. First, using a principal components analysis of trait judgments of emotionally neutral faces, we identify two orthogonal dimensions, valence and dominance, that are sufficient to describe face evaluation and show that these dimensions can be approximated by judgments of trustworthiness and dominance. Second, using a data-driven statistical model for face representation, we build and validate models for representing face trustworthiness and face dominance. Third, using these models, we show that, whereas valence evaluation is more sensitive to features resembling expressions signaling whether the person should be avoided or approached, dominance evaluation is more sensitive to features signaling physical strength/weakness. Fourth, we show that important social judgments, such as threat, can be reproduced as a function of the two orthogonal dimensions of valence and dominance. The findings suggest that face evaluation involves an overgeneralization of adaptive mechanisms for inferring harmful intentions and the ability to cause harm and can account for rapid, yet not necessarily accurate, judgments from faces. PMID:18685089

Evaluation model of distribution network development based on ANP and grey correlation analysis

NASA Astrophysics Data System (ADS)

Ma, Kaiqiang; Zhan, Zhihong; Zhou, Ming; Wu, Qiang; Yan, Jun; Chen, Genyong

2018-06-01

The existing distribution network evaluation system cannot scientifically and comprehensively reflect the distribution network development status. Furthermore, the evaluation model is monotonous and it is not suitable for horizontal analysis of many regional power grids. For these reason, this paper constructs a set of universal adaptability evaluation index system and model of distribution network development. Firstly, distribution network evaluation system is set up by power supply capability, power grid structure, technical equipment, intelligent level, efficiency of the power grid and development benefit of power grid. Then the comprehensive weight of indices is calculated by combining the AHP with the grey correlation analysis. Finally, the index scoring function can be obtained by fitting the index evaluation criterion to the curve, and then using the multiply plus operator to get the result of sample evaluation. The example analysis shows that the model can reflect the development of distribution network and find out the advantages and disadvantages of distribution network development. Besides, the model provides suggestions for the development and construction of distribution network.

Betterment, undermining, support and distortion: A heuristic model for the analysis of pressure on evaluators.

PubMed

Pleger, Lyn; Sager, Fritz

2016-09-18

Evaluations can only serve as a neutral evidence base for policy decision-making as long as they have not been altered along non-scientific criteria. Studies show that evaluators are repeatedly put under pressure to deliver results in line with given expectations. The study of pressure and influence to misrepresent findings is hence an important research strand for the development of evaluation praxis. A conceptual challenge in the area of evaluation ethics research is the fact that pressure can be not only negative, but also positive. We develop a heuristic model of influence on evaluations that does justice to this ambivalence of influence: the BUSD-model (betterment, undermining, support, distortion). The model is based on the distinction of two dimensions, namely 'explicitness of pressure' and 'direction of influence'. We demonstrate how the model can be applied to understand pressure and offer a practical tool to distinguish positive from negative influence in the form of three so-called differentiators (awareness, accordance, intention). The differentiators comprise a practical component by assisting evaluators who are confronted with influence. Copyright © 2016 Elsevier Ltd. All rights reserved.

Multi-objective optimization for evaluation of simulation fidelity for precipitation, cloudiness and insolation in regional climate models

NASA Astrophysics Data System (ADS)

Lee, H.

2016-12-01

Precipitation is one of the most important climate variables that are taken into account in studying regional climate. Nevertheless, how precipitation will respond to a changing climate and even its mean state in the current climate are not well represented in regional climate models (RCMs). Hence, comprehensive and mathematically rigorous methodologies to evaluate precipitation and related variables in multiple RCMs are required. The main objective of the current study is to evaluate the joint variability of climate variables related to model performance in simulating precipitation and condense multiple evaluation metrics into a single summary score. We use multi-objective optimization, a mathematical process that provides a set of optimal tradeoff solutions based on a range of evaluation metrics, to characterize the joint representation of precipitation, cloudiness and insolation in RCMs participating in the North American Regional Climate Change Assessment Program (NARCCAP) and Coordinated Regional Climate Downscaling Experiment-North America (CORDEX-NA). We also leverage ground observations, NASA satellite data and the Regional Climate Model Evaluation System (RCMES). Overall, the quantitative comparison of joint probability density functions between the three variables indicates that performance of each model differs markedly between sub-regions and also shows strong seasonal dependence. Because of the large variability across the models, it is important to evaluate models systematically and make future projections using only models showing relatively good performance. Our results indicate that the optimized multi-model ensemble always shows better performance than the arithmetic ensemble mean and may guide reliable future projections.

An interdisciplinary framework for participatory modeling design and evaluation—What makes models effective participatory decision tools?

NASA Astrophysics Data System (ADS)

Falconi, Stefanie M.; Palmer, Richard N.

2017-02-01

Increased requirements for public involvement in water resources management (WRM) over the past century have stimulated the development of more collaborative decision-making methods. Participatory modeling (PM) uses computer models to inform and engage stakeholders in the planning process in order to influence collaborative decisions in WRM. Past evaluations of participatory models focused on process and final outcomes, yet, were hindered by diversity of purpose and inconsistent documentation. This paper presents a two-stage framework for evaluating PM based on mechanisms for improving model effectiveness as participatory tools. The five dimensions characterize the "who, when, how, and why" of each participatory effort (stage 1). Models are evaluated as "boundary objects," a concept used to describe tools that bridge understanding and translate different bodies of knowledge to improve credibility, salience, and legitimacy (stage 2). This evaluation framework is applied to five existing case studies from the literature. Though the goals of participation can be diverse, the novel contribution of the two-stage proposed framework is the flexibility it has to evaluate a wide range of cases that differ in scope, modeling approach, and participatory context. Also, the evaluation criteria provide a structured vocabulary based on clear mechanisms that extend beyond previous process-based and outcome-based evaluations. Effective models are those that take advantage of mechanisms that facilitate dialogue and resolution and improve the accessibility and applicability of technical knowledge. Furthermore, the framework can help build more complete records and systematic documentation of evidence to help standardize the field of PM.

Logic Models for Program Design, Implementation, and Evaluation: Workshop Toolkit. REL 2015-057

ERIC Educational Resources Information Center

Shakman, Karen; Rodriguez, Sheila M.

2015-01-01

The Logic Model Workshop Toolkit is designed to help practitioners learn the purpose of logic models, the different elements of a logic model, and the appropriate steps for developing and using a logic model for program evaluation. Topics covered in the sessions include an overview of logic models, the elements of a logic model, an introduction to…

The Discrepancy Evaluation Model: A Systematic Approach for the Evaluation of Career Planning and Placement Programs.

ERIC Educational Resources Information Center

Buttram, Joan L.; Covert, Robert W.

The Discrepancy Evaluation Model (DEM), developed in 1966 by Malcolm Provus, provides information for program assessment and program improvement. Under the DEM, evaluation is defined as the comparison of an actual performance to a desired standard. The DEM embodies five stages of evaluation based upon a program's natural development: program…

A framework for evaluating forest landscape model predictions using empirical data and knowledge

Treesearch

Wen J. Wang; Hong S. He; Martin A. Spetich; Stephen R. Shifley; Frank R. Thompson; William D. Dijak; Qia Wang

2014-01-01

Evaluation of forest landscape model (FLM) predictions is indispensable to establish the credibility of predictions. We present a framework that evaluates short- and long-term FLM predictions at site and landscape scales. Site-scale evaluation is conducted through comparing raster cell-level predictions with inventory plot data whereas landscape-scale evaluation is...

Guidelines for Evaluating a Superintendent (To Assist School Board Members in Planning and in Evaluation).

ERIC Educational Resources Information Center

California School Boards Association, Sacramento.

This publication is intended to aid local school board members in establishing procedures and priorities for evaluating the performance of their district superintendent. Except for a brief introductory section, the entire publication consists of a model comprehensive evaluation instrument. The evaluation model is organized in two main sections,…

DEVELOPMENT OF GUIDELINES FOR CALIBRATING, VALIDATING, AND EVALUATING HYDROLOGIC AND WATER QUALITY MODELS: ASABE ENGINEERING PRACTICE 621

USDA-ARS?s Scientific Manuscript database

Information to support application of hydrologic and water quality (H/WQ) models abounds, yet modelers commonly use arbitrary, ad hoc methods to conduct, document, and report model calibration, validation, and evaluation. Consistent methods are needed to improve model calibration, validation, and e...

Metrics for evaluating performance and uncertainty of Bayesian network models

Treesearch

Bruce G. Marcot

2012-01-01

This paper presents a selected set of existing and new metrics for gauging Bayesian network model performance and uncertainty. Selected existing and new metrics are discussed for conducting model sensitivity analysis (variance reduction, entropy reduction, case file simulation); evaluating scenarios (influence analysis); depicting model complexity (numbers of model...

Learners' Epistemic Criteria for Good Scientific Models

ERIC Educational Resources Information Center

Pluta, William J.; Chinn, Clark A.; Duncan, Ravit Golan

2011-01-01

Epistemic criteria are the standards used to evaluate scientific products (e.g., models, evidence, arguments). In this study, we analyzed epistemic criteria for good models generated by 324 middle-school students. After evaluating a range of scientific models, but before extensive instruction or experience with model-based reasoning practices,…

A systematic review of Markov models evaluating multicomponent disease management programs in diabetes.

PubMed

Kirsch, Florian

2015-01-01

Diabetes is the most expensive chronic disease; therefore, disease management programs (DMPs) were introduced. The aim of this review is to determine whether Markov models are adequate to evaluate the cost-effectiveness of complex interventions such as DMPs. Additionally, the quality of the models was evaluated using Philips and Caro quality appraisals. The five reviewed models incorporated the DMP into the model differently: two models integrated effectiveness rates derived from one clinical trial/meta-analysis and three models combined interventions from different sources into a DMP. The results range from cost savings and a QALY gain to costs of US$85,087 per QALY. The Spearman's rank coefficient assesses no correlation between the quality appraisals. With restrictions to the data selection process, Markov models are adequate to determine the cost-effectiveness of DMPs; however, to allow prioritization of medical services, more flexibility in the models is necessary to enable the evaluation of single additional interventions.

Feasibility of quasi-random band model in evaluating atmospheric radiance

NASA Technical Reports Server (NTRS)

Tiwari, S. N.; Mirakhur, N.

1980-01-01

The use of the quasi-random band model in evaluating upwelling atmospheric radiation is investigated. The spectral transmittance and total band adsorptance are evaluated for selected molecular bands by using the line by line model, quasi-random band model, exponential sum fit method, and empirical correlations, and these are compared with the available experimental results. The atmospheric transmittance and upwelling radiance were calculated by using the line by line and quasi random band models and were compared with the results of an existing program called LOWTRAN. The results obtained by the exponential sum fit and empirical relations were not in good agreement with experimental results and their use cannot be justified for atmospheric studies. The line by line model was found to be the best model for atmospheric applications, but it is not practical because of high computational costs. The results of the quasi random band model compare well with the line by line and experimental results. The use of the quasi random band model is recommended for evaluation of the atmospheric radiation.

A risk evaluation model and its application in online retailing trustfulness

NASA Astrophysics Data System (ADS)

Ye, Ruyi; Xu, Yingcheng

2017-08-01

Building a general model for risks evaluation in advance could improve the convenience, normality and comparability of the results of repeating risks evaluation in the case that the repeating risks evaluating are in the same area and for a similar purpose. One of the most convenient and common risks evaluation models is an index system including of several index, according weights and crediting method. One method to build a risk evaluation index system that guarantees the proportional relationship between the resulting credit and the expected risk loss is proposed and an application example is provided in online retailing in this article.

Genetic evaluation and selection response for growth in meat-type quail through random regression models using B-spline functions and Legendre polynomials.

PubMed

Mota, L F M; Martins, P G M A; Littiere, T O; Abreu, L R A; Silva, M A; Bonafé, C M

2018-04-01

The objective was to estimate (co)variance functions using random regression models (RRM) with Legendre polynomials, B-spline function and multi-trait models aimed at evaluating genetic parameters of growth traits in meat-type quail. A database containing the complete pedigree information of 7000 meat-type quail was utilized. The models included the fixed effects of contemporary group and generation. Direct additive genetic and permanent environmental effects, considered as random, were modeled using B-spline functions considering quadratic and cubic polynomials for each individual segment, and Legendre polynomials for age. Residual variances were grouped in four age classes. Direct additive genetic and permanent environmental effects were modeled using 2 to 4 segments and were modeled by Legendre polynomial with orders of fit ranging from 2 to 4. The model with quadratic B-spline adjustment, using four segments for direct additive genetic and permanent environmental effects, was the most appropriate and parsimonious to describe the covariance structure of the data. The RRM using Legendre polynomials presented an underestimation of the residual variance. Lesser heritability estimates were observed for multi-trait models in comparison with RRM for the evaluated ages. In general, the genetic correlations between measures of BW from hatching to 35 days of age decreased as the range between the evaluated ages increased. Genetic trend for BW was positive and significant along the selection generations. The genetic response to selection for BW in the evaluated ages presented greater values for RRM compared with multi-trait models. In summary, RRM using B-spline functions with four residual variance classes and segments were the best fit for genetic evaluation of growth traits in meat-type quail. In conclusion, RRM should be considered in genetic evaluation of breeding programs.

Deriving the expected utility of a predictive model when the utilities are uncertain.

PubMed

Cooper, Gregory F; Visweswaran, Shyam

2005-01-01

Predictive models are often constructed from clinical databases with the goal of eventually helping make better clinical decisions. Evaluating models using decision theory is therefore natural. When constructing a model using statistical and machine learning methods, however, we are often uncertain about precisely how the model will be used. Thus, decision-independent measures of classification performance, such as the area under an ROC curve, are popular. As a complementary method of evaluation, we investigate techniques for deriving the expected utility of a model under uncertainty about the model's utilities. We demonstrate an example of the application of this approach to the evaluation of two models that diagnose coronary artery disease.

Evaluating the Bias of Alternative Cost Progress Models: Tests Using Aerospace Industry Acquisition Programs

DTIC Science & Technology

1992-12-01

suspect :mat, -n2 extent predict:.on cas jas ccsiziveiv crrei:=e amonc e v:arious models, :he fandom *.;aik, learn ha r ur e, i;<ea- variable and Bemis...Functions, Production Rate Adjustment Model, Learning Curve Model. Random Walk Model. Bemis Model. Evaluating Model Bias, Cost Prediction Bias. Cost...of four cost progress models--a random walk model, the tradiuonai learning curve model, a production rate model Ifixed-variable model). and a model

Developing R&D portfolio business validity simulation model and system.

PubMed

Yeo, Hyun Jin; Im, Kwang Hyuk

2015-01-01

The R&D has been recognized as critical method to take competitiveness by not only companies but also nations with its value creation such as patent value and new product. Therefore, R&D has been a decision maker's burden in that it is hard to decide how much money to invest, how long time one should spend, and what technology to develop which means it accompanies resources such as budget, time, and manpower. Although there are diverse researches about R&D evaluation, business factors are not concerned enough because almost all previous studies are technology oriented evaluation with one R&D technology based. In that, we early proposed R&D business aspect evaluation model which consists of nine business model components. In this research, we develop a simulation model and system evaluating a company or industry's R&D portfolio with business model point of view and clarify default and control parameters to facilitate evaluator's business validity work in each evaluation module by integrate to one screen.

Developing R&D Portfolio Business Validity Simulation Model and System

PubMed Central

2015-01-01

The R&D has been recognized as critical method to take competitiveness by not only companies but also nations with its value creation such as patent value and new product. Therefore, R&D has been a decision maker's burden in that it is hard to decide how much money to invest, how long time one should spend, and what technology to develop which means it accompanies resources such as budget, time, and manpower. Although there are diverse researches about R&D evaluation, business factors are not concerned enough because almost all previous studies are technology oriented evaluation with one R&D technology based. In that, we early proposed R&D business aspect evaluation model which consists of nine business model components. In this research, we develop a simulation model and system evaluating a company or industry's R&D portfolio with business model point of view and clarify default and control parameters to facilitate evaluator's business validity work in each evaluation module by integrate to one screen. PMID:25893209

THE ATMOSPHERIC MODEL EVALUATION (AMET): METEOROLOGY MODULE

EPA Science Inventory

An Atmospheric Model Evaluation Tool (AMET), composed of meteorological and air quality components, is being developed to examine the error and uncertainty in the model simulations. AMET matches observations with the corresponding model-estimated values in space and time, and the...

Towards Systematic Benchmarking of Climate Model Performance

NASA Astrophysics Data System (ADS)

Gleckler, P. J.

2014-12-01

The process by which climate models are evaluated has evolved substantially over the past decade, with the Coupled Model Intercomparison Project (CMIP) serving as a centralizing activity for coordinating model experimentation and enabling research. Scientists with a broad spectrum of expertise have contributed to the CMIP model evaluation process, resulting in many hundreds of publications that have served as a key resource for the IPCC process. For several reasons, efforts are now underway to further systematize some aspects of the model evaluation process. First, some model evaluation can now be considered routine and should not require "re-inventing the wheel" or a journal publication simply to update results with newer models. Second, the benefit of CMIP research to model development has not been optimal because the publication of results generally takes several years and is usually not reproducible for benchmarking newer model versions. And third, there are now hundreds of model versions and many thousands of simulations, but there is no community-based mechanism for routinely monitoring model performance changes. An important change in the design of CMIP6 can help address these limitations. CMIP6 will include a small set standardized experiments as an ongoing exercise (CMIP "DECK": ongoing Diagnostic, Evaluation and Characterization of Klima), so that modeling groups can submit them at any time and not be overly constrained by deadlines. In this presentation, efforts to establish routine benchmarking of existing and future CMIP simulations will be described. To date, some benchmarking tools have been made available to all CMIP modeling groups to enable them to readily compare with CMIP5 simulations during the model development process. A natural extension of this effort is to make results from all CMIP simulations widely available, including the results from newer models as soon as the simulations become available for research. Making the results from routine performance tests readily accessible will help advance a more transparent model evaluation process.

Evaluation of the Navys Sea/Shore Flow Policy

DTIC Science & Technology

2016-06-01

CNA developed an independent Discrete -Event Simulation model to evaluate and assess the effect of alternative sea/shore flow policies. In this study...remains, even if the system is optimized. In building a Discrete -Event Simulation model, we discovered key factors that should be included in the... Discrete -Event Simulation model to evaluate the impact of sea/shore flow policy (the DES-SSF model) and compared the results with the SSFM for one

Evaluation of Resuspension from Propeller Wash in DoD Harbors

DTIC Science & Technology

2016-09-01

Environmental Research and Development Center FANS FOV ICP-MS Finite Analytical Navier-Stoker Solver Field of View Inductively Coupled Plasma with...Model (1984) and the Finite Analytical Navier- Stoker Solver (FANS) model (Chen et al., 2003) were set up to simulate and evaluate flow velocities and...model for evaluating the resuspension potential of propeller wash by a tugboat and the FANS model for a DDG. The Finite -Analytic Navier-Stokes (FANS

Training Maneuver Evaluation for Reduced Order Modeling of Stability & Control Properties Using Computational Fluid Dynamics

DTIC Science & Technology

2013-03-01

reduced order model is created. Finally, previous research in this area of study will be examined, and its application to this research will be...TRAINING MANEUVER EVALUATION FOR REDUCED ORDER MODELING OF STABILITY & CONTROL PROPERTIES USING COMPUTATIONAL FLUID DYNAMICS THESIS Craig Curtis...Government and is not subject to copyright protection in the United States. AFIT-ENY-13-M-28 TRAINING MANEUVER EVALUATION FOR REDUCED ORDER MODELING OF

Discussion on accuracy degree evaluation of accident velocity reconstruction model

NASA Astrophysics Data System (ADS)

Zou, Tiefang; Dai, Yingbiao; Cai, Ming; Liu, Jike

In order to investigate the applicability of accident velocity reconstruction model in different cases, a method used to evaluate accuracy degree of accident velocity reconstruction model is given. Based on pre-crash velocity in theory and calculation, an accuracy degree evaluation formula is obtained. With a numerical simulation case, Accuracy degrees and applicability of two accident velocity reconstruction models are analyzed; results show that this method is feasible in practice.

“Overview and Evaluation of AQMEII Phase 2 Coupled ...

EPA Pesticide Factsheets

This presentation provides an overview of the second phase of the Air Quality Model Evaluation International Initative (AQMEII). Activities in this phase are focused on the application and evaluation of coupled meteorology-chemistry models to assess how well these models can simulate the observed spatio-temporal variability in the optical and radiative characteristics of atmospheric aerosols and associated feedbacks among aerosols, radiation, clouds, and precipitation. To this end, these modeling systems are being applied for annual simulations over both North America and Europe using common emissions and boundary conditions for all modeling groups. We present an overview of these common input datasets, observational datasets for model evaluation, and case studies for diagnostic evaluation. In addition to this overview, we also present results from AQMEII Phase 2 WRF/CMAQ simulations over North America for both 2006 and 2010. The time period between 2006 and 2010 was characterized by a 35% reduction in U.S. SO2 emissions and 20% reduction in U.S. NOx emissions, providing an opportunity for dynamic model evaluation by investigating the impact of emission reductions on ambient concentrations as well as aerosol/radiation feedback effects. We present results of this dynamic evaluation. We also present a brief overview of initial results from WRF-Chem and GEM-MACH simulations performed for the same time period and domain as part of AQMEII Phase 2. The National Exposu

Teacher Perceptions about New Evaluation Model Implementations

ERIC Educational Resources Information Center

Bush, Charles D.

2017-01-01

The challenge of designing and implementing teacher evaluation reform throughout the U.S. has been represented by different policies, teacher evaluation components, and difficulties with implementation. The purpose of this qualitative embedded single case study was to explore teacher perceptions about new evaluation model implementations and how…

Occupational hazard evaluation model underground coal mine based on unascertained measurement theory

NASA Astrophysics Data System (ADS)

Deng, Quanlong; Jiang, Zhongan; Sun, Yaru; Peng, Ya

2017-05-01

In order to study how to comprehensively evaluate the influence of several occupational hazard on miners’ physical and mental health, based on unascertained measurement theory, occupational hazard evaluation indicator system was established to make quantitative and qualitative analysis. Determining every indicator weight by information entropy and estimating the occupational hazard level by credible degree recognition criteria, the evaluation model was programmed by Visual Basic, applying the evaluation model to occupational hazard comprehensive evaluation of six posts under a coal mine, and the occupational hazard degree was graded, the evaluation results are consistent with actual situation. The results show that dust and noise is most obvious among the coal mine occupational hazard factors. Excavation face support workers are most affected, secondly, heading machine drivers, coal cutter drivers, coalface move support workers, the occupational hazard degree of these four types workers is II mild level. The occupational hazard degree of ventilation workers and safety inspection workers is I level. The evaluation model could evaluate underground coal mine objectively and accurately, and can be employed to the actual engineering.

Model Evaluation and Ensemble Modelling of Surface-Level Ozone in Europe and North America in the Context of AQMEII

EPA Science Inventory

More than ten state-of-the-art regional air quality models have been applied as part of the Air Quality Model Evaluation International Initiative (AQMEII). These models were run by twenty independent groups in Europe and North America. Standardised modelling outputs over a full y...

TThe role of nitrogen availability in land-atmosphere interactions: a systematic evaluation of carbon-nitrogen coupling in a global land surface model using plot-level nitrogen fertilization experiments

NASA Astrophysics Data System (ADS)

Thomas, R. Q.; Goodale, C. L.; Bonan, G. B.; Mahowald, N. M.; Ricciuto, D. M.; Thornton, P. E.

2010-12-01

Recent research from global land surface models emphasizes the important role of nitrogen cycling on global climate, via its control on the terrestrial carbon balance. Despite the implications of nitrogen cycling on global climate predictions, the research community has not performed a systematic evaluation of nitrogen cycling in global models. Here, we present such an evaluation for one global land model, CLM-CN. In the evaluation we simulated 45 plot-scale nitrogen-fertilization experiments distributed across 33 temperate and boreal forest sites. Model predictions were evaluated against field observations by comparing the vegetation and soil carbon responses to the additional nitrogen. Aggregated across all experiments, the model predicted a larger vegetation carbon response and a smaller soil carbon response than observed; the responses partially offset each other, leading to a slightly larger total ecosystem carbon response than observed. However, the model-observation agreement improved for vegetation carbon when the sites with observed negative carbon responses to nitrogen were excluded, which may be because the model lacks mechanisms whereby nitrogen additions increase tree mortality. Among experiments, younger forests and boreal forests’ vegetation carbon responses were less than predicted and mature forests (> 40 years old) were greater than predicted. Specific to the CLM-CN, this study used a systematic evaluation to identify key areas to focus model development, especially soil carbon- nitrogen interactions and boreal forest nitrogen cycling. Applicable to the modeling community, this study demonstrates a standardized protocol for comparing carbon-nitrogen interactions among global land models.

Forecasting biodiversity in breeding birds using best practices

PubMed Central

Taylor, Shawn D.; White, Ethan P.

2018-01-01

Biodiversity forecasts are important for conservation, management, and evaluating how well current models characterize natural systems. While the number of forecasts for biodiversity is increasing, there is little information available on how well these forecasts work. Most biodiversity forecasts are not evaluated to determine how well they predict future diversity, fail to account for uncertainty, and do not use time-series data that captures the actual dynamics being studied. We addressed these limitations by using best practices to explore our ability to forecast the species richness of breeding birds in North America. We used hindcasting to evaluate six different modeling approaches for predicting richness. Hindcasts for each method were evaluated annually for a decade at 1,237 sites distributed throughout the continental United States. All models explained more than 50% of the variance in richness, but none of them consistently outperformed a baseline model that predicted constant richness at each site. The best practices implemented in this study directly influenced the forecasts and evaluations. Stacked species distribution models and “naive” forecasts produced poor estimates of uncertainty and accounting for this resulted in these models dropping in the relative performance compared to other models. Accounting for observer effects improved model performance overall, but also changed the rank ordering of models because it did not improve the accuracy of the “naive” model. Considering the forecast horizon revealed that the prediction accuracy decreased across all models as the time horizon of the forecast increased. To facilitate the rapid improvement of biodiversity forecasts, we emphasize the value of specific best practices in making forecasts and evaluating forecasting methods. PMID:29441230

An Evaluation of Artificial Neural Network Modeling for Manpower Analysis

DTIC Science & Technology

1993-09-01

NAVAL POSTGRADUATE SCHOOL Monterey, California 0- I 1 ’(ft ADV "’r-"A THESIS AN EVALUATION OF ARTIFICIAL NEURAL NETWORK MODELING FOR MANPOWER...AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED September, 1993 4. TITLE AND SUBTITLE An Evaluation Of Artificial Neural Network 5...unlimited. An Evaluation of Artificial Neural Network Modeling for Manpower Analysis by Brian J. Byrne Captain, United States Marine Corps B.S

CoLeMo: A Collaborative Learning Environment for UML Modelling

ERIC Educational Resources Information Center

Chen, Weiqin; Pedersen, Roger Heggernes; Pettersen, Oystein

2006-01-01

This paper presents the design, implementation, and evaluation of a distributed collaborative UML modelling environment, CoLeMo. CoLeMo is designed for students studying UML modelling. It can also be used as a platform for collaborative design of software. We conducted formative evaluations and a summative evaluation to improve the environment and…

Modelling in Evaluating a Working Life Project in Higher Education

ERIC Educational Resources Information Center

Sarja, Anneli; Janhonen, Sirpa; Havukainen, Pirjo; Vesterinen, Anne

2012-01-01

This article describes an evaluation method based on collaboration between the higher education, a care home and university, in a R&D project. The aim of the project was to elaborate modelling as a tool of developmental evaluation for innovation and competence in project cooperation. The approach was based on activity theory. Modelling enabled a…

Evaluating Vocational Educators' Training Programs: A Kirkpatrick-Inspired Evaluation Model

ERIC Educational Resources Information Center

Ravicchio, Fabrizio; Trentin, Guglielmo

2015-01-01

The aim of the article is to describe the assessment model adopted by the SCINTILLA Project, a project in Italy aimed at the online vocational training of young, seriously-disabled subjects and their subsequent work inclusion in smart-work mode. It will thus describe the model worked out for evaluation of the training program conceived for the…

EVALUATION OF THE REAL-TIME AIR-QUALITY MODEL USING THE RAPS (REGIONAL AIR POLLUTION STUDY) DATA BASE. VOLUME 3. PROGRAM USER'S GUIDE

EPA Science Inventory

The theory and programming of statistical tests for evaluating the Real-Time Air-Quality Model (RAM) using the Regional Air Pollution Study (RAPS) data base are fully documented in four volumes. Moreover, the tests are generally applicable to other model evaluation problems. Volu...

Evaluating Social Causality and Responsibility Models: An Initial Report

DTIC Science & Technology

2005-01-01

ICT Technical Report ICT-TR-03-2005 Evaluating Social Causality and Responsibility ... social intelligent agents. In this report, we present a general computational model of social causality and responsibility , and empirical results of...2005 to 00-00-2005 4. TITLE AND SUBTITLE Evaluating Social Causality and Responsibility Models: An Initial Report 5a. CONTRACT NUMBER 5b. GRANT

Using the learning management evaluation model for advancing to life skills of lower secondary students in the 21st century

NASA Astrophysics Data System (ADS)

Kansaart, Preecha; Suikraduang, Arun; Panya, Piyatida

2018-01-01

The aims of this research study were to develop the Learning Management Evaluation Model (LMEM) for advancing to lower secondary students of their life skills in the 21st century with the Research & Development process technique. The research procedures were administered of four steps that composed of analyze, the synthetic indicator to assess learning to advance to their life skills in the 21st century by the 4-educational experts were interviewed. The LMEM model was developed by the information from the first draft format and the educational experts to check a suitability and feasibility of the draft assessment form with a technical symposium multipath characteristics to find consensus dimensional (Multi-Attribute Consensus Reaching: MACR) by 12 specialists who provided the instruction in the form of Assessment and Evaluation Guide (AEG) was brought to five the number of professionals who ensure the proper coverage, a clear assessment of the manual before using the AEG. The LMEM model was to trial at an experiment with different schools in the Secondary Educational Office Area 26 (Maha Sarakham) whereas taught at the upper secondary educational school with the sample consisted of 7 schools with the purposive sampling was selected. Assessing the LMEM model was evaluated the based on the evaluation criteria of the educational development. The assessor was related to the trial consisted of 35 evaluators. Using the interview form with the rubric score and a five rating scale level was analyzed; the qualitative and quantitative data were used. It has found that: The LMEM evaluation model of learning to advance to life skills of students in the 21st century was a chart structure that ties together of 6 relevant components of the evaluation such as; the purpose of the assessment, the evaluation focused assessment methods, the evaluator, the evaluation technique, and the evaluation criteria. The evaluation targets were to assess the management of learning, the factors contributing to learning, feature teacher management learning, and the learning outcomes. Evaluating methods included with the evaluation process, the tool used to evaluate, and duration to assess. Assessing the LMEM model of learning to advance to students of their life skills in the 21st century were appropriated ability. Students' responses of their opportune, practicability, reasonableness, and respectability in terms of overall benefit at a high level are provided.

Evaluation of image quality

NASA Technical Reports Server (NTRS)

Pavel, M.

1993-01-01

This presentation outlines in viewgraph format a general approach to the evaluation of display system quality for aviation applications. This approach is based on the assumption that it is possible to develop a model of the display which captures most of the significant properties of the display. The display characteristics should include spatial and temporal resolution, intensity quantizing effects, spatial sampling, delays, etc. The model must be sufficiently well specified to permit generation of stimuli that simulate the output of the display system. The first step in the evaluation of display quality is an analysis of the tasks to be performed using the display. Thus, for example, if a display is used by a pilot during a final approach, the aesthetic aspects of the display may be less relevant than its dynamic characteristics. The opposite task requirements may apply to imaging systems used for displaying navigation charts. Thus, display quality is defined with regard to one or more tasks. Given a set of relevant tasks, there are many ways to approach display evaluation. The range of evaluation approaches includes visual inspection, rapid evaluation, part-task simulation, and full mission simulation. The work described is focused on two complementary approaches to rapid evaluation. The first approach is based on a model of the human visual system. A model of the human visual system is used to predict the performance of the selected tasks. The model-based evaluation approach permits very rapid and inexpensive evaluation of various design decisions. The second rapid evaluation approach employs specifically designed critical tests that embody many important characteristics of actual tasks. These are used in situations where a validated model is not available. These rapid evaluation tests are being implemented in a workstation environment.

Oncology Modeling for Fun and Profit! Key Steps for Busy Analysts in Health Technology Assessment.

PubMed

Beca, Jaclyn; Husereau, Don; Chan, Kelvin K W; Hawkins, Neil; Hoch, Jeffrey S

2018-01-01

In evaluating new oncology medicines, two common modeling approaches are state transition (e.g., Markov and semi-Markov) and partitioned survival. Partitioned survival models have become more prominent in oncology health technology assessment processes in recent years. Our experience in conducting and evaluating models for economic evaluation has highlighted many important and practical pitfalls. As there is little guidance available on best practices for those who wish to conduct them, we provide guidance in the form of 'Key steps for busy analysts,' who may have very little time and require highly favorable results. Our guidance highlights the continued need for rigorous conduct and transparent reporting of economic evaluations regardless of the modeling approach taken, and the importance of modeling that better reflects reality, which includes better approaches to considering plausibility, estimating relative treatment effects, dealing with post-progression effects, and appropriate characterization of the uncertainty from modeling itself.

TTI CM/AQ evaluation model user`s guide and workshop training materials. Interim research report, September 1993-August 1996

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

1995-08-01

The TTI CM/AQ Evaluation Model evaluates potential projects based on the following criteria: eligibility, travel impacts, emission impacts, and cost-effectiveness. To compare independent projects within a region during the decision process for CM/AQ funding, each project evaluated with this model is given an overall score based on the project`s effects for the criteria listed above. Training workshops were held by TTI in the first quarter of 1995 to teach metropolitan planning organization, state department of transportation, and regional air quality organization staff how to use this model. Basics of sketch-planning applications were also taught. The DRCOG and TTI CM/AQ Evaluationmore » Models represent significant steps toward the development of analytical methodologies for selecting projects for CM/AQ funding. Because the needs of nonattainment and attainment areas change over time, this model is particularly useful as key evaluation criteria can be modified to reflect the changing needs of a metropolitan area.« less

Using satellite observations in performance evaluation for regulatory air quality modeling: Comparison with ground-level measurements

NASA Astrophysics Data System (ADS)

Odman, M. T.; Hu, Y.; Russell, A.; Chai, T.; Lee, P.; Shankar, U.; Boylan, J.

2012-12-01

Regulatory air quality modeling, such as State Implementation Plan (SIP) modeling, requires that model performance meets recommended criteria in the base-year simulations using period-specific, estimated emissions. The goal of the performance evaluation is to assure that the base-year modeling accurately captures the observed chemical reality of the lower troposphere. Any significant deficiencies found in the performance evaluation must be corrected before any base-case (with typical emissions) and future-year modeling is conducted. Corrections are usually made to model inputs such as emission-rate estimates or meteorology and/or to the air quality model itself, in modules that describe specific processes. Use of ground-level measurements that follow approved protocols is recommended for evaluating model performance. However, ground-level monitoring networks are spatially sparse, especially for particulate matter. Satellite retrievals of atmospheric chemical properties such as aerosol optical depth (AOD) provide spatial coverage that can compensate for the sparseness of ground-level measurements. Satellite retrievals can also help diagnose potential model or data problems in the upper troposphere. It is possible to achieve good model performance near the ground, but have, for example, erroneous sources or sinks in the upper troposphere that may result in misleading and unrealistic responses to emission reductions. Despite these advantages, satellite retrievals are rarely used in model performance evaluation, especially for regulatory modeling purposes, due to the high uncertainty in retrievals associated with various contaminations, for example by clouds. In this study, 2007 was selected as the base year for SIP modeling in the southeastern U.S. Performance of the Community Multiscale Air Quality (CMAQ) model, at a 12-km horizontal resolution, for this annual simulation is evaluated using both recommended ground-level measurements and non-traditional satellite retrievals. Evaluation results are assessed against recommended criteria and peer studies in the literature. Further analysis is conducted, based upon these assessments, to discover likely errors in model inputs and potential deficiencies in the model itself. Correlations as well as differences in input errors and model deficiencies revealed by ground-level measurements versus satellite observations are discussed. Additionally, sensitivity analyses are employed to investigate errors in emission-rate estimates using either ground-level measurements or satellite retrievals, and the results are compared against each other considering observational uncertainties. Recommendations are made for how to effectively utilize satellite retrievals in regulatory air quality modeling.

Self-organization comprehensive real-time state evaluation model for oil pump unit on the basis of operating condition classification and recognition

NASA Astrophysics Data System (ADS)

Liang, Wei; Yu, Xuchao; Zhang, Laibin; Lu, Wenqing

2018-05-01

In oil transmission station, the operating condition (OC) of an oil pump unit sometimes switches accordingly, which will lead to changes in operating parameters. If not taking the switching of OCs into consideration while performing a state evaluation on the pump unit, the accuracy of evaluation would be largely influenced. Hence, in this paper, a self-organization Comprehensive Real-Time State Evaluation Model (self-organization CRTSEM) is proposed based on OC classification and recognition. However, the underlying model CRTSEM is built through incorporating the advantages of Gaussian Mixture Model (GMM) and Fuzzy Comprehensive Evaluation Model (FCEM) first. That is to say, independent state models are established for every state characteristic parameter according to their distribution types (i.e. the Gaussian distribution and logistic regression distribution). Meanwhile, Analytic Hierarchy Process (AHP) is utilized to calculate the weights of state characteristic parameters. Then, the OC classification is determined by the types of oil delivery tasks, and CRTSEMs of different standard OCs are built to constitute the CRTSEM matrix. On the other side, the OC recognition is realized by a self-organization model that is established on the basis of Back Propagation (BP) model. After the self-organization CRTSEM is derived through integration, real-time monitoring data can be inputted for OC recognition. At the end, the current state of the pump unit can be evaluated by using the right CRTSEM. The case study manifests that the proposed self-organization CRTSEM can provide reasonable and accurate state evaluation results for the pump unit. Besides, the assumption that the switching of OCs will influence the results of state evaluation is also verified.

APPLICATION AND EVALUATION OF CMAQ IN THE UNITED STATES: AIR QUALITY FORECASTING AND RETROSPECTIVE MODELING

EPA Science Inventory

Presentation slides provide background on model evaluation techniques. Also included in the presentation is an operational evaluation of 2001 Community Multiscale Air Quality (CMAQ) annual simulation, and an evaluation of PM_2.5 for the CMAQ air quality forecast (AQF) ...

Measuring Success: Evaluating Educational Programs

ERIC Educational Resources Information Center

Fisher, Yael

2010-01-01

This paper reveals a new evaluation model, which enables educational program and project managers to evaluate their programs with a simple and easy to understand approach. The "index of success model" is comprised of five parameters that enable to focus on and evaluate both the implementation and results of an educational program. The…

Multilevel Evaluation Systems Project. Final Report.

ERIC Educational Resources Information Center

Herman, Joan L.

Several studies were conducted in 1987 by the Multilevel Evaluation Systems Project, which focuses on developing a model for a multi-purpose, multi-user evaluation system to facilitate educational decision making and evaluation. The project model emphasizes on-going integrated assessment of individuals, classes, and programs using a variety of…

A Holistic Approach to Evaluating Vocational Education: Traditional Chinese Physicians (TCP) Model.

ERIC Educational Resources Information Center

Lee, Lung-Sheng; Chang, Liang-Te

Conventional approaches to evaluating vocational education have often been criticized for failing to deal holistically with the institution or program being evaluated. Integrated quantitative and qualitative evaluation methods have documented benefits; therefore, it would be useful to consider possibility of developing a model for evaluating…

A Critical Analysis of HRD Evaluation Models from a Decision-Making Perspective

ERIC Educational Resources Information Center

Holton, Elwood F., III; Naquin, Sharon

2005-01-01

HRD evaluation models are recommended for use by organizations to improve decisions made about HRD interventions. However, the organizational decision-making literature has been virtually ignored by evaluation researchers. In this article, we review the organizational decision-making literature and critically review HRD evaluation research through…

Emergent climate and CO2 sensitivities of net primary productivity in ecosystem models do not agree with empirical data in temperate forests of eastern North America.

PubMed

Rollinson, Christine R; Liu, Yao; Raiho, Ann; Moore, David J P; McLachlan, Jason; Bishop, Daniel A; Dye, Alex; Matthes, Jaclyn H; Hessl, Amy; Hickler, Thomas; Pederson, Neil; Poulter, Benjamin; Quaife, Tristan; Schaefer, Kevin; Steinkamp, Jörg; Dietze, Michael C

2017-07-01

Ecosystem models show divergent responses of the terrestrial carbon cycle to global change over the next century. Individual model evaluation and multimodel comparisons with data have largely focused on individual processes at subannual to decadal scales. Thus far, data-based evaluations of emergent ecosystem responses to climate and CO 2 at multidecadal and centennial timescales have been rare. We compared the sensitivity of net primary productivity (NPP) to temperature, precipitation, and CO 2 in ten ecosystem models with the sensitivities found in tree-ring reconstructions of NPP and raw ring-width series at six temperate forest sites. These model-data comparisons were evaluated at three temporal extents to determine whether the rapid, directional changes in temperature and CO 2 in the recent past skew our observed responses to multiple drivers of change. All models tested here were more sensitive to low growing season precipitation than tree-ring NPP and ring widths in the past 30 years, although some model precipitation responses were more consistent with tree rings when evaluated over a full century. Similarly, all models had negative or no response to warm-growing season temperatures, while tree-ring data showed consistently positive effects of temperature. Although precipitation responses were least consistent among models, differences among models to CO 2 drive divergence and ensemble uncertainty in relative change in NPP over the past century. Changes in forest composition within models had no effect on climate or CO 2 sensitivity. Fire in model simulations reduced model sensitivity to climate and CO 2 , but only over the course of multiple centuries. Formal evaluation of emergent model behavior at multidecadal and multicentennial timescales is essential to reconciling model projections with observed ecosystem responses to past climate change. Future evaluation should focus on improved representation of disturbance and biomass change as well as the feedbacks with moisture balance and CO 2 in individual models. © 2017 John Wiley & Sons Ltd.

EPA Corporate GHG Goal Evaluation Model

EPA Pesticide Factsheets

The EPA Corporate GHG Goal Evaluation Model provides companies with a transparent and publicly available benchmarking resource to help evaluate and establish new or existing GHG goals that go beyond business as usual for their individual sectors.

Evaluation of gravitational gradients generated by Earth's crustal structures

NASA Astrophysics Data System (ADS)

Novák, Pavel; Tenzer, Robert; Eshagh, Mehdi; Bagherbandi, Mohammad

2013-02-01

Spectral formulas for the evaluation of gravitational gradients generated by upper Earth's mass components are presented in the manuscript. The spectral approach allows for numerical evaluation of global gravitational gradient fields that can be used to constrain gravitational gradients either synthesised from global gravitational models or directly measured by the spaceborne gradiometer on board of the GOCE satellite mission. Gravitational gradients generated by static atmospheric, topographic and continental ice masses are evaluated numerically based on available global models of Earth's topography, bathymetry and continental ice sheets. CRUST2.0 data are then applied for the numerical evaluation of gravitational gradients generated by mass density contrasts within soft and hard sediments, upper, middle and lower crust layers. Combined gravitational gradients are compared to disturbing gravitational gradients derived from a global gravitational model and an idealised Earth's model represented by the geocentric homogeneous biaxial ellipsoid GRS80. The methodology could be used for improved modelling of the Earth's inner structure.

Decision curve analysis: a novel method for evaluating prediction models.

PubMed

Vickers, Andrew J; Elkin, Elena B

2006-01-01

Diagnostic and prognostic models are typically evaluated with measures of accuracy that do not address clinical consequences. Decision-analytic techniques allow assessment of clinical outcomes but often require collection of additional information and may be cumbersome to apply to models that yield a continuous result. The authors sought a method for evaluating and comparing prediction models that incorporates clinical consequences,requires only the data set on which the models are tested,and can be applied to models that have either continuous or dichotomous results. The authors describe decision curve analysis, a simple, novel method of evaluating predictive models. They start by assuming that the threshold probability of a disease or event at which a patient would opt for treatment is informative of how the patient weighs the relative harms of a false-positive and a false-negative prediction. This theoretical relationship is then used to derive the net benefit of the model across different threshold probabilities. Plotting net benefit against threshold probability yields the "decision curve." The authors apply the method to models for the prediction of seminal vesicle invasion in prostate cancer patients. Decision curve analysis identified the range of threshold probabilities in which a model was of value, the magnitude of benefit, and which of several models was optimal. Decision curve analysis is a suitable method for evaluating alternative diagnostic and prognostic strategies that has advantages over other commonly used measures and techniques.

A systematic review of modelling approaches in economic evaluations of health interventions for drug and alcohol problems.

PubMed

Hoang, Van Phuong; Shanahan, Marian; Shukla, Nagesh; Perez, Pascal; Farrell, Michael; Ritter, Alison

2016-04-13

The overarching goal of health policies is to maximize health and societal benefits. Economic evaluations can play a vital role in assessing whether or not such benefits occur. This paper reviews the application of modelling techniques in economic evaluations of drug and alcohol interventions with regard to (i) modelling paradigms themselves; (ii) perspectives of costs and benefits and (iii) time frame. Papers that use modelling approaches for economic evaluations of drug and alcohol interventions were identified by carrying out searches of major databases. Thirty eight papers met the inclusion criteria. Overall, the cohort Markov models remain the most popular approach, followed by decision trees, Individual based model and System dynamics model (SD). Most of the papers adopted a long term time frame to reflect the long term costs and benefits of health interventions. However, it was fairly common among the reviewed papers to adopt a narrow perspective that only takes into account costs and benefits borne by the health care sector. This review paper informs policy makers about the availability of modelling techniques that can be used to enhance the quality of economic evaluations for drug and alcohol treatment interventions.

Airspace Concept Evaluation System (ACES), Concept Simulations using Communication, Navigation and Surveillance (CNS) System Models

NASA Technical Reports Server (NTRS)

Kubat, Greg; Vandrei, Don

2006-01-01

Project Objectives include: a) CNS Model Development; b Design/Integration of baseline set of CNS Models into ACES; c) Implement Enhanced Simulation Capabilities in ACES; d) Design and Integration of Enhanced (2nd set) CNS Models; and e) Continue with CNS Model Integration/Concept evaluations.

Models of Evaluation Utilization: A Meta-Modeling Synthesis of the Literature.

ERIC Educational Resources Information Center

Johnson, R. Burke

An integrative causal process model of evaluation utilization variables is presented. The model was developed through a traditional approach to literature review that lists results from published studies and relates these to the research topic, and through an approach that tries to integrate the models found in the literature search. Meta-modeling…

Advanced error diagnostics of the CMAQ and Chimere modelling systems within the AQMEII3 model evaluation framework

EPA Science Inventory

The work here complements the overview analysis of the modelling systems participating in the third phase of the Air Quality Model Evaluation International Initiative (AQMEII3) by focusing on the performance for hourly surface ozone by two modelling systems, Chimere for Europe an...

EVALUATING PREDICTIVE ERRORS OF A COMPLEX ENVIRONMENTAL MODEL USING A GENERAL LINEAR MODEL AND LEAST SQUARE MEANS

EPA Science Inventory

A General Linear Model (GLM) was used to evaluate the deviation of predicted values from expected values for a complex environmental model. For this demonstration, we used the default level interface of the Regional Mercury Cycling Model (R-MCM) to simulate epilimnetic total mer...

Evaluating bacterial gene-finding HMM structures as probabilistic logic programs.

PubMed

Mørk, Søren; Holmes, Ian

2012-03-01

Probabilistic logic programming offers a powerful way to describe and evaluate structured statistical models. To investigate the practicality of probabilistic logic programming for structure learning in bioinformatics, we undertook a simplified bacterial gene-finding benchmark in PRISM, a probabilistic dialect of Prolog. We evaluate Hidden Markov Model structures for bacterial protein-coding gene potential, including a simple null model structure, three structures based on existing bacterial gene finders and two novel model structures. We test standard versions as well as ADPH length modeling and three-state versions of the five model structures. The models are all represented as probabilistic logic programs and evaluated using the PRISM machine learning system in terms of statistical information criteria and gene-finding prediction accuracy, in two bacterial genomes. Neither of our implementations of the two currently most used model structures are best performing in terms of statistical information criteria or prediction performances, suggesting that better-fitting models might be achievable. The source code of all PRISM models, data and additional scripts are freely available for download at: http://github.com/somork/codonhmm. Supplementary data are available at Bioinformatics online.

An evaluation of the predictive performance of distributional models for flora and fauna in north-east New South Wales.

PubMed

Pearce, J; Ferrier, S; Scotts, D

2001-06-01

To use models of species distributions effectively in conservation planning, it is important to determine the predictive accuracy of such models. Extensive modelling of the distribution of vascular plant and vertebrate fauna species within north-east New South Wales has been undertaken by linking field survey data to environmental and geographical predictors using logistic regression. These models have been used in the development of a comprehensive and adequate reserve system within the region. We evaluate the predictive accuracy of models for 153 small reptile, arboreal marsupial, diurnal bird and vascular plant species for which independent evaluation data were available. The predictive performance of each model was evaluated using the relative operating characteristic curve to measure discrimination capacity. Good discrimination ability implies that a model's predictions provide an acceptable index of species occurrence. The discrimination capacity of 89% of the models was significantly better than random, with 70% of the models providing high levels of discrimination. Predictions generated by this type of modelling therefore provide a reasonably sound basis for regional conservation planning. The discrimination ability of models was highest for the less mobile biological groups, particularly the vascular plants and small reptiles. In the case of diurnal birds, poor performing models tended to be for species which occur mainly within specific habitats not well sampled by either the model development or evaluation data, highly mobile species, species that are locally nomadic or those that display very broad habitat requirements. Particular care needs to be exercised when employing models for these types of species in conservation planning.

Integrating Human Factors into Crew Exploration Vehicle (CEV) Design

NASA Technical Reports Server (NTRS)

Whitmore, Mihriban; Holden, Kritina; Baggerman, Susan; Campbell, Paul

2007-01-01

The purpose of this design process is to apply Human Engineering (HE) requirements and guidelines to hardware/software and to provide HE design, analysis and evaluation of crew interfaces. The topics include: 1) Background/Purpose; 2) HE Activities; 3) CASE STUDY: Net Habitable Volume (NHV) Study; 4) CASE STUDY: Human Modeling Approach; 5) CASE STUDY: Human Modeling Results; 6) CASE STUDY: Human Modeling Conclusions; 7) CASE STUDY: Human-in-the-Loop Evaluation Approach; 8) CASE STUDY: Unsuited Evaluation Results; 9) CASE STUDY: Suited Evaluation Results; 10) CASE STUDY: Human-in-the-Loop Evaluation Conclusions; 11) Near-Term Plan; and 12) In Conclusion

Core Professionalism Education in Surgery: A Systematic Review.

PubMed

Sarıoğlu Büke, Akile; Karabilgin Öztürkçü, Özlem Sürel; Yılmaz, Yusuf; Sayek, İskender

2018-03-15

Professionalism education is one of the major elements of surgical residency education. To evaluate the studies on core professionalism education programs in surgical professionalism education. Systematic review. This systematic literature review was performed to analyze core professionalism programs for surgical residency education published in English with at least three of the following features: program developmental model/instructional design method, aims and competencies, methods of teaching, methods of assessment, and program evaluation model or method. A total of 27083 articles were retrieved using EBSCOHOST, PubMed, Science Direct, Web of Science, and manual search. Eight articles met the selection criteria. The instructional design method was presented in only one article, which described the Analysis, Design, Development, Implementation, and Evaluation model. Six articles were based on the Accreditation Council for Graduate Medical Education criterion, although there was significant variability in content. The most common teaching method was role modeling with scenario- and case-based learning. A wide range of assessment methods for evaluating professionalism education were reported. The Kirkpatrick model was reported in one article as a method for program evaluation. It is suggested that for a core surgical professionalism education program, developmental/instructional design model, aims and competencies, content, teaching methods, assessment methods, and program evaluation methods/models should be well defined, and the content should be comparable.

ATAMM enhancement and multiprocessor performance evaluation

NASA Technical Reports Server (NTRS)

Stoughton, John W.; Mielke, Roland R.; Som, Sukhamoy; Obando, Rodrigo; Malekpour, Mahyar R.; Jones, Robert L., III; Mandala, Brij Mohan V.

1991-01-01

ATAMM (Algorithm To Architecture Mapping Model) enhancement and multiprocessor performance evaluation is discussed. The following topics are included: the ATAMM model; ATAMM enhancement; ADM (Advanced Development Model) implementation of ATAMM; and ATAMM support tools.

FRAMEWORK FOR EVALUATION OF PHYSIOLOGICALLY-BASED PHARMACOKINETIC MODELS FOR USE IN SAFETY OR RISK ASSESSMENT

EPA Science Inventory

ABSTRACT

Proposed applications of increasingly sophisticated biologically-based computational models, such as physiologically-based pharmacokinetic (PBPK) models, raise the issue of how to evaluate whether the models are adequate for proposed uses including safety or risk ...

The Isprs Benchmark on Indoor Modelling

NASA Astrophysics Data System (ADS)

Khoshelham, K.; Díaz Vilariño, L.; Peter, M.; Kang, Z.; Acharya, D.

2017-09-01

Automated generation of 3D indoor models from point cloud data has been a topic of intensive research in recent years. While results on various datasets have been reported in literature, a comparison of the performance of different methods has not been possible due to the lack of benchmark datasets and a common evaluation framework. The ISPRS benchmark on indoor modelling aims to address this issue by providing a public benchmark dataset and an evaluation framework for performance comparison of indoor modelling methods. In this paper, we present the benchmark dataset comprising several point clouds of indoor environments captured by different sensors. We also discuss the evaluation and comparison of indoor modelling methods based on manually created reference models and appropriate quality evaluation criteria. The benchmark dataset is available for download at: http://www2.isprs.org/commissions/comm4/wg5/benchmark-on-indoor-modelling.html.

[Study on the quantitative evaluation on the degree of TCM basic syndromes often encountered in patients with primary liver cancer].

PubMed

Li, Dong-tao; Ling, Chang-quan; Zhu, De-zeng

2007-07-01

To establish a quantitative model for evaluating the degree of the TCM basic syndromes often encountered in patients with primary liver cancer (PLC). Medical literatures concerning the clinical investigation and TCM syndrome of PLC were collected and analyzed adopting expert-composed symposium method, and the 100 millimeter scaling was applied in combining with scoring on degree of symptoms to establish a quantitative criterion for symptoms and signs degree classification in patients with PLC. Two models, i.e. the additive model and the additive-multiplicative model, were established by using comprehensive analytic hierarchy process (AHP) as the mathematical tool to estimate the weight of the criterion for evaluating basic syndromes in various layers by specialists. Then the two models were verified in clinical practice and the outcomes were compared with that fuzzy evaluated by specialists. Verification on 459 times/case of PLC showed that the coincidence rate between the outcomes derived from specialists with that from the additive model was 84.53 %, and with that from the additive-multificative model was 62.75 %, the difference between the two showed statistical significance (P<0.01). It could be decided that the additive model is the principle model suitable for quantitative evaluation on the degree of TCM basic syndromes in patients with PLC.

Two Decades of WRF/CMAQ simulations over the continental ...

EPA Pesticide Factsheets

Confidence in the application of models for forecasting and regulatory assessments is furthered by conducting four types of model evaluation: operational, dynamic, diagnostic, and probabilistic. Operational model evaluation alone does not reveal the confidence limits that can be associated with modeled air quality concentrations. This paper presents novel approaches for performing dynamic model evaluation and for evaluating the confidence limits of ozone exceedances using the WRF/CMAQ model simulations over the continental United States for the period from 1990 to 2010. The methodology presented here entails spectral decomposition of ozone time series using the KZ filter to assess the variations in the strengths of the synoptic (i.e., weather-induced variation) and baseline (i.e., long-term variation attributable to emissions, policy, and trends) forcings embedded in the modeled and observed concentrations. A method is presented where the future year observations are estimated based on the changes in the concentrations predicted by the model applied to the current year observations. The proposed method can provide confidence limits for ozone exceedances for a given emission reduction scenario. We present and discuss these new approaches to identify the strengths of the model in representing the changes in simulated O3 air quality over the 21-year period. The National Exposure Research Laboratory (NERL) Computational Exposure Division (CED) develops and evaluates

Abstraction and model evaluation in category learning.

PubMed

Vanpaemel, Wolf; Storms, Gert

2010-05-01

Thirty previously published data sets, from seminal category learning tasks, are reanalyzed using the varying abstraction model (VAM). Unlike a prototype-versus-exemplar analysis, which focuses on extreme levels of abstraction only, a VAM analysis also considers the possibility of partial abstraction. Whereas most data sets support no abstraction when only the extreme possibilities are considered, we show that evidence for abstraction can be provided using the broader view on abstraction provided by the VAM. The present results generalize earlier demonstrations of partial abstraction (Vanpaemel & Storms, 2008), in which only a small number of data sets was analyzed. Following the dominant modus operandi in category learning research, Vanpaemel and Storms evaluated the models on their best fit, a practice known to ignore the complexity of the models under consideration. In the present study, in contrast, model evaluation not only relies on the maximal likelihood, but also on the marginal likelihood, which is sensitive to model complexity. Finally, using a large recovery study, it is demonstrated that, across the 30 data sets, complexity differences between the models in the VAM family are small. This indicates that a (computationally challenging) complexity-sensitive model evaluation method is uncalled for, and that the use of a (computationally straightforward) complexity-insensitive model evaluation method is justified.

Climate Model Diagnostic Analyzer Web Service System

NASA Astrophysics Data System (ADS)

Lee, S.; Pan, L.; Zhai, C.; Tang, B.; Kubar, T. L.; Li, J.; Zhang, J.; Wang, W.

2015-12-01

Both the National Research Council Decadal Survey and the latest Intergovernmental Panel on Climate Change Assessment Report stressed the need for the comprehensive and innovative evaluation of climate models with the synergistic use of global satellite observations in order to improve our weather and climate simulation and prediction capabilities. The abundance of satellite observations for fundamental climate parameters and the availability of coordinated model outputs from CMIP5 for the same parameters offer a great opportunity to understand and diagnose model biases in climate models. In addition, the Obs4MIPs efforts have created several key global observational datasets that are readily usable for model evaluations. However, a model diagnostic evaluation process requires physics-based multi-variable comparisons that typically involve large-volume and heterogeneous datasets, making them both computationally- and data-intensive. In response, we have developed a novel methodology to diagnose model biases in contemporary climate models and implementing the methodology as a web-service based, cloud-enabled, provenance-supported climate-model evaluation system. The evaluation system is named Climate Model Diagnostic Analyzer (CMDA), which is the product of the research and technology development investments of several current and past NASA ROSES programs. The current technologies and infrastructure of CMDA are designed and selected to address several technical challenges that the Earth science modeling and model analysis community faces in evaluating and diagnosing climate models. In particular, we have three key technology components: (1) diagnostic analysis methodology; (2) web-service based, cloud-enabled technology; (3) provenance-supported technology. The diagnostic analysis methodology includes random forest feature importance ranking, conditional probability distribution function, conditional sampling, and time-lagged correlation map. We have implemented the new methodology as web services and incorporated the system into the Cloud. We have also developed a provenance management system for CMDA where CMDA service semantics modeling, service search and recommendation, and service execution history management are designed and implemented.

An evaluation of soil moisture models for countermine application

NASA Astrophysics Data System (ADS)

Mason, George L.

2004-09-01

The focus of this study is the evaluation of emerging soil moisture models as they apply to infrared, radar, and acoustic sensors within the scope of countermine operations. Physical, chemical, and biological processes changing the signature of the ground are considered. The available models were not run in-house, but were evaluated by the theory by which they were constructed and the supporting documentation. The study was conducted between September and October of 2003 and represents a subset of existing models. The objective was to identify those models suited for simulation, define the general constraints of the models, and summarize the emerging functionalities which would support sensor modeling for mine detection.

Integrated Assessment Model Evaluation

NASA Astrophysics Data System (ADS)

Smith, S. J.; Clarke, L.; Edmonds, J. A.; Weyant, J. P.

2012-12-01

Integrated assessment models of climate change (IAMs) are widely used to provide insights into the dynamics of the coupled human and socio-economic system, including emission mitigation analysis and the generation of future emission scenarios. Similar to the climate modeling community, the integrated assessment community has a two decade history of model inter-comparison, which has served as one of the primary venues for model evaluation and confirmation. While analysis of historical trends in the socio-economic system has long played a key role in diagnostics of future scenarios from IAMs, formal hindcast experiments are just now being contemplated as evaluation exercises. Some initial thoughts on setting up such IAM evaluation experiments are discussed. Socio-economic systems do not follow strict physical laws, which means that evaluation needs to take place in a context, unlike that of physical system models, in which there are few fixed, unchanging relationships. Of course strict validation of even earth system models is not possible (Oreskes etal 2004), a fact borne out by the inability of models to constrain the climate sensitivity. Energy-system models have also been grappling with some of the same questions over the last quarter century. For example, one of "the many questions in the energy field that are waiting for answers in the next 20 years" identified by Hans Landsberg in 1985 was "Will the price of oil resume its upward movement?" Of course we are still asking this question today. While, arguably, even fewer constraints apply to socio-economic systems, numerous historical trends and patterns have been identified, although often only in broad terms, that are used to guide the development of model components, parameter ranges, and scenario assumptions. IAM evaluation exercises are expected to provide useful information for interpreting model results and improving model behavior. A key step is the recognition of model boundaries, that is, what is inside and outside the IAM. All IAM projections to date are conditional on assumed inputs such as population dynamics and economic growth. A key part of evaluation exercises will be the substantial effort needed to develop the necessary historical datasets. Given the fundamentally uncertain characteristics of the socio-economic system, alternative formulations of the evaluation question may turn out to be useful. For example, is is likely useful to ask: how much needs to be specified on order to be able to reproduce historical trends to within a given accuracy? There is also a close, and fundamental, link between evaluation and diagnostic exercises that aim to evaluate the characteristics of future scenarios (rates of growth, technology diffusion, etc.) against historical behavior. These exercises are currently being conducted by individual groups due, in part, due to the large diversity if IAM designs and goals. While all climate models are, to first order, modeling the same system, boundary conditions, and physical laws, this is not true for IAMs. The structure, and even feasibility, of a hindcast-style evaluation exercise can be very different depending on the structure of each specific integrated assessment model.

Evaluation of Human and Anthropomorphic Test Device Finite Element Models under Spaceflight Loading Conditions

NASA Technical Reports Server (NTRS)

Putnam, Jacob P.; Untaroiu, Costin; Somers. Jeffrey

2014-01-01

In an effort to develop occupant protection standards for future multipurpose crew vehicles, the National Aeronautics and Space Administration (NASA) has looked to evaluate the test device for human occupant restraint with the modification kit (THOR-K) anthropomorphic test device (ATD) in relevant impact test scenarios. With the allowance and support of the National Highway Traffic Safety Administration, NASA has performed a series of sled impact tests on the latest developed THOR-K ATD. These tests were performed to match test conditions from human volunteer data previously collected by the U.S. Air Force. The objective of this study was to evaluate the THOR-K finite element (FE) model and the Total HUman Model for Safety (THUMS) FE model with respect to the tests performed. These models were evaluated in spinal and frontal impacts against kinematic and kinetic data recorded in ATD and human testing. Methods: The FE simulations were developed based on recorded pretest ATD/human position and sled acceleration pulses measured during testing. Predicted responses by both human and ATD models were compared to test data recorded under the same impact conditions. The kinematic responses of the models were quantitatively evaluated using the ISO-metric curve rating system. In addition, ATD injury criteria and human stress/strain data were calculated to evaluate the risk of injury predicted by the ATD and human model, respectively. Results: Preliminary results show well-correlated response between both FE models and their physical counterparts. In addition, predicted ATD injury criteria and human model stress/strain values are shown to positively relate. Kinematic comparison between human and ATD models indicates promising biofidelic response, although a slightly stiffer response is observed within the ATD. Conclusion: As a compliment to ATD testing, numerical simulation provides efficient means to assess vehicle safety throughout the design process and further improve the design of physical ATDs. The assessment of the THOR-K and THUMS FE models in a spaceflight testing condition is an essential first step to implementing these models in the computational evaluation of spacecraft occupant safety. Promising results suggest future use of these models in the aerospace field.

External evaluation of population pharmacokinetic models of vancomycin in neonates: the transferability of published models to different clinical settings

PubMed Central

Zhao, Wei; Kaguelidou, Florentia; Biran, Valérie; Zhang, Daolun; Allegaert, Karel; Capparelli, Edmund V; Holford, Nick; Kimura, Toshimi; Lo, Yoke-Lin; Peris, José-Esteban; Thomson, Alison; Anker, John N; Fakhoury, May; Jacqz-Aigrain, Evelyne

2013-01-01

Aims Vancomycin is one of the most evaluated antibiotics in neonates using modeling and simulation approaches. However no clear consensus on optimal dosing has been achieved. The objective of the present study was to perform an external evaluation of published models, in order to test their predictive performances in an independent dataset and to identify the possible study-related factors influencing the transferability of pharmacokinetic models to different clinical settings. Method Published neonatal vancomycin pharmacokinetic models were screened from the literature. The predictive performance of six models was evaluated using an independent dataset (112 concentrations from 78 neonates). The evaluation procedures used simulation-based diagnostics [visual predictive check (VPC) and normalized prediction distribution errors (NPDE)]. Results Differences in predictive performances of models for vancomycin pharmacokinetics in neonates were found. The mean of NPDE for six evaluated models were 1.35, −0.22, −0.36, 0.24, 0.66 and 0.48, respectively. These differences were explained, at least partly, by taking into account the method used to measure serum creatinine concentrations. The adult conversion factor of 1.3 (enzymatic to Jaffé) was tested with an improvement in the VPC and NPDE, but it still needs to be evaluated and validated in neonates. Differences were also identified between analytical methods for vancomycin. Conclusion The importance of analytical techniques for serum creatinine concentrations and vancomycin as predictors of vancomycin concentrations in neonates have been confirmed. Dosage individualization of vancomycin in neonates should consider not only patients' characteristics and clinical conditions, but also the methods used to measure serum creatinine and vancomycin. PMID:23148919

Using Student Perceptions of the Learning Environment to Evaluate the Effectiveness of a Teacher Professional Development Programme

ERIC Educational Resources Information Center

Soebari, Titien S.; Aldridge, Jill M.

2015-01-01

The focus of this article is two-fold. First, it describes a model that can be used to guide the evaluation of teacher professional development. The model combines important components of existing models and incorporates the use of students' perceptions for examining teacher change. Second, the article reports the evaluation of a teacher…

Kirkpatrick and Beyond: A Review of Models of Training Evaluation. IES Report.

ERIC Educational Resources Information Center

Tamkin, P.; Yarnall, J.; Kerrin, M.

Many organizations are not satisfied that their methods of evaluating training are rigorous or extensive enough to answer questions of value to them. Complaints about Kirkpatrick's popular four-step model (1959) of training evaluation are that each level is assumed to be associated with the previous and next levels and that the model is too simple…

An Application of the PMI Model at the Project Level Evaluation of ESEA Title IV-C Projects.

ERIC Educational Resources Information Center

McBeath, Marcia

All of the papers presented as part of a symposium concerned the application of the Planning, Monitoring, and Implementation Model (PMI) to the evaluation of the District of Columbia Public Schools' programs supported by the Elementary Secondary Education Act (ESEA) Title IV-C. PMI was developed to provide a model for systematic evaluation of…

Comparison of Development Test and Evaluation and Overall Program Estimate at Completion

DTIC Science & Technology

2011-03-01

of the overall model and parameter. In addition to 36 the Shapiro-Wilkes test , and Cook’s Distance overlay plot we used the Breusch - Pagan test to...Transformed Model Finally, we evaluated our log transformed model using the Breusch - Pagan test . The results return a value of .51, thus confirming our...COMPARISON OF DEVELOPMENT TEST AND EVALUATION AND OVERALL

An Evaluation of High School Curricula Employing Using the Element-Based Curriculum Development Model

ERIC Educational Resources Information Center

Aslan, Dolgun; Günay, Rafet

2016-01-01

This study was conducted with the aim of evaluating the curricula that constitute the basis of education provision at high schools in Turkey from the perspective of the teachers involved. A descriptive survey model, a quantitative research method was employed in this study. An item-based curriculum evaluation model was employed as part of the…

Using a Systematic Conceptual Model for a Process Evaluation of a Middle School Obesity Risk-Reduction Nutrition Curriculum Intervention: "Choice, Control & Change"

ERIC Educational Resources Information Center

Lee, Heewon; Contento, Isobel R.; Koch, Pamela

2013-01-01

Objective: To use and review a conceptual model of process evaluation and to examine the implementation of a nutrition education curriculum, "Choice, Control & Change", designed to promote dietary and physical activity behaviors that reduce obesity risk. Design: A process evaluation study based on a systematic conceptual model. Setting: Five…

A Generic Evaluation Model for Semantic Web Services

NASA Astrophysics Data System (ADS)

Shafiq, Omair

Semantic Web Services research has gained momentum over the last few Years and by now several realizations exist. They are being used in a number of industrial use-cases. Soon software developers will be expected to use this infrastructure to build their B2B applications requiring dynamic integration. However, there is still a lack of guidelines for the evaluation of tools developed to realize Semantic Web Services and applications built on top of them. In normal software engineering practice such guidelines can already be found for traditional component-based systems. Also some efforts are being made to build performance models for servicebased systems. Drawing on these related efforts in component-oriented and servicebased systems, we identified the need for a generic evaluation model for Semantic Web Services applicable to any realization. The generic evaluation model will help users and customers to orient their systems and solutions towards using Semantic Web Services. In this chapter, we have presented the requirements for the generic evaluation model for Semantic Web Services and further discussed the initial steps that we took to sketch such a model. Finally, we discuss related activities for evaluating semantic technologies.

Development of a program logic model and evaluation plan for a participatory ergonomics intervention in construction.

PubMed

Jaegers, Lisa; Dale, Ann Marie; Weaver, Nancy; Buchholz, Bryan; Welch, Laura; Evanoff, Bradley

2014-03-01

Intervention studies in participatory ergonomics (PE) are often difficult to interpret due to limited descriptions of program planning and evaluation. In an ongoing PE program with floor layers, we developed a logic model to describe our program plan, and process and summative evaluations designed to describe the efficacy of the program. The logic model was a useful tool for describing the program elements and subsequent modifications. The process evaluation measured how well the program was delivered as intended, and revealed the need for program modifications. The summative evaluation provided early measures of the efficacy of the program as delivered. Inadequate information on program delivery may lead to erroneous conclusions about intervention efficacy due to Type III error. A logic model guided the delivery and evaluation of our intervention and provides useful information to aid interpretation of results. © 2013 Wiley Periodicals, Inc.

Development of a Program Logic Model and Evaluation Plan for a Participatory Ergonomics Intervention in Construction

PubMed Central

Jaegers, Lisa; Dale, Ann Marie; Weaver, Nancy; Buchholz, Bryan; Welch, Laura; Evanoff, Bradley

2013-01-01

Background Intervention studies in participatory ergonomics (PE) are often difficult to interpret due to limited descriptions of program planning and evaluation. Methods In an ongoing PE program with floor layers, we developed a logic model to describe our program plan, and process and summative evaluations designed to describe the efficacy of the program. Results The logic model was a useful tool for describing the program elements and subsequent modifications. The process evaluation measured how well the program was delivered as intended, and revealed the need for program modifications. The summative evaluation provided early measures of the efficacy of the program as delivered. Conclusions Inadequate information on program delivery may lead to erroneous conclusions about intervention efficacy due to Type III error. A logic model guided the delivery and evaluation of our intervention and provides useful information to aid interpretation of results. PMID:24006097

Did you have an impact? A theory-based method for planning and evaluating knowledge-transfer and exchange activities in occupational health and safety.

PubMed

Kramer, Desré M; Wells, Richard P; Carlan, Nicolette; Aversa, Theresa; Bigelow, Philip P; Dixon, Shane M; McMillan, Keith

2013-01-01

Few evaluation tools are available to assess knowledge-transfer and exchange interventions. The objective of this paper is to develop and demonstrate a theory-based knowledge-transfer and exchange method of evaluation (KEME) that synthesizes 3 theoretical frameworks: the promoting action on research implementation of health services (PARiHS) model, the transtheoretical model of change, and a model of knowledge use. It proposes a new term, keme, to mean a unit of evidence-based transferable knowledge. The usefulness of the evaluation method is demonstrated with 4 occupational health and safety knowledge transfer and exchange (KTE) implementation case studies that are based upon the analysis of over 50 pre-existing interviews. The usefulness of the evaluation model has enabled us to better understand stakeholder feedback, frame our interpretation, and perform a more comprehensive evaluation of the knowledge use outcomes of our KTE efforts.

A systematic review of model-based economic evaluations of diagnostic and therapeutic strategies for lower extremity artery disease.

PubMed

Vaidya, Anil; Joore, Manuela A; ten Cate-Hoek, Arina J; Kleinegris, Marie-Claire; ten Cate, Hugo; Severens, Johan L

2014-01-01

Lower extremity artery disease (LEAD) is a sign of wide spread atherosclerosis also affecting coronary, cerebral and renal arteries and is associated with increased risk of cardiovascular events. Many economic evaluations have been published for LEAD due to its clinical, social and economic importance. The aim of this systematic review was to assess modelling methods used in published economic evaluations in the field of LEAD. Our review appraised and compared the general characteristics, model structure and methodological quality of published models. Electronic databases MEDLINE and EMBASE were searched until February 2013 via OVID interface. Cochrane database of systematic reviews, Health Technology Assessment database hosted by National Institute for Health research and National Health Services Economic Evaluation Database (NHSEED) were also searched. The methodological quality of the included studies was assessed by using the Philips' checklist. Sixteen model-based economic evaluations were identified and included. Eleven models compared therapeutic health technologies; three models compared diagnostic tests and two models compared a combination of diagnostic and therapeutic options for LEAD. Results of this systematic review revealed an acceptable to low methodological quality of the included studies. Methodological diversity and insufficient information posed a challenge for valid comparison of the included studies. In conclusion, there is a need for transparent, methodologically comparable and scientifically credible model-based economic evaluations in the field of LEAD. Future modelling studies should include clinically and economically important cardiovascular outcomes to reflect the wider impact of LEAD on individual patients and on the society.

Evaluating and Improving Cloud Processes in the Multi-Scale Modeling Framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ackerman, Thomas P.

2015-03-01

The research performed under this grant was intended to improve the embedded cloud model in the Multi-scale Modeling Framework (MMF) for convective clouds by using a 2-moment microphysics scheme rather than the single moment scheme used in all the MMF runs to date. The technical report and associated documents describe the results of testing the cloud resolving model with fixed boundary conditions and evaluation of model results with data. The overarching conclusion is that such model evaluations are problematic because errors in the forcing fields control the results so strongly that variations in parameterization values cannot be usefully constrained

Evaluation of the CORDEX-Africa multi-RCM hindcast: systematic model errors

NASA Astrophysics Data System (ADS)

Kim, J.; Waliser, Duane E.; Mattmann, Chris A.; Goodale, Cameron E.; Hart, Andrew F.; Zimdars, Paul A.; Crichton, Daniel J.; Jones, Colin; Nikulin, Grigory; Hewitson, Bruce; Jack, Chris; Lennard, Christopher; Favre, Alice

2014-03-01

Monthly-mean precipitation, mean (TAVG), maximum (TMAX) and minimum (TMIN) surface air temperatures, and cloudiness from the CORDEX-Africa regional climate model (RCM) hindcast experiment are evaluated for model skill and systematic biases. All RCMs simulate basic climatological features of these variables reasonably, but systematic biases also occur across these models. All RCMs show higher fidelity in simulating precipitation for the west part of Africa than for the east part, and for the tropics than for northern Sahara. Interannual variation in the wet season rainfall is better simulated for the western Sahel than for the Ethiopian Highlands. RCM skill is higher for TAVG and TMAX than for TMIN, and regionally, for the subtropics than for the tropics. RCM skill in simulating cloudiness is generally lower than for precipitation or temperatures. For all variables, multi-model ensemble (ENS) generally outperforms individual models included in ENS. An overarching conclusion in this study is that some model biases vary systematically for regions, variables, and metrics, posing difficulties in defining a single representative index to measure model fidelity, especially for constructing ENS. This is an important concern in climate change impact assessment studies because most assessment models are run for specific regions/sectors with forcing data derived from model outputs. Thus, model evaluation and ENS construction must be performed separately for regions, variables, and metrics as required by specific analysis and/or assessments. Evaluations using multiple reference datasets reveal that cross-examination, quality control, and uncertainty estimates of reference data are crucial in model evaluations.

A general Bayesian framework for calibrating and evaluating stochastic models of annual multi-site hydrological data

NASA Astrophysics Data System (ADS)

Frost, Andrew J.; Thyer, Mark A.; Srikanthan, R.; Kuczera, George

2007-07-01

SummaryMulti-site simulation of hydrological data are required for drought risk assessment of large multi-reservoir water supply systems. In this paper, a general Bayesian framework is presented for the calibration and evaluation of multi-site hydrological data at annual timescales. Models included within this framework are the hidden Markov model (HMM) and the widely used lag-1 autoregressive (AR(1)) model. These models are extended by the inclusion of a Box-Cox transformation and a spatial correlation function in a multi-site setting. Parameter uncertainty is evaluated using Markov chain Monte Carlo techniques. Models are evaluated by their ability to reproduce a range of important extreme statistics and compared using Bayesian model selection techniques which evaluate model probabilities. The case study, using multi-site annual rainfall data situated within catchments which contribute to Sydney's main water supply, provided the following results: Firstly, in terms of model probabilities and diagnostics, the inclusion of the Box-Cox transformation was preferred. Secondly the AR(1) and HMM performed similarly, while some other proposed AR(1)/HMM models with regionally pooled parameters had greater posterior probability than these two models. The practical significance of parameter and model uncertainty was illustrated using a case study involving drought security analysis for urban water supply. It was shown that ignoring parameter uncertainty resulted in a significant overestimate of reservoir yield and an underestimation of system vulnerability to severe drought.

Computationally inexpensive identification of noninformative model parameters by sequential screening

NASA Astrophysics Data System (ADS)

Cuntz, Matthias; Mai, Juliane; Zink, Matthias; Thober, Stephan; Kumar, Rohini; Schäfer, David; Schrön, Martin; Craven, John; Rakovec, Oldrich; Spieler, Diana; Prykhodko, Vladyslav; Dalmasso, Giovanni; Musuuza, Jude; Langenberg, Ben; Attinger, Sabine; Samaniego, Luis

2015-08-01

Environmental models tend to require increasing computational time and resources as physical process descriptions are improved or new descriptions are incorporated. Many-query applications such as sensitivity analysis or model calibration usually require a large number of model evaluations leading to high computational demand. This often limits the feasibility of rigorous analyses. Here we present a fully automated sequential screening method that selects only informative parameters for a given model output. The method requires a number of model evaluations that is approximately 10 times the number of model parameters. It was tested using the mesoscale hydrologic model mHM in three hydrologically unique European river catchments. It identified around 20 informative parameters out of 52, with different informative parameters in each catchment. The screening method was evaluated with subsequent analyses using all 52 as well as only the informative parameters. Subsequent Sobol's global sensitivity analysis led to almost identical results yet required 40% fewer model evaluations after screening. mHM was calibrated with all and with only informative parameters in the three catchments. Model performances for daily discharge were equally high in both cases with Nash-Sutcliffe efficiencies above 0.82. Calibration using only the informative parameters needed just one third of the number of model evaluations. The universality of the sequential screening method was demonstrated using several general test functions from the literature. We therefore recommend the use of the computationally inexpensive sequential screening method prior to rigorous analyses on complex environmental models.

Computationally inexpensive identification of noninformative model parameters by sequential screening

NASA Astrophysics Data System (ADS)

Mai, Juliane; Cuntz, Matthias; Zink, Matthias; Thober, Stephan; Kumar, Rohini; Schäfer, David; Schrön, Martin; Craven, John; Rakovec, Oldrich; Spieler, Diana; Prykhodko, Vladyslav; Dalmasso, Giovanni; Musuuza, Jude; Langenberg, Ben; Attinger, Sabine; Samaniego, Luis

2016-04-01

Environmental models tend to require increasing computational time and resources as physical process descriptions are improved or new descriptions are incorporated. Many-query applications such as sensitivity analysis or model calibration usually require a large number of model evaluations leading to high computational demand. This often limits the feasibility of rigorous analyses. Here we present a fully automated sequential screening method that selects only informative parameters for a given model output. The method requires a number of model evaluations that is approximately 10 times the number of model parameters. It was tested using the mesoscale hydrologic model mHM in three hydrologically unique European river catchments. It identified around 20 informative parameters out of 52, with different informative parameters in each catchment. The screening method was evaluated with subsequent analyses using all 52 as well as only the informative parameters. Subsequent Sobol's global sensitivity analysis led to almost identical results yet required 40% fewer model evaluations after screening. mHM was calibrated with all and with only informative parameters in the three catchments. Model performances for daily discharge were equally high in both cases with Nash-Sutcliffe efficiencies above 0.82. Calibration using only the informative parameters needed just one third of the number of model evaluations. The universality of the sequential screening method was demonstrated using several general test functions from the literature. We therefore recommend the use of the computationally inexpensive sequential screening method prior to rigorous analyses on complex environmental models.

Presenting an evaluation model of the trauma registry software.

PubMed

Asadi, Farkhondeh; Paydar, Somayeh

2018-04-01

Trauma is a major cause of 10% death in the worldwide and is considered as a global concern. This problem has made healthcare policy makers and managers to adopt a basic strategy in this context. Trauma registry has an important and basic role in decreasing the mortality and the disabilities due to injuries resulted from trauma. Today, different software are designed for trauma registry. Evaluation of this software improves management, increases efficiency and effectiveness of these systems. Therefore, the aim of this study is to present an evaluation model for trauma registry software. The present study is an applied research. In this study, general and specific criteria of trauma registry software were identified by reviewing literature including books, articles, scientific documents, valid websites and related software in this domain. According to general and specific criteria and related software, a model for evaluating trauma registry software was proposed. Based on the proposed model, a checklist designed and its validity and reliability evaluated. Mentioned model by using of the Delphi technique presented to 12 experts and specialists. To analyze the results, an agreed coefficient of %75 was determined in order to apply changes. Finally, when the model was approved by the experts and professionals, the final version of the evaluation model for the trauma registry software was presented. For evaluating of criteria of trauma registry software, two groups were presented: 1- General criteria, 2- Specific criteria. General criteria of trauma registry software were classified into four main categories including: 1- usability, 2- security, 3- maintainability, and 4-interoperability. Specific criteria were divided into four main categories including: 1- data submission and entry, 2- reporting, 3- quality control, 4- decision and research support. The presented model in this research has introduced important general and specific criteria of trauma registry software and sub criteria related to each main criteria separately. This model was validated by experts in this field. Therefore, this model can be used as a comprehensive model and a standard evaluation tool for measuring efficiency and effectiveness and performance improvement of trauma registry software. Copyright © 2018 Elsevier B.V. All rights reserved.

Operational model evaluation for particulate matter in Europe and North America in the context of AQMEII

NASA Astrophysics Data System (ADS)

Solazzo, Efisio; Bianconi, Roberto; Pirovano, Guido; Matthias, Volker; Vautard, Robert; Moran, Michael D.; Wyat Appel, K.; Bessagnet, Bertrand; Brandt, Jørgen; Christensen, Jesper H.; Chemel, Charles; Coll, Isabelle; Ferreira, Joana; Forkel, Renate; Francis, Xavier V.; Grell, Georg; Grossi, Paola; Hansen, Ayoe B.; Miranda, Ana Isabel; Nopmongcol, Uarporn; Prank, Marje; Sartelet, Karine N.; Schaap, Martijn; Silver, Jeremy D.; Sokhi, Ranjeet S.; Vira, Julius; Werhahn, Johannes; Wolke, Ralf; Yarwood, Greg; Zhang, Junhua; Rao, S. Trivikrama; Galmarini, Stefano

2012-06-01

Ten state-of-the-science regional air quality (AQ) modeling systems have been applied to continental-scale domains in North America and Europe for full-year simulations of 2006 in the context of Air Quality Model Evaluation International Initiative (AQMEII), whose main goals are model inter-comparison and evaluation. Standardised modeling outputs from each group have been shared on the web-distributed ENSEMBLE system, which allows statistical and ensemble analyses to be performed. In this study, the one-year model simulations are inter-compared and evaluated with a large set of observations for ground-level particulate matter (PM10 and PM2.5) and its chemical components. Modeled concentrations of gaseous PM precursors, SO2 and NO2, have also been evaluated against observational data for both continents. Furthermore, modeled deposition (dry and wet) and emissions of several species relevant to PM are also inter-compared. The unprecedented scale of the exercise (two continents, one full year, fifteen modeling groups) allows for a detailed description of AQ model skill and uncertainty with respect to PM. Analyses of PM10 yearly time series and mean diurnal cycle show a large underestimation throughout the year for the AQ models included in AQMEII. The possible causes of PM bias, including errors in the emissions and meteorological inputs (e.g., wind speed and precipitation), and the calculated deposition are investigated. Further analysis of the coarse PM components, PM2.5 and its major components (SO4, NH4, NO3, elemental carbon), have also been performed, and the model performance for each component evaluated against measurements. Finally, the ability of the models to capture high PM concentrations has been evaluated by examining two separate PM2.5 episodes in Europe and North America. A large variability among models in predicting emissions, deposition, and concentration of PM and its precursors during the episodes has been found. Major challenges still remain with regards to identifying and eliminating the sources of PM bias in the models. Although PM2.5 was found to be much better estimated by the models than PM10, no model was found to consistently match the observations for all locations throughout the entire year.

Educator Evaluation and the Impact on Teacher Effectiveness

ERIC Educational Resources Information Center

Carreiro, Diane M.

2017-01-01

Educator evaluation is described in the literature as those systems in place used to supervise educator excellence as well as to maximize and foster teacher capacity. There have been many changes within the last five years in the Massachusetts educator evaluation model, now called the Massachusetts Model System for Educator Evaluation. Once…

Peer Evaluation of Teaching in an Online Information Literacy Course

ERIC Educational Resources Information Center

Vega García, Susan A.; Stacy-Bates, Kristine K.; Alger, Jeff; Marupova, Rano

2017-01-01

This paper reports on the development and implementation of a process of peer evaluation of teaching to assess librarian instruction in a high-enrollment online information literacy course for undergraduates. This paper also traces a shift within libraries from peer coaching to peer evaluation models. One common model for peer evaluation, using…

A Model for Administrative Evaluation by Subordinates.

ERIC Educational Resources Information Center

Budig, Jeanne E.

Under the administrator evaluation program adopted at Vincennes University, all faculty and professional staff are invited to evaluate each administrator above them in the chain of command. Originally based on the Purdue University "cafeteria" system, this evaluation model has been used biannually for 10 years. In an effort to simplify the system,…

Dig into Learning: A Program Evaluation of an Agricultural Literacy Innovation

ERIC Educational Resources Information Center

Edwards, Erica Brown

2016-01-01

This study is a mixed-methods program evaluation of an agricultural literacy innovation in a local school district in rural eastern North Carolina. This evaluation describes the use of a theory-based framework, the Concerns-Based Adoption Model (CBAM), in accordance with Stufflebeam's Context, Input, Process, Product (CIPP) model by evaluating the…

Evaluation and Dissemination of the Electrical Power Engineering Technology Curriculum Model. Final Report.

ERIC Educational Resources Information Center

McNeill, Perry R.; And Others

Described is a project initiated to evaluate and disseminate the Electrical Power Engineering Technology Curriculum developed at Oklahoma State University. The objective of the evaluation phase, to have the original model curriculum evaluated by both present and potential employers, was accomplished in a two-day workshop with participation of…

Evaluating the Assessment Models for Young Children with Special Needs in Taiwan

ERIC Educational Resources Information Center

Ho, Hua-Kuo

2009-01-01

The purpose of this study was intended to evaluate the assessment models of two representative centers of team evaluation for children's development in Taiwan. Documentary analysis and phone interview were employed in the study to collect the research data needed. Two centers of team evaluation for children's development were selected and…

Evaluation of Turkish and Mathematics Curricula According to Value-Based Evaluation Model

ERIC Educational Resources Information Center

Duman, Serap Nur; Akbas, Oktay

2017-01-01

This study evaluated secondary school seventh-grade Turkish and mathematics programs using the Context-Input-Process-Product Evaluation Model based on student, teacher, and inspector views. The convergent parallel mixed method design was used in the study. Student values were identified using the scales for socio-level identification, traditional…

Evaluating Special Educator Effectiveness: Addressing Issues Inherent to Value-Added Modeling

ERIC Educational Resources Information Center

Steinbrecher, Trisha D.; Selig, James P.; Cosbey, Joanna; Thorstensen, Beata I.

2014-01-01

States are increasingly using value-added approaches to evaluate teacher effectiveness. There is much debate regarding whether these methods should be employed and, if employed, what role such methods should play in comprehensive teacher evaluation systems. In this article, we consider the use of value-added modeling (VAM) to evaluate special…

Evaluation of Career Guidance Programs: Models, Methods, and Microcomputers. Information Series No. 317.

ERIC Educational Resources Information Center

Crites, John O.

Evaluating the effectiveness of career guidance programs is a complex process, and few comprehensive models for evaluating such programs exist. Evaluation of career guidance programs has been hampered by the myth that program outcomes are uniform and monolithic. Findings from studies of attribute treatment interactions have revealed only a few…

An Evaluation of the Private High School Curriculum in Turkey

ERIC Educational Resources Information Center

Aslan, Dolgun

2016-01-01

This study aims at evaluating curricula of private high schools in line with opinions of teachers working at the related high schools, and identifying any related problems. Screening model is used as a quantitative research method in the study. The "element-based curriculum evaluation model" is taken as basis for evaluation of the…

State of the Art Methodology for the Design and Analysis of Future Large Scale Evaluations: A Selective Examination.

ERIC Educational Resources Information Center

Burstein, Leigh

Two specific methods of analysis in large-scale evaluations are considered: structural equation modeling and selection modeling/analysis of non-equivalent control group designs. Their utility in large-scale educational program evaluation is discussed. The examination of these methodological developments indicates how people (evaluators,…

INVERSE MODEL ESTIMATION AND EVALUATION OF SEASONAL NH 3 EMISSIONS

EPA Science Inventory

The presentation topic is inverse modeling for estimate and evaluation of emissions. The case study presented is the need for seasonal estimates of NH₃ emissions for air quality modeling. The inverse modeling application approach is first described, and then the NH

Organisational Interoperability: Evaluation and Further Development of the OIM Model

DTIC Science & Technology

2003-06-01

an Organizational Interoperability Maturity Model (OIM) to evaluate interoperability at the organizational level. The OIM considers the human ... activity aspects of military operations, which are not covered in other models. This paper describes how the model has been used to identify problems and to

Review of Airport Ground Traffic Models Including an Evaluation of the ASTS Computer Program

DOT National Transportation Integrated Search

1972-12-01

The report covers an evaluation of Airport Ground Traffic models for the purpose of simulating an Autonomous Local Intersection Controller. All known models were reviewed and a detailed study was performed on the two in-house models the ASTS and ROSS...

Application of Wavelet Filters in an Evaluation of Photochemical Model Performance

EPA Science Inventory

Air quality model evaluation can be enhanced with time-scale specific comparisons of outputs and observations. For example, high-frequency (hours to one day) time scale information in observed ozone is not well captured by deterministic models and its incorporation into model pe...

iFlorida model deployment final evaluation report.

DOT National Transportation Integrated Search

2009-01-01

This document is the final report for the evaluation of the USDOT-sponsored Surface Transportation Security and Reliability Information System Model Deployment, or iFlorida Model Deployment. This report discusses findings in the following areas: ITS ...

iFlorida model deployment final evaluation report

DOT National Transportation Integrated Search

2009-01-01

This document is the final report for the evaluation of the USDOT-sponsored Surface Transportation Security and Reliability Information System Model Deployment, or iFlorida Model Deployment. This report discusses findings in the following areas: ITS ...

Evaluation of air traffic control models and simulations.

DOT National Transportation Integrated Search

1971-06-01

Approximately two hundred reports were identified as describing Air Traffic Control (ATC) modeling and simulation efforts. Of these, about ninety analytical and simulation models dealing with virtually all aspects of ATC were formally evaluated. The ...

Model of Auctioneer Estimation of Swordtip Squid (Loligo edulis) Quality

NASA Astrophysics Data System (ADS)

Nakamura, Makoto; Matsumoto, Keisuke; Morimoto, Eiji; Ezoe, Satoru; Maeda, Toshimichi; Hirano, Takayuki

The knowledge of experienced auctioneers regarding the circulation of marine products is an essential skill and is necessary for evaluating product quality and managing aspects such as freshness. In the present study, the ability of an auctioneer to quickly evaluate the freshness of swordtip squid (Loligo edulis) at fish markets was analyzed. Evaluation characteristics used by an auctioneer were analyzed and developed using a fuzzy logic model. Forty boxes containing 247 swordtip squid with mantles measuring 220 mm that had been evaluated and assigned to one of five quality categories by an auctioneer were used for the analysis and the modeling. The relationships between the evaluations of appearance, body color, and muscle freshness were statistically analyzed. It was found that a total of four indexes of the epidermis color strongly reflected evaluations of appearance: dispersion ratio of the head, chroma on the head-end mantle and the difference in the chroma and brightness of the mantle. The fuzzy logic model used these indexes for the antecedent-part of the linguistic rules. The results of both simulation and evaluations demonstrate that the model is robust, with the predicted results corresponding with more than 96% of the quality assignments of the auctioneers.

Best practices for evaluating the capability of nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for damage characterization

NASA Astrophysics Data System (ADS)

Aldrin, John C.; Annis, Charles; Sabbagh, Harold A.; Lindgren, Eric A.

2016-02-01

A comprehensive approach to NDE and SHM characterization error (CE) evaluation is presented that follows the framework of the `ahat-versus-a' regression analysis for POD assessment. Characterization capability evaluation is typically more complex with respect to current POD evaluations and thus requires engineering and statistical expertise in the model-building process to ensure all key effects and interactions are addressed. Justifying the statistical model choice with underlying assumptions is key. Several sizing case studies are presented with detailed evaluations of the most appropriate statistical model for each data set. The use of a model-assisted approach is introduced to help assess the reliability of NDE and SHM characterization capability under a wide range of part, environmental and damage conditions. Best practices of using models are presented for both an eddy current NDE sizing and vibration-based SHM case studies. The results of these studies highlight the general protocol feasibility, emphasize the importance of evaluating key application characteristics prior to the study, and demonstrate an approach to quantify the role of varying SHM sensor durability and environmental conditions on characterization performance.

EvalPartners: Facilitating the Development of a New Model of Voluntary Organization for Professional Evaluation to Support the Development of National Evaluation Capacities

ERIC Educational Resources Information Center

Kosheleva, Natalia; Segone, Marco

2013-01-01

In many less developed democracies Voluntary Organizations for Professional Evaluation (VOPEs) face the challenges of low demand for evaluation and the resulting low economic capacity of national evaluation communities. The VOPE model that evolved in well-developed democracies is not directly applicable under these circumstances, so a new model…

Beyond the buildingcentric approach: A vision for an integrated evaluation of sustainable buildings

DOE Office of Scientific and Technical Information (OSTI.GOV)

Conte, Emilia, E-mail: conte@poliba.it; Monno, Valeria, E-mail: vmonno@poliba.it

2012-04-15

The available sustainable building evaluation systems have produced a new environmental design paradigm. However, there is an increasing need to overcome the buildingcentric approach of these systems, in order to further exploit their innovate potential for sustainable building practices. The paper takes this challenge by developing a cross-scale evaluation approach focusing on the reliability of sustainable building design solutions for the context in which the building is situated. An integrated building-urban evaluation model is proposed based on the urban matrix, which is a conceptualisation of the built environment as a social-ecological system. The model aims at evaluating the sustainability ofmore » a building considering it as an active entity contributing to the resilience of the urban matrix. Few holistic performance indicators are used for evaluating such contribution, so expressing the building reliability. The discussion on the efficacy of the model shows that it works as a heuristic tool, supporting the acquisition of a better insight into the complexity which characterises the relationships between the building and the built environment sustainability. Shading new lights on the meaning of sustainable buildings, the model can play a positive role in innovating sustainable building design practices, thus complementing current evaluation systems. - Highlights: Black-Right-Pointing-Pointer We model an integrated building-urban evaluation approach. Black-Right-Pointing-Pointer The urban matrix represents the social-ecological functioning of the urban context. Black-Right-Pointing-Pointer We introduce the concept of reliability to evaluate sustainable buildings. Black-Right-Pointing-Pointer Holistic indicators express the building reliability. Black-Right-Pointing-Pointer The evaluation model works as heuristic tool and complements other tools.« less

Metaphors, models and organisational ethics in health care

PubMed Central

McCrickerd, J.

2000-01-01

Crucial to discussions in organisational ethics is an evaluation of the metaphors and models we use to understand the organisations we are discussing. I briefly defend this contention and evaluate three possible models: the current corporate model, an orchestrator model which puts hospitals in the same class as malls and airports, and a community model. I argue that the corporate and orchestrator model push to the background some important organisational ethics issues and bias us inappropriately towards certain solutions. Furthermore, I argue that the community model allows these to be more easily brought up. I also respond to the likely challenge that hospitals really are corporations by arguing that this is not relevant to evaluations of the appropriateness of the corporate model. Key Words: Metaphor • model • organisational ethics • health care ethics PMID:11055036

Improving the Impact and Implementation of Disaster Education: Programs for Children Through Theory-Based Evaluation.

PubMed

Johnson, Victoria A; Ronan, Kevin R; Johnston, David M; Peace, Robin

2016-11-01

A main weakness in the evaluation of disaster education programs for children is evaluators' propensity to judge program effectiveness based on changes in children's knowledge. Few studies have articulated an explicit program theory of how children's education would achieve desired outcomes and impacts related to disaster risk reduction in households and communities. This article describes the advantages of constructing program theory models for the purpose of evaluating disaster education programs for children. Following a review of some potential frameworks for program theory development, including the logic model, the program theory matrix, and the stage step model, the article provides working examples of these frameworks. The first example is the development of a program theory matrix used in an evaluation of ShakeOut, an earthquake drill practiced in two Washington State school districts. The model illustrates a theory of action; specifically, the effectiveness of school earthquake drills in preventing injuries and deaths during disasters. The second example is the development of a stage step model used for a process evaluation of What's the Plan Stan?, a voluntary teaching resource distributed to all New Zealand primary schools for curricular integration of disaster education. The model illustrates a theory of use; specifically, expanding the reach of disaster education for children through increased promotion of the resource. The process of developing the program theory models for the purpose of evaluation planning is discussed, as well as the advantages and shortcomings of the theory-based approaches. © 2015 Society for Risk Analysis.

Evaluation of a black-footed ferret resource utilization function model

USGS Publications Warehouse

Eads, D.A.; Millspaugh, J.J.; Biggins, D.E.; Jachowski, D.S.; Livieri, T.M.

2011-01-01

Resource utilization function (RUF) models permit evaluation of potential habitat for endangered species; ideally such models should be evaluated before use in management decision-making. We evaluated the predictive capabilities of a previously developed black-footed ferret (Mustela nigripes) RUF. Using the population-level RUF, generated from ferret observations at an adjacent yet distinct colony, we predicted the distribution of ferrets within a black-tailed prairie dog (Cynomys ludovicianus) colony in the Conata Basin, South Dakota, USA. We evaluated model performance, using data collected during post-breeding spotlight surveys (2007-2008) by assessing model agreement via weighted compositional analysis and count-metrics. Compositional analysis of home range use and colony-level availability, and core area use and home range availability, demonstrated ferret selection of the predicted Very high and High occurrence categories in 2007 and 2008. Simple count-metrics corroborated these findings and suggested selection of the Very high category in 2007 and the Very high and High categories in 2008. Collectively, these results suggested that the RUF was useful in predicting occurrence and intensity of space use of ferrets at our study site, the 2 objectives of the RUF. Application of this validated RUF would increase the resolution of habitat evaluations, permitting prediction of the distribution of ferrets within distinct colonies. Additional model evaluation at other sites, on other black-tailed prairie dog colonies of varying resource configuration and size, would increase understanding of influences upon model performance and the general utility of the RUF. ?? 2011 The Wildlife Society.

Effects of sample survey design on the accuracy of classification tree models in species distribution models

USGS Publications Warehouse

Edwards, T.C.; Cutler, D.R.; Zimmermann, N.E.; Geiser, L.; Moisen, Gretchen G.

2006-01-01

We evaluated the effects of probabilistic (hereafter DESIGN) and non-probabilistic (PURPOSIVE) sample surveys on resultant classification tree models for predicting the presence of four lichen species in the Pacific Northwest, USA. Models derived from both survey forms were assessed using an independent data set (EVALUATION). Measures of accuracy as gauged by resubstitution rates were similar for each lichen species irrespective of the underlying sample survey form. Cross-validation estimates of prediction accuracies were lower than resubstitution accuracies for all species and both design types, and in all cases were closer to the true prediction accuracies based on the EVALUATION data set. We argue that greater emphasis should be placed on calculating and reporting cross-validation accuracy rates rather than simple resubstitution accuracy rates. Evaluation of the DESIGN and PURPOSIVE tree models on the EVALUATION data set shows significantly lower prediction accuracy for the PURPOSIVE tree models relative to the DESIGN models, indicating that non-probabilistic sample surveys may generate models with limited predictive capability. These differences were consistent across all four lichen species, with 11 of the 12 possible species and sample survey type comparisons having significantly lower accuracy rates. Some differences in accuracy were as large as 50%. The classification tree structures also differed considerably both among and within the modelled species, depending on the sample survey form. Overlap in the predictor variables selected by the DESIGN and PURPOSIVE tree models ranged from only 20% to 38%, indicating the classification trees fit the two evaluated survey forms on different sets of predictor variables. The magnitude of these differences in predictor variables throws doubt on ecological interpretation derived from prediction models based on non-probabilistic sample surveys. ?? 2006 Elsevier B.V. All rights reserved.

Model description and evaluation of the mark-recapture survival model used to parameterize the 2012 status and threats analysis for the Florida manatee (Trichechus manatus latirostris)

USGS Publications Warehouse

Langtimm, Catherine A.; Kendall, William L.; Beck, Cathy A.; Kochman, Howard I.; Teague, Amy L.; Meigs-Friend, Gaia; Peñaloza, Claudia L.

2016-11-30

This report provides supporting details and evidence for the rationale, validity and efficacy of a new mark-recapture model, the Barker Robust Design, to estimate regional manatee survival rates used to parameterize several components of the 2012 version of the Manatee Core Biological Model (CBM) and Threats Analysis (TA).  The CBM and TA provide scientific analyses on population viability of the Florida manatee subspecies (Trichechus manatus latirostris) for U.S. Fish and Wildlife Service’s 5-year reviews of the status of the species as listed under the Endangered Species Act.  The model evaluation is presented in a standardized reporting framework, modified from the TRACE (TRAnsparent and Comprehensive model Evaluation) protocol first introduced for environmental threat analyses.  We identify this new protocol as TRACE-MANATEE SURVIVAL and this model evaluation specifically as TRACE-MANATEE SURVIVAL, Barker RD version 1. The longer-term objectives of the manatee standard reporting format are to (1) communicate to resource managers consistent evaluation information over sequential modeling efforts; (2) build understanding and expertise on the structure and function of the models; (3) document changes in model structures and applications in response to evolving management objectives, new biological and ecological knowledge, and new statistical advances; and (4) provide greater transparency for management and research review.

Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland.

PubMed

Kulczycki, Emanuel; Rozkosz, Ewa A

2017-01-01

This article discusses the Polish Journal Ranking, which is used in the research evaluation system in Poland. In 2015, the ranking, which represents all disciplines, allocated 17,437 journals into three lists: A, B, and C. The B list constitutes a ranking of Polish journals that are indexed neither in the Web of Science nor the European Reference Index for the Humanities. This ranking was built by evaluating journals in three dimensions: formal, bibliometric, and expert-based. We have analysed data on 2035 Polish journals from the B list. Our study aims to determine how an expert-based evaluation influenced the results of final evaluation. In our study, we used structural equation modelling, which is regression based, and we designed three pairs of theoretical models for three fields of science: (1) humanities, (2) social sciences, and (3) engineering, natural sciences, and medical sciences. Each pair consisted of the full model and the reduced model (i.e., the model without the expert-based evaluation). Our analysis revealed that the multidimensional evaluation of local journals should not rely only on the bibliometric indicators, which are based on the Web of Science or Scopus. Moreover, we have shown that the expert-based evaluation plays a major role in all fields of science. We conclude with recommendations that the formal evaluation should be reduced to verifiable parameters and that the expert-based evaluation should be based on common guidelines for the experts.

Modifying climate change habitat models using tree species-specific assessments of model uncertainty and life history-factors

Treesearch

Stephen N. Matthews; Louis R. Iverson; Anantha M. Prasad; Matthew P. Peters; Paul G. Rodewald

2011-01-01

Species distribution models (SDMs) to evaluate trees' potential responses to climate change are essential for developing appropriate forest management strategies. However, there is a great need to better understand these models' limitations and evaluate their uncertainties. We have previously developed statistical models of suitable habitat, based on both...

Development of a Logic Model to Guide Evaluations of the ASCA National Model for School Counseling Programs

ERIC Educational Resources Information Center

Martin, Ian; Carey, John

2014-01-01

A logic model was developed based on an analysis of the 2012 American School Counselor Association (ASCA) National Model in order to provide direction for program evaluation initiatives. The logic model identified three outcomes (increased student achievement/gap reduction, increased school counseling program resources, and systemic change and…

A systematic and critical review of model-based economic evaluations of pharmacotherapeutics in patients with bipolar disorder.

PubMed

Mohiuddin, Syed

2014-08-01

Bipolar disorder (BD) is a chronic and relapsing mental illness with a considerable health-related and economic burden. The primary goal of pharmacotherapeutics for BD is to improve patients' well-being. The use of decision-analytic models is key in assessing the added value of the pharmacotherapeutics aimed at treating the illness, but concerns have been expressed about the appropriateness of different modelling techniques and about the transparency in the reporting of economic evaluations. This paper aimed to identify and critically appraise published model-based economic evaluations of pharmacotherapeutics in BD patients. A systematic review combining common terms for BD and economic evaluation was conducted in MEDLINE, EMBASE, PSYCINFO and ECONLIT. Studies identified were summarised and critically appraised in terms of the use of modelling technique, model structure and data sources. Considering the prognosis and management of BD, the possible benefits and limitations of each modelling technique are discussed. Fourteen studies were identified using model-based economic evaluations of pharmacotherapeutics in BD patients. Of these 14 studies, nine used Markov, three used discrete-event simulation (DES) and two used decision-tree models. Most of the studies (n = 11) did not include the rationale for the choice of modelling technique undertaken. Half of the studies did not include the risk of mortality. Surprisingly, no study considered the risk of having a mixed bipolar episode. This review identified various modelling issues that could potentially reduce the comparability of one pharmacotherapeutic intervention with another. Better use and reporting of the modelling techniques in the future studies are essential. DES modelling appears to be a flexible and comprehensive technique for evaluating the comparability of BD treatment options because of its greater flexibility of depicting the disease progression over time. However, depending on the research question, modelling techniques other than DES might also be appropriate in some cases.

R&D for computational cognitive and social models : foundations for model evaluation through verification and validation (final LDRD report).

DOE Office of Scientific and Technical Information (OSTI.GOV)

Slepoy, Alexander; Mitchell, Scott A.; Backus, George A.

2008-09-01

Sandia National Laboratories is investing in projects that aim to develop computational modeling and simulation applications that explore human cognitive and social phenomena. While some of these modeling and simulation projects are explicitly research oriented, others are intended to support or provide insight for people involved in high consequence decision-making. This raises the issue of how to evaluate computational modeling and simulation applications in both research and applied settings where human behavior is the focus of the model: when is a simulation 'good enough' for the goals its designers want to achieve? In this report, we discuss two years' worthmore » of review and assessment of the ASC program's approach to computational model verification and validation, uncertainty quantification, and decision making. We present a framework that extends the principles of the ASC approach into the area of computational social and cognitive modeling and simulation. In doing so, we argue that the potential for evaluation is a function of how the modeling and simulation software will be used in a particular setting. In making this argument, we move from strict, engineering and physics oriented approaches to V&V to a broader project of model evaluation, which asserts that the systematic, rigorous, and transparent accumulation of evidence about a model's performance under conditions of uncertainty is a reasonable and necessary goal for model evaluation, regardless of discipline. How to achieve the accumulation of evidence in areas outside physics and engineering is a significant research challenge, but one that requires addressing as modeling and simulation tools move out of research laboratories and into the hands of decision makers. This report provides an assessment of our thinking on ASC Verification and Validation, and argues for further extending V&V research in the physical and engineering sciences toward a broader program of model evaluation in situations of high consequence decision-making.« less

Modeling subjective evaluation of soundscape quality in urban open spaces: An artificial neural network approach.

PubMed

Yu, Lei; Kang, Jian

2009-09-01

This research aims to explore the feasibility of using computer-based models to predict the soundscape quality evaluation of potential users in urban open spaces at the design stage. With the data from large scale field surveys in 19 urban open spaces across Europe and China, the importance of various physical, behavioral, social, demographical, and psychological factors for the soundscape evaluation has been statistically analyzed. Artificial neural network (ANN) models have then been explored at three levels. It has been shown that for both subjective sound level and acoustic comfort evaluation, a general model for all the case study sites is less feasible due to the complex physical and social environments in urban open spaces; models based on individual case study sites perform well but the application range is limited; and specific models for certain types of location/function would be reliable and practical. The performance of acoustic comfort models is considerably better than that of sound level models. Based on the ANN models, soundscape quality maps can be produced and this has been demonstrated with an example.

Evaluation of the US DOE's conceptual model of hydrothermal activity at Yucca Mountain, Nevada

NASA Astrophysics Data System (ADS)

Dublyansky, Y. V.

2014-08-01

A unique conceptual model describing the conductive heating of rocks in the thick unsaturated zone of Yucca Mountain, Nevada by a silicic pluton emplaced several kilometers away is accepted by the US Department of Energy (DOE) as an explanation of the elevated depositional temperatures measured in fluid inclusions in secondary fluorite and calcite. Acceptance of this model allowed the DOE to keep from considering hydrothermal activity in the performance assessment of the proposed high-level nuclear waste disposal facility. The evaluation presented in this paper shows that no computational modeling results have yet produced a satisfactory match with the empirical benchmark data, specifically with age and fluid inclusion data that indicate high temperatures (up to ca. 80 °C) in the unsaturated zone of Yucca Mountain. Auxiliary sub-models complementing the DOE model, as well as observations at a natural analog site, have also been evaluated. Summarily, the model cannot be considered as validated. Due to the lack of validation, the reliance on this model must be discontinued and the appropriateness of decisions which rely on this model must be re-evaluated.

Extracting the Evaluations of Stereotypes: Bi-factor Model of the Stereotype Content Structure

PubMed Central

Sayans-Jiménez, Pablo; Cuadrado, Isabel; Rojas, Antonio J.; Barrada, Juan R.

2017-01-01

Stereotype dimensions—competence, morality and sociability—are fundamental to studying the perception of other groups. These dimensions have shown moderate/high positive correlations with each other that do not reflect the theoretical expectations. The explanation for this (e.g., halo effect) undervalues the utility of the shared variance identified. In contrast, in this work we propose that this common variance could represent the global evaluation of the perceived group. Bi-factor models are proposed to improve the internal structure and to take advantage of the information representing the shared variance among dimensions. Bi-factor models were compared with first order models and other alternative models in three large samples (300–309 participants). The relationships among the global and specific bi-factor dimensions with a global evaluation dimension (measured through a semantic differential) were estimated. The results support the use of bi-factor models rather than first order models (and other alternative models). Bi-factor models also show a greater utility to directly and more easily explore the stereotype content including its evaluative content. PMID:29085313

Validating the ACE Model for Evaluating Student Performance Using a Teaching-Learning Process Based on Computational Modeling Systems

ERIC Educational Resources Information Center

Louzada, Alexandre Neves; Elia, Marcos da Fonseca; Sampaio, Fábio Ferrentini; Vidal, Andre Luiz Pestana

2014-01-01

The aim of this work is to adapt and test, in a Brazilian public school, the ACE model proposed by Borkulo for evaluating student performance as a teaching-learning process based on computational modeling systems. The ACE model is based on different types of reasoning involving three dimensions. In addition to adapting the model and introducing…

Development and validation of a nursing professionalism evaluation model in a career ladder system.

PubMed

Kim, Yeon Hee; Jung, Young Sun; Min, Ja; Song, Eun Young; Ok, Jung Hui; Lim, Changwon; Kim, Kyunghee; Kim, Ji-Su

2017-01-01

The clinical ladder system categorizes the degree of nursing professionalism and rewards and is an important human resource tool for managing nursing. We developed a model to evaluate nursing professionalism, which determines the clinical ladder system levels, and verified its validity. Data were collected using a clinical competence tool developed in this study, and existing methods such as the nursing professionalism evaluation tool, peer reviews, and face-to-face interviews to evaluate promotions and verify the presented content in a medical institution. Reliability and convergent and discriminant validity of the clinical competence evaluation tool were verified using SmartPLS software. The validity of the model for evaluating overall nursing professionalism was also analyzed. Clinical competence was determined by five dimensions of nursing practice: scientific, technical, ethical, aesthetic, and existential. The structural model explained 66% of the variance. Clinical competence scales, peer reviews, and face-to-face interviews directly determined nursing professionalism levels. The evaluation system can be used for evaluating nurses' professionalism in actual medical institutions from a nursing practice perspective. A conceptual framework for establishing a human resources management system for nurses and a tool for evaluating nursing professionalism at medical institutions is provided.

Black Model Appearance and Product Evaluations.

ERIC Educational Resources Information Center

Kerin, Roger A.

1979-01-01

Examines a study of how human models affect the impression conveyed by an advertisement, particularly the effect of a Black model's physical characteristics on product evaluations among Black and White females.Results show that the physical appearance of the model influenced impressions of product quality and suitability for personal use. (JMF)

Applying an integrated model to the evaluation of travel demand management policies in the Sacramento Region

DOT National Transportation Integrated Search

2001-09-01

The objective of this study was to use an advanced integrated land use and transportation model to evaluate transit and supportive land use and pricing policies; the Sacramento MEPLAN model was to used to simulate these policies. The model represents...

USE OF PHARMACOKINETIC MODELING TO DESIGN STUDIES FOR PATHWAY-SPECIFIC EXPOSURE MODEL EVALUATION

EPA Science Inventory

Validating an exposure pathway model is difficult because the biomarker, which is often used to evaluate the model prediction, is an integrated measure for exposures from all the exposure routes/pathways. The purpose of this paper is to demonstrate a method to use pharmacokeneti...

EVALUATION OF PHYSIOLOGY COMPUTER MODELS, AND THE FEASIBILITY OF THEIR USE IN RISK ASSESSMENT.

EPA Science Inventory

This project will evaluate the current state of quantitative models that simulate physiological processes, and the how these models might be used in conjunction with the current use of PBPK and BBDR models in risk assessment. The work will include a literature search to identify...

Evaluating, interpreting, and communicating performance of hydrologic/water quality models considering intended use: A review and recommendations

USDA-ARS?s Scientific Manuscript database

Previous publications have outlined recommended practices for hydrologic and water quality (H/WQ) modeling, but none have formulated comprehensive guidelines for the final stage of modeling applications, namely evaluation, interpretation, and communication of model results and the consideration of t...

A Moderate Constructivist E-Learning Instructional Model Evaluated on Computer Specialists

ERIC Educational Resources Information Center

Alonso, Fernando; Manrique, Daniel; Vines, Jose M.

2009-01-01

This paper presents a novel instructional model for e-learning and an evaluation study to determine the effectiveness of this model for teaching Java language programming to information technology specialists working for the Spanish Public Administration. This is a general-purpose model that combines objectivist and constructivist learning…

Teachers' Development Model to Authentic Assessment by Empowerment Evaluation Approach

ERIC Educational Resources Information Center

Charoenchai, Charin; Phuseeorn, Songsak; Phengsawat, Waro

2015-01-01

The purposes of this study were 1) Study teachers authentic assessment, teachers comprehension of authentic assessment and teachers needs for authentic assessment development. 2) To create teachers development model. 3) Experiment of teachers development model. 4) Evaluate effectiveness of teachers development model. The research is divided into 4…

Investigation of remote sensing techniques of measuring soil moisture

NASA Technical Reports Server (NTRS)

Newton, R. W. (Principal Investigator); Blanchard, A. J.; Nieber, J. L.; Lascano, R.; Tsang, L.; Vanbavel, C. H. M.

1981-01-01

Major activities described include development and evaluation of theoretical models that describe both active and passive microwave sensing of soil moisture, the evaluation of these models for their applicability, the execution of a controlled field experiment during which passive microwave measurements were acquired to validate these models, and evaluation of previously acquired aircraft microwave measurements. The development of a root zone soil water and soil temperature profile model and the calibration and evaluation of gamma ray attenuation probes for measuring soil moisture profiles are considered. The analysis of spatial variability of soil information as related to remote sensing is discussed as well as the implementation of an instrumented field site for acquisition of soil moisture and meteorologic information for use in validating the soil water profile and soil temperature profile models.

A Universal Model for Evaluating Basic Electronic Courses in Terms of Field Utilization of Training.

ERIC Educational Resources Information Center

Air Force Occupational Measurement Center, Lackland AFB, TX.

The main purpose of the Air Force project was to develop a universal model to evaluate usage of basic electronic principles training. The criterion used by the model to evaluate electronic theory training is a determination of the usefulness of the training vis-a-vis the performance of assigned tasks in the various electronic career fields. Data…

Using the Many-Facet Rasch Model to Evaluate Standard-Setting Judgments: Setting Performance Standards for Advanced Placement® Examinations

ERIC Educational Resources Information Center

Kaliski, Pamela; Wind, Stefanie A.; Engelhard, George, Jr.; Morgan, Deanna; Plake, Barbara; Reshetar, Rosemary

2012-01-01

The Many-Facet Rasch (MFR) Model is traditionally used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR Model by examining the quality of ratings obtained from a…

Identfying the Needs of Pre-Service Classroom Teachers about Science Teaching Methodology Course in Terms of Parlett's Illuminative Program Evaluation Model

ERIC Educational Resources Information Center

Çaliskan, Ilke

2014-01-01

The aim of this study was to identify the needs of third grade classroom teaching students about science teaching course in terms of Parlett's Illuminative program evaluation model. Phenomographic research design was used in this study. Illuminative program evaluation model was chosen for this study in terms of its eclectic and process-based…

Identfying the Needs of Pre-Service Classroom Teachers about Science Teaching Methodology Courses in Terms of Parlett's Illuminative Program Evaluation Model

ERIC Educational Resources Information Center

Çaliskan, Ilke

2014-01-01

The aim of this study was to identify the needs of third grade classroom teaching students about science teaching course in terms of Parlett's Illuminative program evaluation model. Phenomographic research design was used in this study. Illuminative program evaluation model was chosen for this study in terms of its eclectic and process-based…

Evaluating Regional-Scale Air Quality Models

EPA Science Inventory

Numerical air quality models are being used to understand the complex interplay among emission loading meteorology, and atmospheric chemistry leading to the formation and accumulation of pollutants in the atmosphere. A model evaluation framework is presented here that considers ...

A Novel Immunocompetent Mouse Model of Pancreatic Cancer with Robust Stroma: a Valuable Tool for Preclinical Evaluation of New Therapies

PubMed Central

Majumder, Kaustav; Arora, Nivedita; Modi, Shrey; Chugh, Rohit; Nomura, Alice; Giri, Bhuwan; Dawra, Rajinder; Ramakrishnan, Sundaram; Banerjee, Sulagna; Saluja, Ashok; Dudeja, Vikas

2017-01-01

A valid preclinical tumor model should recapitulate the tumor microenvironment. Immune and stromal components are absent in immunodeficient models of pancreatic cancer. While these components are present in genetically engineered models such as KrasG12D; Trp53R172H; Pdx-1Cre (KPC), immense variability in development of invasive disease makes them unsuitable for evaluation of novel therapies. We have generated a novel mouse model of pancreatic cancer by implanting tumor fragments from KPC mice into the pancreas of wild type mice. Three-millimeter tumor pieces from KPC mice were implanted into the pancreas of C57BL/6J mice. Four to eight weeks later, tumors were harvested, and stromal and immune components were evaluated. The efficacy of Minnelide, a novel compound which has been shown to be effective against pancreatic cancer in a number of preclinical murine models, was evaluated. In our model, consistent tumor growth and metastases were observed. Tumors demonstrated intense desmoplasia and leukocytic infiltration which was comparable to that in the genetically engineered KPC model and significantly more than that observed in KPC tumor-derived cell line implantation model. Minnelide treatment resulted in a significant decrease in the tumor weight and volume. This novel model demonstrates a consistent growth rate and tumor-associated mortality and recapitulates the tumor microenvironment. This convenient model is a valuable tool to evaluate novel therapies. PMID:26582596

Evaluation of a Multi-Decadal Simulation of Stratospheric Ozone by Comparison with Total Ozone Mapping Spectrometer (TOMS) Observations

NASA Technical Reports Server (NTRS)

Douglass, Anne R.; Stolarski, Richard S.; Steenrod, Steven; Pawson, Steven

2003-01-01

One key application of atmospheric chemistry and transport models is prediction of the response of ozone and other constituents to various natural and anthropogenic perturbations. These include changes in composition, such as the previous rise and recent decline in emission of man-made chlorofluorcarbons, changes in aerosol loading due to volcanic eruption, and changes in solar forcing. Comparisons of hindcast model results for the past few decades with observations are a key element of model evaluation and provide a sense of the reliability of model predictions. The 25 year data set from Total Ozone Mapping Spectrometers is a cornerstone of such model evaluation. Here we report evaluation of three-dimensional multi-decadal simulation of stratospheric composition. Meteorological fields for this off-line calculation are taken from a 50 year simulation of a general circulation model. Model fields are compared with observations from TOMS and also with observations from the Stratospheric Aerosol and Gas Experiment (SAGE), Microwave Limb Sounder (MLS), Cryogenic Limb Array Etalon Spectrometer (CLAES), and the Halogen Occultation Experiment (HALOE). This overall evaluation will emphasize the spatial, seasonal, and interannual variability of the simulation compared with observed atmospheric variability.

Precipitation-runoff modeling system; user's manual

USGS Publications Warehouse

Leavesley, G.H.; Lichty, R.W.; Troutman, B.M.; Saindon, L.G.

1983-01-01

The concepts, structure, theoretical development, and data requirements of the precipitation-runoff modeling system (PRMS) are described. The precipitation-runoff modeling system is a modular-design, deterministic, distributed-parameter modeling system developed to evaluate the impacts of various combinations of precipitation, climate, and land use on streamflow, sediment yields, and general basin hydrology. Basin response to normal and extreme rainfall and snowmelt can be simulated to evaluate changes in water balance relationships, flow regimes, flood peaks and volumes, soil-water relationships, sediment yields, and groundwater recharge. Parameter-optimization and sensitivity analysis capabilites are provided to fit selected model parameters and evaluate their individual and joint effects on model output. The modular design provides a flexible framework for continued model system enhancement and hydrologic modeling research and development. (Author 's abstract)

Robust network data envelopment analysis approach to evaluate the efficiency of regional electricity power networks under uncertainty.

PubMed

Fathollah Bayati, Mohsen; Sadjadi, Seyed Jafar

2017-01-01

In this paper, new Network Data Envelopment Analysis (NDEA) models are developed to evaluate the efficiency of regional electricity power networks. The primary objective of this paper is to consider perturbation in data and develop new NDEA models based on the adaptation of robust optimization methodology. Furthermore, in this paper, the efficiency of the entire networks of electricity power, involving generation, transmission and distribution stages is measured. While DEA has been widely used to evaluate the efficiency of the components of electricity power networks during the past two decades, there is no study to evaluate the efficiency of the electricity power networks as a whole. The proposed models are applied to evaluate the efficiency of 16 regional electricity power networks in Iran and the effect of data uncertainty is also investigated. The results are compared with the traditional network DEA and parametric SFA methods. Validity and verification of the proposed models are also investigated. The preliminary results indicate that the proposed models were more reliable than the traditional Network DEA model.

Robust network data envelopment analysis approach to evaluate the efficiency of regional electricity power networks under uncertainty

PubMed Central

Sadjadi, Seyed Jafar

2017-01-01

In this paper, new Network Data Envelopment Analysis (NDEA) models are developed to evaluate the efficiency of regional electricity power networks. The primary objective of this paper is to consider perturbation in data and develop new NDEA models based on the adaptation of robust optimization methodology. Furthermore, in this paper, the efficiency of the entire networks of electricity power, involving generation, transmission and distribution stages is measured. While DEA has been widely used to evaluate the efficiency of the components of electricity power networks during the past two decades, there is no study to evaluate the efficiency of the electricity power networks as a whole. The proposed models are applied to evaluate the efficiency of 16 regional electricity power networks in Iran and the effect of data uncertainty is also investigated. The results are compared with the traditional network DEA and parametric SFA methods. Validity and verification of the proposed models are also investigated. The preliminary results indicate that the proposed models were more reliable than the traditional Network DEA model. PMID:28953900

Logic Modeling as a Tool to Prepare to Evaluate Disaster and Emergency Preparedness, Response, and Recovery in Schools

ERIC Educational Resources Information Center

Zantal-Wiener, Kathy; Horwood, Thomas J.

2010-01-01

The authors propose a comprehensive evaluation framework to prepare for evaluating school emergency management programs. This framework involves a logic model that incorporates Government Performance and Results Act (GPRA) measures as a foundation for comprehensive evaluation that complements performance monitoring used by the U.S. Department of…

Evaluating University-Industry Collaboration: The European Foundation of Quality Management Excellence Model-Based Evaluation of University-Industry Collaboration

ERIC Educational Resources Information Center

Kauppila, Osmo; Mursula, Anu; Harkonen, Janne; Kujala, Jaakko

2015-01-01

The growth in university-industry collaboration has resulted in an increasing demand for methods to evaluate it. This paper presents one way to evaluate an organization's collaborative activities based on the European Foundation of Quality Management excellence model. Success factors of collaboration are derived from literature and compared…

A Faculty Evaluation Model for Online Instructors: Mentoring and Evaluation in the Online Classroom

ERIC Educational Resources Information Center

Mandernach, B. Jean; Donnelli, Emily; Dailey, Amber; Schulte, Marthann

2005-01-01

The rapid growth of online learning has mandated the development of faculty evaluation models geared specifically toward the unique demands of the online classroom. With a foundation in the best practices of online learning, adapted to meet the dynamics of a growing online program, the Online Instructor Evaluation System created at Park University…

An Evaluation of a Summer Reading Institute, 1968.

ERIC Educational Resources Information Center

Rosenfeld, Michael

This document describes part of the evaluation of a six-week reading institute for 69 K-3 teachers from the Raymond School, Model School Division (MSD), Washington, D.C. and thereby provides an evaluation model for schools to use in their own inservice training programs. Two evaluation instruments developed by an MSD innovation team in cooperation…

The use of modelling to evaluate and adapt strategies for animal disease control.

PubMed

Saegerman, C; Porter, S R; Humblet, M F

2011-08-01

Disease is often associated with debilitating clinical signs, disorders or production losses in animals and/or humans, leading to severe socio-economic repercussions. This explains the high priority that national health authorities and international organisations give to selecting control strategies for and the eradication of specific diseases. When a control strategy is selected and implemented, an effective method of evaluating its efficacy is through modelling. To illustrate the usefulness of models in evaluating control strategies, the authors describe several examples in detail, including three examples of classification and regression tree modelling to evaluate and improve the early detection of disease: West Nile fever in equids, bovine spongiform encephalopathy (BSE) and multifactorial diseases, such as colony collapse disorder (CCD) in the United States. Also examined are regression modelling to evaluate skin test practices and the efficacy of an awareness campaign for bovine tuberculosis (bTB); mechanistic modelling to monitor the progress of a control strategy for BSE; and statistical nationwide modelling to analyse the spatio-temporal dynamics of bTB and search for potential risk factors that could be used to target surveillance measures more effectively. In the accurate application of models, an interdisciplinary rather than a multidisciplinary approach is required, with the fewest assumptions possible.

Evaluating the 239Pu prompt fission neutron spectrum induced by thermal to 30 MeV neutrons

DOE PAGES

Neudecker, Denise; Talou, Patrick; Kawano, Toshihiko; ...

2016-03-15

We present a new evaluation of the 239Pu prompt fission neutron spectrum (PFNS) induced by thermal to 30 MeV neutrons. Compared to the ENDF/B-VII.1 evaluation, this one includes recently published experimental data as well as an improved and extended model description to predict PFNS. For instance, the pre-equilibrium neutron emission component to the PFNS is considered and the incident energy dependence of model parameters is parametrized more realistically. Experimental and model parameter uncertainties and covariances are estimated in detail. Also, evaluated covariances are provided between all PFNS at different incident neutron energies. In conclusion, selected evaluation results and first benchmarkmore » calculations using this evaluation are briefly discussed.« less

Micro-CT evaluation of the marginal fit of CAD/CAM all ceramic crowns

NASA Astrophysics Data System (ADS)

Brenes, Christian

Objectives: Evaluate the marginal fit of CAD/CAM all ceramic crowns made from lithium disilicate and zirconia using two different fabrication protocols (model and model-less). METHODS: Forty anterior all ceramic restorations (20 lithium disilicate, 20 zirconia) were fabricated using a CEREC Bluecam scanner. Two different fabrication methods were used: a full digital approach and a printed model. Completed crowns were cemented and marginal gap was evaluated using Micro-CT. Each specimen was analyzed in sagittal and trans-axial orientations, allowing a 360° evaluation of the vertical and horizontal fit. RESULTS: Vertical measurements in the lingual, distal and mesial views had and estimated marginal gap from 101.9 to 133.9 microns for E-max crowns and 126.4 to 165.4 microns for zirconia. No significant differences were found between model and model-less techniques. CONCLUSION: Lithium disilicate restorations exhibited a more accurate and consistent marginal adaptation when compared to zirconia crowns. No statistically significant differences were observed when comparing model or model-less approaches.

Evaluation of airborne lidar data to predict vegetation Presence/Absence

USGS Publications Warehouse

Palaseanu-Lovejoy, M.; Nayegandhi, A.; Brock, J.; Woodman, R.; Wright, C.W.

2009-01-01

This study evaluates the capabilities of the Experimental Advanced Airborne Research Lidar (EAARL) in delineating vegetation assemblages in Jean Lafitte National Park, Louisiana. Five-meter-resolution grids of bare earth, canopy height, canopy-reflection ratio, and height of median energy were derived from EAARL data acquired in September 2006. Ground-truth data were collected along transects to assess species composition, canopy cover, and ground cover. To decide which model is more accurate, comparisons of general linear models and generalized additive models were conducted using conventional evaluation methods (i.e., sensitivity, specificity, Kappa statistics, and area under the curve) and two new indexes, net reclassification improvement and integrated discrimination improvement. Generalized additive models were superior to general linear models in modeling presence/absence in training vegetation categories, but no statistically significant differences between the two models were achieved in determining the classification accuracy at validation locations using conventional evaluation methods, although statistically significant improvements in net reclassifications were observed. ?? 2009 Coastal Education and Research Foundation.

Using a logic model to relate the strategic to the tactical in program planning and evaluation: an illustration based on social norms interventions.

PubMed

Keller, Adrienne; Bauerle, Jennifer A

2009-01-01

Logic models are a ubiquitous tool for specifying the tactics--including implementation and evaluation--of interventions in the public health, health and social behaviors arenas. Similarly, social norms interventions are a common strategy, particularly in college settings, to address hazardous drinking and other dangerous or asocial behaviors. This paper illustrates an extension of logic models to include strategic as well as tactical components, using a specific example developed for social norms interventions. Placing the evaluation of projects within the context of this kind of logic model addresses issues related to the lack of a research design to evaluate effectiveness.

Core Professionalism Education in Surgery: A Systematic Review

PubMed Central

Sarıoğlu Büke, Akile; Karabilgin Öztürkçü, Özlem Sürel; Yılmaz, Yusuf; Sayek, İskender

2018-01-01

Background: Professionalism education is one of the major elements of surgical residency education. Aims: To evaluate the studies on core professionalism education programs in surgical professionalism education. Study Design: Systematic review. Methods: This systematic literature review was performed to analyze core professionalism programs for surgical residency education published in English with at least three of the following features: program developmental model/instructional design method, aims and competencies, methods of teaching, methods of assessment, and program evaluation model or method. A total of 27083 articles were retrieved using EBSCOHOST, PubMed, Science Direct, Web of Science, and manual search. Results: Eight articles met the selection criteria. The instructional design method was presented in only one article, which described the Analysis, Design, Development, Implementation, and Evaluation model. Six articles were based on the Accreditation Council for Graduate Medical Education criterion, although there was significant variability in content. The most common teaching method was role modeling with scenario- and case-based learning. A wide range of assessment methods for evaluating professionalism education were reported. The Kirkpatrick model was reported in one article as a method for program evaluation. Conclusion: It is suggested that for a core surgical professionalism education program, developmental/instructional design model, aims and competencies, content, teaching methods, assessment methods, and program evaluation methods/models should be well defined, and the content should be comparable. PMID:29553464

Evaluating the Quality of the Learning Outcome in Healthcare Sector: The Expero4care Model

ERIC Educational Resources Information Center

Cervai, Sara; Polo, Federica

2015-01-01

Purpose: This paper aims to present the Expero4care model. Considering the growing need for a training evaluation model that does not simply fix processes, the Expero4care model represents the first attempt of a "quality model" dedicated to the learning outcomes of healthcare trainings. Design/Methodology/Approach: Created as development…

Empirical Evaluation of a Mathematical Model of Ethnolinguistic Vitality: The Case of Voro

ERIC Educational Resources Information Center

Ehala, Martin; Niglas, Katrin

2007-01-01

The paper presents the results of an empirical evaluation of a mathematical model of ethnolinguistic vitality. The model adds several new factors to the set used in previous models of ethnolinguistic vitality and operationalises it in a manner that would make it easier to compare the vitality of different groups. According to the model, the…

Evaluating Model Fit for Growth Curve Models: Integration of Fit Indices from SEM and MLM Frameworks

ERIC Educational Resources Information Center

Wu, Wei; West, Stephen G.; Taylor, Aaron B.

2009-01-01

Evaluating overall model fit for growth curve models involves 3 challenging issues. (a) Three types of longitudinal data with different implications for model fit may be distinguished: balanced on time with complete data, balanced on time with data missing at random, and unbalanced on time. (b) Traditional work on fit from the structural equation…

What does it mean to "employ" the RE-AIM model?

PubMed

Kessler, Rodger S; Purcell, E Peyton; Glasgow, Russell E; Klesges, Lisa M; Benkeser, Rachel M; Peek, C J

2013-03-01

Many grant proposals identify the use of a given evaluation model or framework but offer little about how such models are implemented. The authors discuss what it means to employ a specific model, RE-AIM, and key dimensions from this model for program planning, implementation, evaluation, and reporting. The authors report both conceptual and content specifications for the use of the RE-AIM model and a content review of 42 recent dissemination and implementation grant applications to National Institutes of Health that proposed the use of this model. Outcomes include the extent to which proposals addressed the overall RE-AIM model and specific items within the five dimensions in their methods or evaluation plans. The majority of grants used only some elements of the model (less than 10% contained thorough measures across all RE-AIM dimensions). Few met criteria for "fully developed use" of RE-AIM and the percentage of key issues addressed varied from, on average, 45% to 78% across the RE-AIM dimensions. The results and discussion of key criteria should help investigators in their use of RE-AIM and illuminate the broader issue of comprehensive use of evaluation models.

Highway Air Pollution Dispersion Modeling : Preliminary Evaluation of Thirteen Models

DOT National Transportation Integrated Search

1978-06-01

Thirteen highway air pollution dispersion models have been tested, using a portion of the Airedale air quality data base. The Transportation Air Pollution Studies (TAPS) System, a data base management system specifically designed for evaluating dispe...

Highway Air Pollution Dispersion Modeling : Preliminary Evaluation of Thirteen Models

DOT National Transportation Integrated Search

1977-01-01

Thirteen highway air pollution dispersion models have been tested, using a portion of the Airedale air quality data base. The Transportation Air Pollution Studies (TAPS) System, a data base management system specifically designed for evaluating dispe...

Quantized Step-up Model for Evaluation of Internship in Teaching of Prospective Science Teachers.

ERIC Educational Resources Information Center

Sindhu, R. S.

2002-01-01

Describes the quantized step-up model developed for the evaluation purposes of internship in teaching which is an analogous model of the atomic structure. Assesses prospective teachers' abilities in lesson delivery. (YDS)

Evaluation of origin-destination matrix estimation techniques to support aspects of traffic modeling.

DOT National Transportation Integrated Search

2014-05-01

Travel demand forecasting models are used to predict future traffic volumes to evaluate : roadway improvement alternatives. Each of the metropolitan planning organizations (MPO) in : Alabama maintains a travel demand model to support planning efforts...

A Literature Survey and Experimental Evaluation of the State-of-the-Art in Uplift Modeling: A Stepping Stone Toward the Development of Prescriptive Analytics.

PubMed

Devriendt, Floris; Moldovan, Darie; Verbeke, Wouter

2018-03-01

Prescriptive analytics extends on predictive analytics by allowing to estimate an outcome in function of control variables, allowing as such to establish the required level of control variables for realizing a desired outcome. Uplift modeling is at the heart of prescriptive analytics and aims at estimating the net difference in an outcome resulting from a specific action or treatment that is applied. In this article, a structured and detailed literature survey on uplift modeling is provided by identifying and contrasting various groups of approaches. In addition, evaluation metrics for assessing the performance of uplift models are reviewed. An experimental evaluation on four real-world data sets provides further insight into their use. Uplift random forests are found to be consistently among the best performing techniques in terms of the Qini and Gini measures, although considerable variability in performance across the various data sets of the experiments is observed. In addition, uplift models are frequently observed to be unstable and display a strong variability in terms of performance across different folds in the cross-validation experimental setup. This potentially threatens their actual use for business applications. Moreover, it is found that the available evaluation metrics do not provide an intuitively understandable indication of the actual use and performance of a model. Specifically, existing evaluation metrics do not facilitate a comparison of uplift models and predictive models and evaluate performance either at an arbitrary cutoff or over the full spectrum of potential cutoffs. In conclusion, we highlight the instability of uplift models and the need for an application-oriented approach to assess uplift models as prime topics for further research.

Observational uncertainty and regional climate model evaluation: A pan-European perspective

NASA Astrophysics Data System (ADS)

Kotlarski, Sven; Szabó, Péter; Herrera, Sixto; Räty, Olle; Keuler, Klaus; Soares, Pedro M.; Cardoso, Rita M.; Bosshard, Thomas; Pagé, Christian; Boberg, Fredrik; Gutiérrez, José M.; Jaczewski, Adam; Kreienkamp, Frank; Liniger, Mark. A.; Lussana, Cristian; Szepszo, Gabriella

2017-04-01

Local and regional climate change assessments based on downscaling methods crucially depend on the existence of accurate and reliable observational reference data. In dynamical downscaling via regional climate models (RCMs) observational data can influence model development itself and, later on, model evaluation, parameter calibration and added value assessment. In empirical-statistical downscaling, observations serve as predictand data and directly influence model calibration with corresponding effects on downscaled climate change projections. Focusing on the evaluation of RCMs, we here analyze the influence of uncertainties in observational reference data on evaluation results in a well-defined performance assessment framework and on a European scale. For this purpose we employ three different gridded observational reference grids, namely (1) the well-established EOBS dataset (2) the recently developed EURO4M-MESAN regional re-analysis, and (3) several national high-resolution and quality-controlled gridded datasets that recently became available. In terms of climate models five reanalysis-driven experiments carried out by five different RCMs within the EURO-CORDEX framework are used. Two variables (temperature and precipitation) and a range of evaluation metrics that reflect different aspects of RCM performance are considered. We furthermore include an illustrative model ranking exercise and relate observational spread to RCM spread. The results obtained indicate a varying influence of observational uncertainty on model evaluation depending on the variable, the season, the region and the specific performance metric considered. Over most parts of the continent, the influence of the choice of the reference dataset for temperature is rather small for seasonal mean values and inter-annual variability. Here, model uncertainty (as measured by the spread between the five RCM simulations considered) is typically much larger than reference data uncertainty. For parameters of the daily temperature distribution and for the spatial pattern correlation, however, important dependencies on the reference dataset can arise. The related evaluation uncertainties can be as large or even larger than model uncertainty. For precipitation the influence of observational uncertainty is, in general, larger than for temperature. It often dominates model uncertainty especially for the evaluation of the wet day frequency, the spatial correlation and the shape and location of the distribution of daily values. But even the evaluation of large-scale seasonal mean values can be considerably affected by the choice of the reference. When employing a simple and illustrative model ranking scheme on these results it is found that RCM ranking in many cases depends on the reference dataset employed.

A review of typhoid fever transmission dynamic models and economic evaluations of vaccination.

PubMed

Watson, Conall H; Edmunds, W John

2015-06-19

Despite a recommendation by the World Health Organization (WHO) that typhoid vaccines be considered for the control of endemic disease and outbreaks, programmatic use remains limited. Transmission models and economic evaluation may be informative in decision making about vaccine programme introductions and their role alongside other control measures. A literature search found few typhoid transmission models or economic evaluations relative to analyses of other infectious diseases of similar or lower health burden. Modelling suggests vaccines alone are unlikely to eliminate endemic disease in the short to medium term without measures to reduce transmission from asymptomatic carriage. The single identified data-fitted transmission model of typhoid vaccination suggests vaccines can reduce disease burden substantially when introduced programmatically but that indirect protection depends on the relative contribution of carriage to transmission in a given setting. This is an important source of epidemiological uncertainty, alongside the extent and nature of natural immunity. Economic evaluations suggest that typhoid vaccination can be cost-saving to health services if incidence is extremely high and cost-effective in other high-incidence situations, when compared to WHO norms. Targeting vaccination to the highest incidence age-groups is likely to improve cost-effectiveness substantially. Economic perspective and vaccine costs substantially affect estimates, with disease incidence, case-fatality rates, and vaccine efficacy over time also important determinants of cost-effectiveness and sources of uncertainty. Static economic models may under-estimate benefits of typhoid vaccination by omitting indirect protection. Typhoid fever transmission models currently require per-setting epidemiological parameterisation to inform their use in economic evaluation, which may limit their generalisability. We found no economic evaluation based on transmission dynamic modelling, and no economic evaluation of typhoid vaccination against interventions such as improvements in sanitation or hygiene. Copyright © 2015. Published by Elsevier Ltd.

A Tentative Study on the Evaluation of Community Health Service Quality*

NASA Astrophysics Data System (ADS)

Ma, Zhi-qiang; Zhu, Yong-yue

Community health service is the key point of health reform in China. Based on pertinent studies, this paper constructed an indicator system for the community health service quality evaluation from such five perspectives as visible image, reliability, responsiveness, assurance and sympathy, according to service quality evaluation scale designed by Parasuraman, Zeithaml and Berry. A multilevel fuzzy synthetical evaluation model was constructed to evaluate community health service by fuzzy mathematics theory. The applicability and maneuverability of the evaluation indicator system and evaluation model were verified by empirical analysis.

Fuzzy Evaluating Customer Satisfaction of Jet Fuel Companies

NASA Astrophysics Data System (ADS)

Cheng, Haiying; Fang, Guoyi

Based on the market characters of jet fuel companies, the paper proposes an evaluation index system of jet fuel company customer satisfaction from five dimensions as time, business, security, fee and service. And a multi-level fuzzy evaluation model composing with the analytic hierarchy process approach and fuzzy evaluation approach is given. Finally a case of one jet fuel company customer satisfaction evaluation is studied and the evaluation results response the feelings of the jet fuel company customers, which shows the fuzzy evaluation model is effective and efficient.

Using a Mixed Model to Explore Evaluation Criteria for Bank Supervision: A Banking Supervision Law Perspective

PubMed Central

Tsai, Sang-Bing; Chen, Kuan-Yu; Zhao, Hongrui; Wei, Yu-Min; Wang, Cheng-Kuang; Zheng, Yuxiang; Chang, Li-Chung; Wang, Jiangtao

2016-01-01

Financial supervision means that monetary authorities have the power to supervise and manage financial institutions according to laws. Monetary authorities have this power because of the requirements of improving financial services, protecting the rights of depositors, adapting to industrial development, ensuring financial fair trade, and maintaining stable financial order. To establish evaluation criteria for bank supervision in China, this study integrated fuzzy theory and the decision making trial and evaluation laboratory (DEMATEL) and proposes a fuzzy-DEMATEL model. First, fuzzy theory was applied to examine bank supervision criteria and analyze fuzzy semantics. Second, the fuzzy-DEMATEL model was used to calculate the degree to which financial supervision criteria mutually influenced one another and their causal relationship. Finally, an evaluation criteria model for evaluating bank and financial supervision was established. PMID:27992449

Using a Mixed Model to Explore Evaluation Criteria for Bank Supervision: A Banking Supervision Law Perspective.

PubMed

Tsai, Sang-Bing; Chen, Kuan-Yu; Zhao, Hongrui; Wei, Yu-Min; Wang, Cheng-Kuang; Zheng, Yuxiang; Chang, Li-Chung; Wang, Jiangtao

2016-01-01

Financial supervision means that monetary authorities have the power to supervise and manage financial institutions according to laws. Monetary authorities have this power because of the requirements of improving financial services, protecting the rights of depositors, adapting to industrial development, ensuring financial fair trade, and maintaining stable financial order. To establish evaluation criteria for bank supervision in China, this study integrated fuzzy theory and the decision making trial and evaluation laboratory (DEMATEL) and proposes a fuzzy-DEMATEL model. First, fuzzy theory was applied to examine bank supervision criteria and analyze fuzzy semantics. Second, the fuzzy-DEMATEL model was used to calculate the degree to which financial supervision criteria mutually influenced one another and their causal relationship. Finally, an evaluation criteria model for evaluating bank and financial supervision was established.

A modular method for evaluating the performance of picture archiving and communication systems.

PubMed

Sanders, W H; Kant, L A; Kudrimoti, A

1993-08-01

Modeling can be used to predict the performance of picture archiving and communication system (PACS) configurations under various load conditions at an early design stage. This is important because choices made early in the design of a system can have a significant impact on the performance of the resulting implementation. Because PACS consist of many types of components, it is important to do such evaluations in a modular manner, so that alternative configurations and designs can be easily investigated. Stochastic activity networks (SANs) and reduced base model construction methods can aid in doing this. SANs are a model type particularly suited to the evaluation of systems in which several activities may be in progress concurrently, and each activity may affect the others through the results of its completion. Together with SANs, reduced base model construction methods provide a means to build highly modular models, in which models of particular components can be easily reused. In this article, we investigate the use of SANs and reduced base model construction techniques in evaluating PACS. Construction and solution of the models is done using UltraSAN, a graphic-oriented software tool for model specification, analysis, and simulation. The method is illustrated via the evaluation of a realistically sized PACS for a typical United States hospital of 300 to 400 beds, and the derivation of system response times and component utilizations.

Regionalized PM2.5 Community Multiscale Air Quality model performance evaluation across a continuous spatiotemporal domain.

PubMed

Reyes, Jeanette M; Xu, Yadong; Vizuete, William; Serre, Marc L

2017-01-01

The regulatory Community Multiscale Air Quality (CMAQ) model is a means to understanding the sources, concentrations and regulatory attainment of air pollutants within a model's domain. Substantial resources are allocated to the evaluation of model performance. The Regionalized Air quality Model Performance (RAMP) method introduced here explores novel ways of visualizing and evaluating CMAQ model performance and errors for daily Particulate Matter ≤ 2.5 micrometers (PM2.5) concentrations across the continental United States. The RAMP method performs a non-homogenous, non-linear, non-homoscedastic model performance evaluation at each CMAQ grid. This work demonstrates that CMAQ model performance, for a well-documented 2001 regulatory episode, is non-homogeneous across space/time. The RAMP correction of systematic errors outperforms other model evaluation methods as demonstrated by a 22.1% reduction in Mean Square Error compared to a constant domain wide correction. The RAMP method is able to accurately reproduce simulated performance with a correlation of r = 76.1%. Most of the error coming from CMAQ is random error with only a minority of error being systematic. Areas of high systematic error are collocated with areas of high random error, implying both error types originate from similar sources. Therefore, addressing underlying causes of systematic error will have the added benefit of also addressing underlying causes of random error.

The comparative evaluation of expanded national immunization policies in Korea using an analytic hierarchy process.

PubMed

Shin, Taeksoo; Kim, Chun-Bae; Ahn, Yang-Heui; Kim, Hyo-Youl; Cha, Byung Ho; Uh, Young; Lee, Joo-Heon; Hyun, Sook-Jung; Lee, Dong-Han; Go, Un-Yeong

2009-01-29

The purpose of this paper is to propose new evaluation criteria and an analytic hierarchy process (AHP) model to assess the expanded national immunization programs (ENIPs) and to evaluate two alternative health care policies. One of the alternative policies is that private clinics and hospitals would offer free vaccination services to children and the other of them is that public health centers would offer these free vaccination services. Our model to evaluate the ENIPs was developed using brainstorming, Delphi techniques, and the AHP model. We first used the brainstorming and Delphi techniques, as well as literature reviews, to determine 25 criteria with which to evaluate the national immunization policy; we then proposed a hierarchical structure of the AHP model to assess ENIPs. By applying the proposed AHP model to the assessment of ENIPs for Korean immunization policies, we show that free vaccination services should be provided by private clinics and hospitals rather than public health centers.

Histopathological Evaluation of Skeletal Muscle with Specific Reference to Mouse Models of Muscular Dystrophy.

PubMed

Terry, Rebecca L; Wells, Dominic J

2016-12-01

The muscular dystrophies are a diverse group of degenerative diseases for which many mouse models are available. These models are frequently used to assess potential therapeutic interventions and histological evaluation of multiple muscles is an important part of this assessment. Histological evaluation is especially useful when combined with tests of muscle function. This unit describes a protocol for necropsy, processing, cryosectioning, and histopathological evaluation of murine skeletal muscles, which is applicable to both models of muscular dystrophy and other neuromuscular conditions. Key histopathological features of dystrophic muscle are discussed using the mdx mouse (a model of Duchenne muscular dystrophy) as an example. Optimal handling during dissection, processing and sectioning is vital to avoid artifacts that can confound or prevent future analyses. Muscles carefully processed using this protocol are suitable for further evaluation using immunohistochemistry, immunofluorescence, special histochemical stains, and immuoblotting. © 2016 by John Wiley & Sons, Inc. Copyright © 2016 John Wiley & Sons, Inc.

Model evaluation using a community benchmarking system for land surface models

NASA Astrophysics Data System (ADS)

Mu, M.; Hoffman, F. M.; Lawrence, D. M.; Riley, W. J.; Keppel-Aleks, G.; Kluzek, E. B.; Koven, C. D.; Randerson, J. T.

2014-12-01

Evaluation of atmosphere, ocean, sea ice, and land surface models is an important step in identifying deficiencies in Earth system models and developing improved estimates of future change. For the land surface and carbon cycle, the design of an open-source system has been an important objective of the International Land Model Benchmarking (ILAMB) project. Here we evaluated CMIP5 and CLM models using a benchmarking system that enables users to specify models, data sets, and scoring systems so that results can be tailored to specific model intercomparison projects. Our scoring system used information from four different aspects of global datasets, including climatological mean spatial patterns, seasonal cycle dynamics, interannual variability, and long-term trends. Variable-to-variable comparisons enable investigation of the mechanistic underpinnings of model behavior, and allow for some control of biases in model drivers. Graphics modules allow users to evaluate model performance at local, regional, and global scales. Use of modular structures makes it relatively easy for users to add new variables, diagnostic metrics, benchmarking datasets, or model simulations. Diagnostic results are automatically organized into HTML files, so users can conveniently share results with colleagues. We used this system to evaluate atmospheric carbon dioxide, burned area, global biomass and soil carbon stocks, net ecosystem exchange, gross primary production, ecosystem respiration, terrestrial water storage, evapotranspiration, and surface radiation from CMIP5 historical and ESM historical simulations. We found that the multi-model mean often performed better than many of the individual models for most variables. We plan to publicly release a stable version of the software during fall of 2014 that has land surface, carbon cycle, hydrology, radiation and energy cycle components.

High Tech Educators Network Evaluation.

ERIC Educational Resources Information Center

O'Shea, Dan

A process evaluation was conducted to assess the High Tech Educators Network's (HTEN's) activities. Four basic components to the evaluation approach were documentation review, program logic model, written survey, and participant interviews. The model mapped the basic goals and objectives, assumptions, activities, outcome expectations, and…

Computer Simulation of Human Service Program Evaluations.

ERIC Educational Resources Information Center

Trochim, William M. K.; Davis, James E.

1985-01-01

Describes uses of computer simulations for the context of human service program evaluation. Presents simple mathematical models for most commonly used human service outcome evaluation designs (pretest-posttest randomized experiment, pretest-posttest nonequivalent groups design, and regression-discontinuity design). Translates models into single…

Evaluation of a distributed energy balance model for a high-altitude glacier on the Tibetan Plateau using a time lapse camera system

NASA Astrophysics Data System (ADS)

Huintjes, Eva; Sauter, Tobias; Krenscher, Tobias; Maussion, Fabien; Kropacek, Jan; Yang, Wei; Zhang, Guoshuai; Kang, Shichang; Buchroithner, Manfred; Scherer, Dieter; Schneider, Christoph

2013-04-01

In the remote and high-altitude mountain areas of the Tibetan Plateau, climate observations as well as glacier-wide mass and energy balance determinations are scarce. Therefore, the application of models to determine reliable information on mass balance and runoff is important. Simultaneously, these circumstances make it difficult to evaluate the models. Since 2009, we operate an automatic weather station (AWS) in the ablation zone of Zhadang Glacier (5.665 m a.s.l.). The glacier is easily accessible. It is situated in the southern-central part of the Tibetan Plateau (30.5°N) in the Nam Co drainage basin and ranges between 5.400 and 5.900 m a.s.l. Based on these measurements over 2009-2012, we run and evaluate a physically based, distributed energy and mass balance model. The applied model couples an energy balance to a multilayer snow model and therefore accounts for subsurface processes like refreezing, subsurface melt and densification of the snowpack. First, the model is evaluated at point scale against measurements from the AWS. The results show that modelled accumulation and ablation patterns reproduce the observed changes in surface height very well. To evaluate the distributed model, we use daily images of a time lapse camera system installed nearby the glacier over 2010-2012. Therefore the non calibrated slope images had to be orthorectified using ground control points measured during field campaigns. The temporally and spatially highly resolved time series allows a detailed evaluation of the distributed energy balance model by analyzing the spatial and temporal heterogeneity of the snow line during the ablation season. First results show that the model captures the observed spatial heterogeneity of melt on the glacier surface. Subsequently to the evaluation the model will be applied on several glaciers and small ice caps in remote areas on the Tibetan Plateau to determine the linkages between climate fluctuations and glacier variability. The work is part of research projects funded by the DFG Priority Programme 1372: "Tibetan Plateau: Formation-Climate-Ecosystems" (TiP) and the BMBF research program "Central Asia and Tibet: Monsoon dynamics and geo-ecosystems" (CAME).

Evaluating energy saving system of data centers based on AHP and fuzzy comprehensive evaluation model

NASA Astrophysics Data System (ADS)

Jiang, Yingni

2018-03-01

Due to the high energy consumption of communication, energy saving of data centers must be enforced. But the lack of evaluation mechanisms has restrained the process on energy saving construction of data centers. In this paper, energy saving evaluation index system of data centers was constructed on the basis of clarifying the influence factors. Based on the evaluation index system, analytical hierarchy process was used to determine the weights of the evaluation indexes. Subsequently, a three-grade fuzzy comprehensive evaluation model was constructed to evaluate the energy saving system of data centers.

Predicting occupancy for pygmy rabbits in Wyoming: an independent evaluation of two species distribution models

USGS Publications Warehouse

Germaine, Stephen S.; Ignizio, Drew; Keinath, Doug; Copeland, Holly

2014-01-01

Species distribution models are an important component of natural-resource conservation planning efforts. Independent, external evaluation of their accuracy is important before they are used in management contexts. We evaluated the classification accuracy of two species distribution models designed to predict the distribution of pygmy rabbit Brachylagus idahoensis habitat in southwestern Wyoming, USA. The Nature Conservancy model was deductive and based on published information and expert opinion, whereas the Wyoming Natural Diversity Database model was statistically derived using historical observation data. We randomly selected 187 evaluation survey points throughout southwestern Wyoming in areas predicted to be habitat and areas predicted to be nonhabitat for each model. The Nature Conservancy model correctly classified 39 of 77 (50.6%) unoccupied evaluation plots and 65 of 88 (73.9%) occupied plots for an overall classification success of 63.3%. The Wyoming Natural Diversity Database model correctly classified 53 of 95 (55.8%) unoccupied plots and 59 of 88 (67.0%) occupied plots for an overall classification success of 61.2%. Based on 95% asymptotic confidence intervals, classification success of the two models did not differ. The models jointly classified 10.8% of the area as habitat and 47.4% of the area as nonhabitat, but were discordant in classifying the remaining 41.9% of the area. To evaluate how anthropogenic development affected model predictive success, we surveyed 120 additional plots among three density levels of gas-field road networks. Classification success declined sharply for both models as road-density level increased beyond 5 km of roads per km-squared area. Both models were more effective at predicting habitat than nonhabitat in relatively undeveloped areas, and neither was effective at accounting for the effects of gas-energy-development road networks. Resource managers who wish to know the amount of pygmy rabbit habitat present in an area or wanting to direct gas-drilling efforts away from pygmy rabbit habitat may want to consider both models in an ensemble manner, where more confidence is placed in mapped areas (i.e., pixels) for which both models agree than for areas where there is model disagreement.

Maritime Platform Sleep and Performance Study: Evaluating the SAFTE Model for Maritime Workplace Application

DTIC Science & Technology

2012-06-01

SLEEP AND PERFORMANCE STUDY: EVALUATING THE SAFTE MODEL FOR MARITIME WORKPLACE APPLICATION by Stephanie A. T. Brown June 2012 Thesis...REPORT DATE June 2012 3. REPORT TYPE AND DATES COVERED Master’s Thesis 4. TITLE AND SUBTITLE Maritime Platform Sleep and Performance Study...Evaluating the SAFTE Model for Maritime Workplace Application 5. FUNDING NUMBERS 6. AUTHOR(S) Stephanie A. T. Brown 7. PERFORMING ORGANIZATION

Using the Many-Faceted Rasch Model to Evaluate Standard Setting Judgments: An Illustration with the Advanced Placement Environmental Science Exam

ERIC Educational Resources Information Center

Kaliski, Pamela K.; Wind, Stefanie A.; Engelhard, George, Jr.; Morgan, Deanna L.; Plake, Barbara S.; Reshetar, Rosemary A.

2013-01-01

The many-faceted Rasch (MFR) model has been used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR model for examining the quality of ratings obtained from a standard…

Teratologic Evaluation of a Model Perfluorinated Acid, NDFDA

DTIC Science & Technology

1981-01-01

perfluorocarboxylic and perfluorosulfonic acids. I & G Product Research and Development. Vol. 1, No. 3, 165-169. Olson, C. T. and K. C. Back 1978...AFAMRL-TR-81 -14 TERATOLOGIC EVALUATION OF A MODEL PERFLUORINATED ACID, NDFDA INEZ R. BA CON UNIVERSITY OF THE DISTRICT OF COLUMBIA DEPARTMENT OF...TYPE OF REPORT & PERIOD COVERED TERATOLOGIC EVALUATION OF A MODEL PERFLUORINATED ACID, NDFDA 6. PERFORMING ORG. REPORT NUMBER 7. AUTHOR(s) S. CONTRACT

Evaluation of the Regional Atmospheric Modeling System in the Eastern Range Dispersion Assessment System

NASA Technical Reports Server (NTRS)

Case, Jonathan

2000-01-01

The Applied Meteorology Unit is conducting an evaluation of the Regional Atmospheric Modeling System (RAMS) contained within the Eastern Range Dispersion Assessment System (ERDAS). ERDAS provides emergency response guidance for operations at the Cape Canaveral Air Force Station and the Kennedy Space Center in the event of an accidental hazardous material release or aborted vehicle launch. The prognostic data from RAMS is available to ERDAS for display and is used to initialize the 45th Range Safety (45 SW/SE) dispersion model. Thus, the accuracy of the 45 SW/SE dispersion model is dependent upon the accuracy of RAMS forecasts. The RAMS evaluation task consists of an objective and subjective component for the Florida warm and cool seasons of 1999-2000. The objective evaluation includes gridded and point error statistics at surface and upper-level observational sites, a comparison of the model errors to a coarser grid configuration of RAMS, and a benchmark of RAMS against the widely accepted Eta model. The warm-season subjective evaluation involves a verification of the onset and movement of the Florida east coast sea breeze and RAMS forecast precipitation. This interim report provides a summary of the RAMS objective and subjective evaluation for the 1999 Florida warm season only.

Goodness of Model-Data Fit and Invariant Measurement

ERIC Educational Resources Information Center

Engelhard, George, Jr.; Perkins, Aminah

2013-01-01

In this commentary, Englehard and Perkins remark that Maydeu-Olivares has presented a framework for evaluating the goodness of model-data fit for item response theory (IRT) models and correctly points out that overall goodness-of-fit evaluations of IRT models and data are not generally explored within most applications in educational and…

An Evaluation of the Preceptor Model versus the Formal Teaching Model.

ERIC Educational Resources Information Center

Shamian, Judith; Lemieux, Suzanne

1984-01-01

This study evaluated the effectiveness of two teaching methods to determine which is more effective in enhancing the knowledge base of participating nurses: the preceptor model embodies decentralized instruction by a member of the nursing staff, and the formal teaching model uses centralized teaching by the inservice education department. (JOW)

Surface Modeling, Solid Modeling and Finite Element Modeling. Analysis Capabilities of Computer-Assisted Design and Manufacturing Systems.

ERIC Educational Resources Information Center

Nee, John G.; Kare, Audhut P.

1987-01-01

Explores several concepts in computer assisted design/computer assisted manufacturing (CAD/CAM). Defines, evaluates, reviews and compares advanced computer-aided geometric modeling and analysis techniques. Presents the results of a survey to establish the capabilities of minicomputer based-systems with the CAD/CAM packages evaluated. (CW)

Assessment of the MACC reanalysis and its influence as chemical boundary conditions for regional air quality modeling in AQMEII-2

EPA Science Inventory

The Air Quality Model Evaluation International Initiative (AQMEII) has now reached its second phase which is dedicated to the evaluation of online coupled chemistry-meteorology models. Sixteen modeling groups from Europe and five from North America have run regional air quality m...

Seasonal ozone vertical profiles over North America using the AQMEII group of air quality models: model inter-comparison and stratospheric intrusion

EPA Science Inventory

This study utilizes simulations for the North American domain from four modeling groups that participated in the third phase of the Air Quality Model Evaluation International Initiative (AQMEII3) to evaluate seasonal ozone vertical profiles simulated for the year 2010 against ozo...

Evaluation of Aerosol-cloud Interaction in the GISS Model E Using ARM Observations

NASA Technical Reports Server (NTRS)

DeBoer, G.; Bauer, S. E.; Toto, T.; Menon, Surabi; Vogelmann, A. M.

2013-01-01

Observations from the US Department of Energy's Atmospheric Radiation Measurement (ARM) program are used to evaluate the ability of the NASA GISS ModelE global climate model in reproducing observed interactions between aerosols and clouds. Included in the evaluation are comparisons of basic meteorology and aerosol properties, droplet activation, effective radius parameterizations, and surface-based evaluations of aerosol-cloud interactions (ACI). Differences between the simulated and observed ACI are generally large, but these differences may result partially from vertical distribution of aerosol in the model, rather than the representation of physical processes governing the interactions between aerosols and clouds. Compared to the current observations, the ModelE often features elevated droplet concentrations for a given aerosol concentration, indicating that the activation parameterizations used may be too aggressive. Additionally, parameterizations for effective radius commonly used in models were tested using ARM observations, and there was no clear superior parameterization for the cases reviewed here. This lack of consensus is demonstrated to result in potentially large, statistically significant differences to surface radiative budgets, should one parameterization be chosen over another.

Uncertainty Evaluation and Appropriate Distribution for the RDHM in the Rockies

NASA Astrophysics Data System (ADS)

Kim, J.; Bastidas, L. A.; Clark, E. P.

2010-12-01

The problems that hydrologic models have in properly reproducing the processes involved in mountainous areas, and in particular the Rocky Mountains, are widely acknowledged. Herein, we present an application of the National Weather Service RDHM distributed model over the Durango River basin in Colorado. We focus primarily in the assessment of the model prediction uncertainty associated with the parameter estimation and the comparison of the model performance using parameters obtained with a priori estimation following the procedure of Koren et al., and those obtained via inverse modeling using a variety of Markov chain Monte Carlo based optimization algorithms. The model evaluation is based on traditional procedures as well as non-traditional ones based on the use of shape matching functions, which are more appropriate for the evaluation of distributed information (e.g. Hausdorff distance, earth movers distance). The variables used for the model performance evaluation are discharge (with internal nodes), snow cover and snow water equivalent. An attempt to establish the proper degree of distribution, for the Durango basin with the RDHM model, is also presented.

GEM-AQ, an On-line Global Multiscale Chemical Weather System: Model Description and Evaluation of Gas Phase Chemistry Processes

NASA Astrophysics Data System (ADS)

Neary, L.; Kaminski, J. W.; Struzewska, J.; Ainslie, B.; McConnell, J. C.

2007-12-01

Tropospheric chemistry and air quality processes were implemented on-line in the Global Environmental Multiscale model. The integrated model, GEM-AQ, has been developed as a platform to investigate chemical weather at scales from global to urban. On the global scale, the model was exercised for five years (2001-2005) to evaluate its ability to simulate seasonal variations and regional distributions of trace gases such as ozone, nitrogen dioxide and carbon monoxide. The model results are compared with observations from satellites, aircraft measurement campaigns and balloon sondes. The same model has also been evaluated on the regional (~15km resolution) and urban scale (~3km resolution). A simulation of the formation and transport of photooxidants during the European heat wave of 2006 was performed and compared with surface observations throughout central and eastern Europe. The complex topographic region of the Lower Fraser Valley in British Columbia was the focus of another model evaluation during the PACIFIC 2001 field campaign. Comparison of model results with observations during this period will be shown.

IEA-Task 31 WAKEBENCH: Towards a protocol for wind farm flow model evaluation. Part 2: Wind farm wake models

NASA Astrophysics Data System (ADS)

Moriarty, Patrick; Sanz Rodrigo, Javier; Gancarski, Pawel; Chuchfield, Matthew; Naughton, Jonathan W.; Hansen, Kurt S.; Machefaux, Ewan; Maguire, Eoghan; Castellani, Francesco; Terzi, Ludovico; Breton, Simon-Philippe; Ueda, Yuko

2014-06-01

Researchers within the International Energy Agency (IEA) Task 31: Wakebench have created a framework for the evaluation of wind farm flow models operating at the microscale level. The framework consists of a model evaluation protocol integrated with a web-based portal for model benchmarking (www.windbench.net). This paper provides an overview of the building-block validation approach applied to wind farm wake models, including best practices for the benchmarking and data processing procedures for validation datasets from wind farm SCADA and meteorological databases. A hierarchy of test cases has been proposed for wake model evaluation, from similarity theory of the axisymmetric wake and idealized infinite wind farm, to single-wake wind tunnel (UMN-EPFL) and field experiments (Sexbierum), to wind farm arrays in offshore (Horns Rev, Lillgrund) and complex terrain conditions (San Gregorio). A summary of results from the axisymmetric wake, Sexbierum, Horns Rev and Lillgrund benchmarks are used to discuss the state-of-the-art of wake model validation and highlight the most relevant issues for future development.

A comprehensive model to evaluate implementation of the world health organization framework convention of tobacco control

PubMed Central

Sarrafzadegan, Nizal; Kelishad, Roya; Rabiei, Katayoun; Abedi, Heidarali; Mohaseli, Khadijeh Fereydoun; Masooleh, Hasan Azaripour; Alavi, Mousa; Heidari, Gholamreza; Ghaffari, Mostafa; O’Loughlin, Jennifer

2012-01-01

Background: Iran is one of the countries that has ratified the World Health Organization Framework Convention of Tobacco Control (WHO-FCTC), and has implemented a series of tobacco control interventions including the Comprehensive Tobacco Control Law. Enforcement of this legislation and assessment of its outcome requires a dedicated evaluation system. This study aimed to develop a generic model to evaluate the implementation of the Comprehensive Tobacco Control Law in Iran that was provided based on WHO-FCTC articles. Materials and Methods: Using a grounded theory approach, qualitative data were collected from 265 subjects in individual interviews and focus group discussions with policymakers who designed the legislation, key stakeholders, and members of the target community. In addition, field observations data in supermarkets/shops, restaurants, teahouses and coffee shops were collected. Data were analyzed in two stages through conceptual theoretical coding. Findings: Overall, 617 open codes were extracted from the data into tables; 72 level-3 codes were retained from the level-2 code series. Using a Model Met paradigm, the relationships between the components of each paradigm were depicted graphically. The evaluation model entailed three levels, namely: short-term results, process evaluation and long-term results. Conclusions: Central concept of the process of evaluation is that enforcing the law influences a variety of internal and environmental factors including legislative changes. These factors will be examined during the process evaluation and context evaluation. The current model can be applicable for providing FCTC evaluation tools across other jurisdictions. PMID:23833621

Accuracies of univariate and multivariate genomic prediction models in African cassava.

PubMed

Okeke, Uche Godfrey; Akdemir, Deniz; Rabbi, Ismail; Kulakow, Peter; Jannink, Jean-Luc

2017-12-04

Genomic selection (GS) promises to accelerate genetic gain in plant breeding programs especially for crop species such as cassava that have long breeding cycles. Practically, to implement GS in cassava breeding, it is necessary to evaluate different GS models and to develop suitable models for an optimized breeding pipeline. In this paper, we compared (1) prediction accuracies from a single-trait (uT) and a multi-trait (MT) mixed model for a single-environment genetic evaluation (Scenario 1), and (2) accuracies from a compound symmetric multi-environment model (uE) parameterized as a univariate multi-kernel model to a multivariate (ME) multi-environment mixed model that accounts for genotype-by-environment interaction for multi-environment genetic evaluation (Scenario 2). For these analyses, we used 16 years of public cassava breeding data for six target cassava traits and a fivefold cross-validation scheme with 10-repeat cycles to assess model prediction accuracies. In Scenario 1, the MT models had higher prediction accuracies than the uT models for all traits and locations analyzed, which amounted to on average a 40% improved prediction accuracy. For Scenario 2, we observed that the ME model had on average (across all locations and traits) a 12% improved prediction accuracy compared to the uE model. We recommend the use of multivariate mixed models (MT and ME) for cassava genetic evaluation. These models may be useful for other plant species.

A Simulation Model to Evaluate Aircraft Survivability and Target Damage during Offensive Counterair Operations.

DTIC Science & Technology

1984-03-01

D-R14i 324 A SIMULATION MODEL TO EVALUATE AIRCRAFT SURVIVABILITY V/3 AND TARGET DAMAGE 0.. (U) AIR FORCE INST OF TECH WRIGHT-PATTERSON AFB OH SCHOOL...MICROCOPY RESOLUTION TEST CHART NATIONAL BUREAU OF STANDARDS- 1963-A J.1 AFIT/GST/0S/84-18 TS I°TI w ’ i A SIMULATION MODEL TO E’VALLUATE AIRCRAFT...numberp Title: A SIMULATION MODEL TO EVALUATE AIRCRAFT SURVIVABILITY AND jARGET DAMAGE DURING OFFENSIVE COUNTERAIR OPERATIONS Thesis Chairma#: James R

Evaluation of Cost Leadership Strategy in Shipping Enterprises with Simulation Model

NASA Astrophysics Data System (ADS)

Ferfeli, Maria V.; Vaxevanou, Anthi Z.; Damianos, Sakas P.

2009-08-01

The present study will attempt the evaluation of cost leadership strategy that prevails in certain shipping enterprises and the creation of simulation models based on strategic model STAIR. The above model is an alternative method of strategic applications evaluation. This is held in order to be realised if the strategy of cost leadership creates competitive advantage [1] and this will be achieved via the technical simulation which appreciates the interactions between the operations of an enterprise and the decision-making strategy in conditions of uncertainty with reduction of undertaken risk.

Evaluation of improved land use data and canopy representation in BEIS with biogenic VOC measurements in California

EPA Pesticide Factsheets

The link provided access to all the datasets and metadata used in this manuscript for the model development and evaluation per Geoscientific Model Development's publication guidelines with the exception of the model output due to its size. This dataset is associated with the following publication:Bash , J., K. Baker , and M. Beaver. Evaluation of improved land use and canopy representation in BEIS v3.61 with biogenic VOC measurements in California. Geoscientific Model Development. Copernicus Publications, Katlenburg-Lindau, GERMANY, 9: 2191-2207, (2016).

A Framework for Evaluating Regional-Scale Numerical Photochemical Modeling Systems

EPA Science Inventory

This paper discusses the need for critically evaluating regional-scale (~ 200-2000 km) three dimensional numerical photochemical air quality modeling systems to establish a model's credibility in simulating the spatio-temporal features embedded in the observations. Because of li...

Confronting the WRF and RAMS mesoscale models with innovative observations in the Netherlands: Evaluating the boundary layer heat budget

NASA Astrophysics Data System (ADS)

Steeneveld, G. J.; Tolk, L. F.; Moene, A. F.; Hartogensis, O. K.; Peters, W.; Holtslag, A. A. M.

2011-12-01

The Weather Research and Forecasting Model (WRF) and the Regional Atmospheric Mesoscale Model System (RAMS) are frequently used for (regional) weather, climate and air quality studies. This paper covers an evaluation of these models for a windy and calm episode against Cabauw tower observations (Netherlands), with a special focus on the representation of the physical processes in the atmospheric boundary layer (ABL). In addition, area averaged sensible heat flux observations by scintillometry are utilized which enables evaluation of grid scale model fluxes and flux observations at the same horizontal scale. Also, novel ABL height observations by ceilometry and of the near surface longwave radiation divergence are utilized. It appears that WRF in its basic set-up shows satisfactory model results for nearly all atmospheric near surface variables compared to field observations, while RAMS needed refining of its ABL scheme. An important inconsistency was found regarding the ABL daytime heat budget: Both model versions are only able to correctly forecast the ABL thermodynamic structure when the modeled surface sensible heat flux is much larger than both the eddy-covariance and scintillometer observations indicate. In order to clarify this discrepancy, model results for each term of the heat budget equation is evaluated against field observations. Sensitivity studies and evaluation of radiative tendencies and entrainment reveal that possible errors in these variables cannot explain the overestimation of the sensible heat flux within the current model infrastructure.

Statistical Evaluation of CRM-Simulated Cloud and Precipitation Structures Using Multi- sensor TRMM Measurements and Retrievals

NASA Astrophysics Data System (ADS)

Posselt, D.; L'Ecuyer, T.; Matsui, T.

2009-05-01

Cloud resolving models are typically used to examine the characteristics of clouds and precipitation and their relationship to radiation and the large-scale circulation. As such, they are not required to reproduce the exact location of each observed convective system, much less each individual cloud. Some of the most relevant information about clouds and precipitation is provided by instruments located on polar-orbiting satellite platforms, but these observations are intermittent "snapshots" in time, making assessment of model performance challenging. In contrast to direct comparison, model results can be evaluated statistically. This avoids the requirement for the model to reproduce the observed systems, while returning valuable information on the performance of the model in a climate-relevant sense. The focus of this talk is a model evaluation study, in which updates to the microphysics scheme used in a three-dimensional version of the Goddard Cumulus Ensemble (GCE) model are evaluated using statistics of observed clouds, precipitation, and radiation. We present the results of multiday (non-equilibrium) simulations of organized deep convection using single- and double-moment versions of a the model's cloud microphysical scheme. Statistics of TRMM multi-sensor derived clouds, precipitation, and radiative fluxes are used to evaluate the GCE results, as are simulated TRMM measurements obtained using a sophisticated instrument simulator suite. We present advantages and disadvantages of performing model comparisons in retrieval and measurement space and conclude by motivating the use of data assimilation techniques for analyzing and improving model parameterizations.

Assessment of phototoxicity, skin irritation, and sensitization potential of polystyrene and TiO2 nanoparticles

NASA Astrophysics Data System (ADS)

Park, Yoon-Hee; Jeong, Sang Hoon; Yi, Sang Min; Hyeok Choi, Byeong; Kim, Yu-Ri; Kim, In-Kyoung; Kim, Meyoung-Kon; Son, Sang Wook

2011-07-01

The human skin equivalent model (HSEM) is well known as an attractive alternative model for evaluation of dermal toxicity. However, only limited data are available on the usefulness of an HSEM for nanotoxicity testing. This study was designed to investigate cutaneous toxicity of polystyrene and TiO2 nanoparticles using cultured keratinocytes, an HSEM, and an animal model. In addition, we also evaluated the skin sensitization potential of nanoparticles using a local lymph node assay with incorporation of BrdU. Findings from the present study indicate that polystyrene and TiO2 nanoparticles do not induce phototoxicity, acute cutaneous irritation, or skin sensitization. Results from evaluation of the HSEMs correspond well with those from animal models. Our findings suggest that the HSEM might be a useful alternative model for evaluation of dermal nanotoxicity.

Evaluation of modeling for groundwater flow and tetrachloroethylene transport in the Milford-Souhegan glacial-drift aquifer at the Savage Municipal Well Superfund site, Milford, New Hampshire, 2011

USGS Publications Warehouse

Harte, Philip T.

2012-01-01

The U.S. Geological Survey and the New Hampshire Department of Environmental Services entered into a cooperative agreement to assist in the evaluation of remedy simulations of the MSGD aquifer that are being performed by various parties to track the remedial progress of the PCE plume. This report summarizes findings from this evaluation. Topics covered include description of groundwater flow and transport models used in the study of the Savage Superfund site (section 2), evaluation of models and their results (section 3), testing of several new simulations (section 4), an assessment of the representation of models to simulate field conditions (section 5), and an assessment of models as a tool in remedial operational decision making (section 6).

Effect of the Modified Glasgow Coma Scale Score Criteria for Mild Traumatic Brain Injury on Mortality Prediction: Comparing Classic and Modified Glasgow Coma Scale Score Model Scores of 13

PubMed Central

Mena, Jorge Humberto; Sanchez, Alvaro Ignacio; Rubiano, Andres M.; Peitzman, Andrew B.; Sperry, Jason L.; Gutierrez, Maria Isabel; Puyana, Juan Carlos

2011-01-01

Objective The Glasgow Coma Scale (GCS) classifies Traumatic Brain Injuries (TBI) as Mild (14–15); Moderate (9–13) or Severe (3–8). The ATLS modified this classification so that a GCS score of 13 is categorized as mild TBI. We investigated the effect of this modification on mortality prediction, comparing patients with a GCS of 13 classified as moderate TBI (Classic Model) to patients with GCS of 13 classified as mild TBI (Modified Model). Methods We selected adult TBI patients from the Pennsylvania Outcome Study database (PTOS). Logistic regressions adjusting for age, sex, cause, severity, trauma center level, comorbidities, and isolated TBI were performed. A second evaluation included the time trend of mortality. A third evaluation also included hypothermia, hypotension, mechanical ventilation, screening for drugs, and severity of TBI. Discrimination of the models was evaluated using the area under receiver operating characteristic curve (AUC). Calibration was evaluated using the Hoslmer-Lemershow goodness of fit (GOF) test. Results In the first evaluation, the AUCs were 0.922 (95 %CI, 0.917–0.926) and 0.908 (95 %CI, 0.903–0.912) for classic and modified models, respectively. Both models showed poor calibration (p<0.001). In the third evaluation, the AUCs were 0.946 (95 %CI, 0.943 – 0.949) and 0.938 (95 %CI, 0.934 –0.940) for the classic and modified models, respectively, with improvements in calibration (p=0.30 and p=0.02 for the classic and modified models, respectively). Conclusion The lack of overlap between ROC curves of both models reveals a statistically significant difference in their ability to predict mortality. The classic model demonstrated better GOF than the modified model. A GCS of 13 classified as moderate TBI in a multivariate logistic regression model performed better than a GCS of 13 classified as mild. PMID:22071923

AQMEII: A New International Initiative on Air Quality Model Evaluation

EPA Science Inventory

We provide a conceptual view of the process of evaluating regional-scale three-dimensional numerical photochemical air quality modeling system, based on an examination of existing approached to the evaluation of such systems as they are currently used in a variety of application....

What Is Essential in Developmental Evaluation? On Integrity, Fidelity, Adultery, Abstinence, Impotence, Long-Term Commitment, Integrity, and Sensitivity in Implementing Evaluation Models

ERIC Educational Resources Information Center

Patton, Michael Quinn

2016-01-01

Fidelity concerns the extent to which a specific evaluation sufficiently incorporates the core characteristics of the overall approach to justify labeling that evaluation by its designated name. Fidelity has traditionally meant implementing a model in exactly the same way each time following the prescribed steps and procedures. The essential…

Supporting Mediated Peer-Evaluation to Grade Answers to Open-Ended Questions

ERIC Educational Resources Information Center

De Marsico, Maria; Sciarrone, Filippo; Sterbini, Andrea; Temperini, Marco

2017-01-01

We show an approach to semi-automatic grading of answers given by students to open ended questions (open answers). We use both peer-evaluation and teacher evaluation. A learner is modeled by her Knowledge and her assessments quality (Judgment). The data generated by the peer- and teacher-evaluations, and by the learner models is represented by a…

Evaluation of "e-rater"® for the "Praxis I"®Writing Test. Research Report. ETS RR-15-03

ERIC Educational Resources Information Center

Ramineni, Chaitanya; Trapani, Catherine S.; Williamson, David M.

2015-01-01

Automated scoring models were trained and evaluated for the essay task in the "Praxis I"® writing test. Prompt-specific and generic "e-rater"® scoring models were built, and evaluation statistics, such as quadratic weighted kappa, Pearson correlation, and standardized differences in mean scores, were examined to evaluate the…

The Design and Implementation of a Model Evaluation Capability. 1975-76 Final Report. Title III Project.

ERIC Educational Resources Information Center

Austin Independent School District, TX. Office of Research and Evaluation.

The Austin Independent School District received an Elementary and Secondary Education Act Title III grant in 1973 to develop an internal research and evaluation capability. Funding was provided the resulting Office of Research and Evaluation (ORE) for three years. The foci of the original grant were (1) to develop a district evaluation model, (2)…

Training of Evaluators in the Third World: Implementation of the Action Training Model (ATM) in Kenya and Botswana.

ERIC Educational Resources Information Center

Bhola, H. S.

The Action Training Model (ATM) was developed for the delivery of evaluation training to development workers in Kenya and Botswana and implemented under the aegis of the German Foundation for International Development. Training of evaluators is a challenge in any context, but in the Third World environment, evaluation training offers special…

Building Effectiveness in Teaching through Targeted Evaluation and Response: Connecting Evaluation to Teaching Improvement in Higher Education

ERIC Educational Resources Information Center

Smith, Calvin

2008-01-01

This paper describes the development of a model for integrating student evaluation of teaching results with academic development opportunities, in new ways that take into account theoretical and practical developments in both fields. The model is described in terms of five phases or components: (1) the basic student evaluation system; (2) an…

Evaluation of Full Reynolds Stress Turbulence Models in FUN3D

NASA Technical Reports Server (NTRS)

Dudek, Julianne C.; Carlson, Jan-Renee

2017-01-01

Full seven-equation Reynolds stress turbulence models are a relatively new and promising tool for todays aerospace technology challenges. This paper uses two stress-omega full Reynolds stress models to evaluate challenging flows including shock-wave boundary layer interactions, separation and mixing layers. The Wilcox and the SSGLRR full second-moment Reynolds stress models are evaluated for four problems: a transonic two-dimensional diffuser, a supersonic axisymmetric compression corner, a compressible planar shear layer, and a subsonic axisymmetric jet. Simulation results are compared with experimental data and results using the more commonly used Spalart-Allmaras (SA) one-equation and the Menter Shear Stress Transport (SST) two-equation models.

Evaluating Measurement of Dynamic Constructs: Defining a Measurement Model of Derivatives

PubMed Central

Estabrook, Ryne

2015-01-01

While measurement evaluation has been embraced as an important step in psychological research, evaluating measurement structures with longitudinal data is fraught with limitations. This paper defines and tests a measurement model of derivatives (MMOD), which is designed to assess the measurement structure of latent constructs both for analyses of between-person differences and for the analysis of change. Simulation results indicate that MMOD outperforms existing models for multivariate analysis and provides equivalent fit to data generation models. Additional simulations show MMOD capable of detecting differences in between-person and within-person factor structures. Model features, applications and future directions are discussed. PMID:24364383

A model for evaluating academic research centers: Case study of the Asian/Pacific Islander Youth Violence Prevention Center.

PubMed

Nishimura, Stephanie T; Hishinuma, Earl S; Goebert, Deborah A; Onoye, Jane M M; Sugimoto-Matsuda, Jeanelle J

2018-02-01

To provide one model for evaluating academic research centers, given their vital role in addressing public health issues. A theoretical framework is described for a comprehensive evaluation plan for research centers. This framework is applied to one specific center by describing the center's Logic Model and Evaluation Plan, including a sample of the center's activities. Formative and summative evaluation information is summarized. In addition, a summary of outcomes is provided: improved practice and policy; reduction of risk factors and increase in protective factors; reduction of interpersonal youth violence in the community; and national prototype for prevention of interpersonal youth violence. Research centers are important mechanisms to advance science and improve people's quality of life. Because of their more infrastructure-intensive and comprehensive approach, they also require substantial resources for success, and thus, also require careful accountability. It is therefore important to comprehensively evaluate these centers. As provided herein, a more systematic and structured approach utilizing logic models, an evaluation plan, and successful processes can provide research centers with a functionally useful method in their evaluation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Metrics for Performance Evaluation of Patient Exercises during Physical Therapy.

PubMed

Vakanski, Aleksandar; Ferguson, Jake M; Lee, Stephen

2017-06-01

The article proposes a set of metrics for evaluation of patient performance in physical therapy exercises. Taxonomy is employed that classifies the metrics into quantitative and qualitative categories, based on the level of abstraction of the captured motion sequences. Further, the quantitative metrics are classified into model-less and model-based metrics, in reference to whether the evaluation employs the raw measurements of patient performed motions, or whether the evaluation is based on a mathematical model of the motions. The reviewed metrics include root-mean square distance, Kullback Leibler divergence, log-likelihood, heuristic consistency, Fugl-Meyer Assessment, and similar. The metrics are evaluated for a set of five human motions captured with a Kinect sensor. The metrics can potentially be integrated into a system that employs machine learning for modelling and assessment of the consistency of patient performance in home-based therapy setting. Automated performance evaluation can overcome the inherent subjectivity in human performed therapy assessment, and it can increase the adherence to prescribed therapy plans, and reduce healthcare costs.

Evaluation of six NEHRP B/C crustal amplification models proposed for use in western North America

USGS Publications Warehouse

Boore, David; Campbell, Kenneth W.

2016-01-01

We evaluate six crustal amplification models based on National Earthquake Hazards Reduction Program (NEHRP) B/C crustal profiles proposed for use in western North America (WNA) and often used in other active crustal regions where crustal properties are unknown. One of the models is based on an interpolation of generic rock velocity profiles previously proposed for WNA and central and eastern North America (CENA), in conjunction with material densities based on an updated velocity–density relationship. A second model is based on the velocity profile used to develop amplification factors for the Next Generation Attenuation (NGA)‐West2 project. A third model is based on a near‐surface velocity profile developed from the NGA‐West2 site database. A fourth model is based on velocity and density profiles originally proposed for use in CENA but recently used to represent crustal properties in California. We propose two alternatives to this latter model that more closely represent WNA crustal properties. We adopt a value of site attenuation (κ0) for each model that is either recommended by the author of the model or proposed by us. Stochastic simulation is used to evaluate the Fourier amplification factors and their impact on response spectra associated with each model. Based on this evaluation, we conclude that among the available models evaluated in this study the NEHRP B/C amplification model of Boore (2016) best represents median crustal amplification in WNA, although the amplification models based on the crustal profiles of Kamai et al. (2013, 2016, unpublished manuscript, see Data and Resources) and Yenier and Atkinson (2015), the latter adjusted to WNA crustal properties, can be used to represent epistemic uncertainty.

Habitat Suitability Index Models: Black-shouldered kite

USGS Publications Warehouse

Faanes, Craig A.; Howard, Rebecca J.

1987-01-01

A review and synthesis of existing information were used to develop a model for evaluating black-shouldered kite habitat quality. The model is scaled to produce an index between 0 (unsuitable habitat) to 1.0 (optimal habitat). Habitat suitability index models are designed for use with the Habitat Evaluation Procedures previously developed by the U.S. Fish and Wildlife Service. Guidelines for model application are provided.

Towards improved and more routine Earth system model evaluation in CMIP

DOE PAGES

Eyring, Veronika; Gleckler, Peter J.; Heinze, Christoph; ...

2016-11-01

The Coupled Model Intercomparison Project (CMIP) has successfully provided the climate community with a rich collection of simulation output from Earth system models (ESMs) that can be used to understand past climate changes and make projections and uncertainty estimates of the future. Confidence in ESMs can be gained because the models are based on physical principles and reproduce many important aspects of observed climate. More research is required to identify the processes that are most responsible for systematic biases and the magnitude and uncertainty of future projections so that more relevant performance tests can be developed. At the same time,more » there are many aspects of ESM evaluation that are well established and considered an essential part of systematic evaluation but have been implemented ad hoc with little community coordination. Given the diversity and complexity of ESM analysis, we argue that the CMIP community has reached a critical juncture at which many baseline aspects of model evaluation need to be performed much more efficiently and consistently. We provide a perspective and viewpoint on how a more systematic, open, and rapid performance assessment of the large and diverse number of models that will participate in current and future phases of CMIP can be achieved, and announce our intention to implement such a system for CMIP6. Accomplishing this could also free up valuable resources as many scientists are frequently "re-inventing the wheel" by re-writing analysis routines for well-established analysis methods. A more systematic approach for the community would be to develop and apply evaluation tools that are based on the latest scientific knowledge and observational reference, are well suited for routine use, and provide a wide range of diagnostics and performance metrics that comprehensively characterize model behaviour as soon as the output is published to the Earth System Grid Federation (ESGF). The CMIP infrastructure enforces data standards and conventions for model output and documentation accessible via the ESGF, additionally publishing observations (obs4MIPs) and reanalyses (ana4MIPs) for model intercomparison projects using the same data structure and organization as the ESM output. This largely facilitates routine evaluation of the ESMs, but to be able to process the data automatically alongside the ESGF, the infrastructure needs to be extended with processing capabilities at the ESGF data nodes where the evaluation tools can be executed on a routine basis. Efforts are already underway to develop community-based evaluation tools, and we encourage experts to provide additional diagnostic codes that would enhance this capability for CMIP. And, at the same time, we encourage the community to contribute observations and reanalyses for model evaluation to the obs4MIPs and ana4MIPs archives. The intention is to produce through the ESGF a widely accepted quasi-operational evaluation framework for CMIP6 that would routinely execute a series of standardized evaluation tasks. Over time, as this capability matures, we expect to produce an increasingly systematic characterization of models which, compared with early phases of CMIP, will more quickly and openly identify the strengths and weaknesses of the simulations. This will also reveal whether long-standing model errors remain evident in newer models and will assist modelling groups in improving their models. Finally, this framework will be designed to readily incorporate updates, including new observations and additional diagnostics and metrics as they become available from the research community.« less

Function-based payment model for inpatient medical rehabilitation: an evaluation.

PubMed

Sutton, J P; DeJong, G; Wilkerson, D

1996-07-01

To describe the components of a function-based prospective payment model for inpatient medical rehabilitation that parallels diagnosis-related groups (DRGs), to evaluate this model in relation to stakeholder objectives, and to detail the components of a quality of care incentive program that, when combined with this payment model, creates an incentive for provides to maximize functional outcomes. This article describes a conceptual model, involving no data collection or data synthesis. The basic payment model described parallels DRGs. Information on the potential impact of this model on medical rehabilitation is gleaned from the literature evaluating the impact of DRGs. The conceptual model described is evaluated against the results of a Delphi Survey of rehabilitation providers, consumers, policymakers, and researchers previously conducted by members of the research team. The major shortcoming of a function-based prospective payment model for inpatient medical rehabilitation is that it contains no inherent incentive to maximize functional outcomes. Linkage of reimbursement to outcomes, however, by withholding a fixed proportion of the standard FRG payment amount, placing that amount in a "quality of care" pool, and distributing that pool annually among providers whose predesignated, facility-level, case-mix-adjusted outcomes are attained, may be one strategy for maximizing outcome goals.

models of congenital heart disease.

PubMed

Biglino, Giovanni; Capelli, Claudio; Leaver, Lindsay-Kay; Schievano, Silvia; Taylor, Andrew M; Wray, Jo

2015-01-01

To develop a participatory approach in the evaluation of 3D printed patient-specific models of congenital heart disease (CHD) with different stakeholders who would potentially benefit from the technology (patients, parents, clinicians and nurses). Workshops, focus groups and teaching sessions were organised, targeting different stakeholders. Sessions involved displaying and discussing different 3D models of CHD. Model evaluation involved response counts from questionnaires and thematic analysis of audio-recorded discussions and written feedback. Stakeholders’ responses indicated the scope and potential for clinical translation of 3D models. As tangible, three-dimensional artefacts, these can have a role in communicative processes. Their patient-specific quality is also important in relation to individual characteristics of CHD. Patients indicated that 3D models can help them visualise ‘what’s going on inside’. Parents agreed that models can spark curiosity in young people. Clinicians indicated that teaching might be the most relevant application. Nurses agreed that 3D models improved their learning experience during a CHD course. Engagement of different stakeholders to evaluate 3D printing technology for CHD identified the potential of the models for improving patient– doctor communication, patient empowerment and training. A participatory approach could benefit the clinical evaluation and translation of 3D printing technology.

Formal implementation of a performance evaluation model for the face recognition system.

PubMed

Shin, Yong-Nyuo; Kim, Jason; Lee, Yong-Jun; Shin, Woochang; Choi, Jin-Young

2008-01-01

Due to usability features, practical applications, and its lack of intrusiveness, face recognition technology, based on information, derived from individuals' facial features, has been attracting considerable attention recently. Reported recognition rates of commercialized face recognition systems cannot be admitted as official recognition rates, as they are based on assumptions that are beneficial to the specific system and face database. Therefore, performance evaluation methods and tools are necessary to objectively measure the accuracy and performance of any face recognition system. In this paper, we propose and formalize a performance evaluation model for the biometric recognition system, implementing an evaluation tool for face recognition systems based on the proposed model. Furthermore, we performed evaluations objectively by providing guidelines for the design and implementation of a performance evaluation system, formalizing the performance test process.

AQMEII Phase 2: Overview and WRF/CMAQ Application over North America

EPA Science Inventory

In this study, we provide an overview of the second phase of the Air Quality Model Evaluation International Initiative (AQMEII). Activities in this phase are focused on the application and evaluation of coupled meteorologychemistry models. Participating modeling systems are being...

Modeling the effect of land use change on hydrology of a forested watershed in coastal South Carolina.

Treesearch

Zhaohua Dai; Devendra M. Amatya; Ge Sun; Changsheng Li; Carl C. Trettin; Harbin Li

2009-01-01

Since hydrology is one of main factors controlling wetland functions, hydrologic models are useful for evaluating the effects of land use change on we land ecosystems. We evaluated two process-based hydrologic models with...

A YEAR-LONG MM5 EVALUATION USING A MODEL EVALUATION TOOLKIT

EPA Science Inventory

Air quality modeling has expanded in both sophistication and application over the past decade. Meteorological and air quality modeling tools are being used for research, forecasting, and regulatory related emission control strategies. Results from air quality simulations have far...

EVALUATION OF ACID DEPOSITION MODELS USING PRINCIPAL COMPONENT SPACES

EPA Science Inventory

An analytical technique involving principal components analysis is proposed for use in the evaluation of acid deposition models. elationships among model predictions are compared to those among measured data, rather than the more common one-to-one comparison of predictions to mea...

[A new model fo the evaluation of measurements of the neurocranium].

PubMed

Seidler, H; Wilfing, H; Weber, G; Traindl-Prohazka, M; zur Nedden, D; Platzer, W

1993-12-01

A simple and user-friendly model for trigonometric description of the neurocranium based on newly defined points of measurement is presented. This model not only provides individual description, but also allows for an evaluation of developmental and phylogenetic aspects.

A reexamination of age-related variation in body weight and morphometry of Maryland nutria

USGS Publications Warehouse

Sherfy, M.H.; Mollett, T.A.; McGowan, K.R.; Daugherty, S.L.

2006-01-01

Age-related variation in morphometry has been documented for many species. Knowledge of growth patterns can be useful for modeling energetics, detecting physiological influences on populations, and predicting age. These benefits have shown value in understanding population dynamics of invasive species, particularly in developing efficient control and eradication programs. However, development and evaluation of descriptive and predictive models is a critical initial step in this process. Accordingly, we used data from necropsies of 1,544 nutria (Myocastor coypus) collected in Maryland, USA, to evaluate the accuracy of previously published models for prediction of nutria age from body weight. Published models underestimated body weights of our animals, especially for ages <3. We used cross-validation procedures to develop and evaluate models for describing nutria growth patterns and for predicting nutria age. We derived models from a randomly selected model-building data set (n = 192-193 M, 217-222 F) and evaluated them with the remaining animals (n = 487-488 M, 642-647 F). We used nonlinear regression to develop Gompertz growth-curve models relating morphometric variables to age. Predicted values of morphometric variables fell within the 95% confidence limits of their true values for most age classes. We also developed predictive models for estimating nutria age from morphometry, using linear regression of log-transformed age on morphometric variables. The evaluation data set corresponded with 95% prediction intervals from the new models. Predictive models for body weight and length provided greater accuracy and less bias than models for foot length and axillary girth. Our growth models accurately described age-related variation in nutria morphometry, and our predictive models provided accurate estimates of ages from morphometry that will be useful for live-captured individuals. Our models offer better accuracy and precision than previously published models, providing a capacity for modeling energetics and growth patterns of Maryland nutria as well as an empirical basis for determining population age structure from live-captured animals.

Using RUFDATA to guide a logic model for a quality assurance process in an undergraduate university program.

PubMed

Sherman, Paul David

2016-04-01

This article presents a framework to identify key mechanisms for developing a logic model blueprint that can be used for an impending comprehensive evaluation of an undergraduate degree program in a Canadian university. The evaluation is a requirement of a comprehensive quality assurance process mandated by the university. A modified RUFDATA (Saunders, 2000) evaluation model is applied as an initiating framework to assist in decision making to provide a guide for conceptualizing a logic model for the quality assurance process. This article will show how an educational evaluation is strengthened by employing a RUFDATA reflective process in exploring key elements of the evaluation process, and then translating this information into a logic model format that could serve to offer a more focussed pathway for the quality assurance activities. Using preliminary program evaluation data from two key stakeholders of the undergraduate program as well as an audit of the curriculum's course syllabi, a case is made for, (1) the importance of inclusivity of key stakeholders participation in the design of the evaluation process to enrich the authenticity and accuracy of program participants' feedback, and (2) the diversification of data collection methods to ensure that stakeholders' narrative feedback is given ample exposure. It is suggested that the modified RUFDATA/logic model framework be applied to all academic programs at the university undergoing the quality assurance process at the same time so that economies of scale may be realized. Copyright © 2015 Elsevier Ltd. All rights reserved.

Chaparral Model 60 Infrasound Sensor Evaluation.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Slad, George William; Merchant, Bion J.

2016-03-01

Sandia National Laboratories has tested and evaluated an infrasound sensor, the Model 60 manufactured by Chaparral Physics, a Division of Geophysical Institute of the University of Alaska, Fairbanks. The purpose of the infrasound sensor evaluation was to determine a measured sensitivity, transfer function, power, self-noise, dynamic range, and seismic sensitivity. The Model 60 infrasound sensor is a new sensor developed by Chaparral Physics intended to be a small, rugged sensor used in more flexible application conditions.

Evaluation of the 29-km Eta Model. Part I: Objective Verification at Three Selected Stations

NASA Technical Reports Server (NTRS)

Manobianco, John; Nutter, Paul

1998-01-01

A subjective evaluation of the National Centers for Environmental Prediction 29-km (meso-) eta model during the 1996 warm (May-August) and cool (October-January) seasons is described. The overall evaluation assessed the utility of the model for operational weather forecasting by the U.S. Air Force 45th Weather Squadron, National Weather Service (NWS) Spaceflight Meteorology Group (SMG) and NWS Office in Melbourne, FL.

Vestibular models for design and evaluation of flight simulator motion

NASA Technical Reports Server (NTRS)

Bussolari, S. R.; Sullivan, R. B.; Young, L. R.

1986-01-01

The use of spatial orientation models in the design and evaluation of control systems for motion-base flight simulators is investigated experimentally. The development of a high-fidelity motion drive controller using an optimal control approach based on human vestibular models is described. The formulation and implementation of the optimal washout system are discussed. The effectiveness of the motion washout system was evaluated by studying the response of six motion washout systems to the NASA/AMES Vertical Motion Simulator for a single dash-quick-stop maneuver. The effects of the motion washout system on pilot performance and simulator acceptability are examined. The data reveal that human spatial orientation models are useful for the design and evaluation of flight simulator motion fidelity.

Evaluation of COSMO-ART in the Framework of the Air Quality Model Evaluation International Initiative (AQMEII)

NASA Astrophysics Data System (ADS)

Giordano, Lea; Brunner, Dominik; Im, Ulas; Galmarini, Stefano

2014-05-01

The Air Quality Model Evaluation International Initiative (AQMEII) coordinated by the EC-JRC and US-EPA, promotes since 2008 research on regional air quality model evaluation across the atmospheric modelling communities of Europe and North America. AQMEII has now reached its Phase 2 that is dedicated to the evaluation of on-line coupled chemistry-meteorology models as opposed to Phase 1 where only off-line models were considered. At European level, AQMEII collaborates with the COST Action "European framework for on-line integrated air quality and meteorology modelling" (EuMetChem). All European groups participating in AQMEII performed simulations over the same spatial domain (Europe at a resolution of about 20 km) and using the same simulation strategy (e.g. no nudging allowed) and the same input data as much as possible. The initial and boundary conditions (IC/BC) were shared between all groups. Emissions were provided by the TNO-MACC database for anthropogenic emissions and the FMI database for biomass burning emissions. Chemical IC/BC data were taken from IFS-MOZART output, and meteorological IC/BC from the ECWMF global model. Evaluation data sets were collected by the Joint Research Center (JRC) and include measurements from surface in situ networks (AirBase and EMEP), vertical profiles from ozone sondes and aircraft (MOZAIC), and remote sensing (AERONET, satellites). Since Phase 2 focuses on on-line coupled models, a special effort is devoted to the detailed speciation of particulate matter components, with the goal of studying feedback processes. For the AQMEII exercise, COSMO-ART has been run with 40 levels of vertical resolution, and a chemical scheme that includes the SCAV module of Knote and Brunner (ACP 2013) for wet-phase chemistry and the SOA treatment according to VBS (volatility basis set) approach (Athanasopoulou et al., ACP 2013). The COSMO-ART evaluation shows that, next to a good performance in the meteorology, the gas phase chemistry is well captured throughout the year; the few cases showing a systematic underestimation of chemical concentrations arise as a consequence of the boundary conditions. Through this exercise we have identified the main critical issues in the COSMO-ART performance: sea salt and dust particulate matter components. The AQMEII exercise has provided an excellent platform to evaluate the COSMO-ART performance against both measurement data and other European regional on-line coupled models. From the analysis we have been able to identify specific model deficiencies and situations where the model cannot satisfactorily reproduce the data. Our future work will be focused on improving their modelling.

Evaluating model structure adequacy: The case of the Maggia Valley groundwater system, southern Switzerland

USGS Publications Warehouse

Hill, Mary C.; L. Foglia,; S. W. Mehl,; P. Burlando,

2013-01-01

Model adequacy is evaluated with alternative models rated using model selection criteria (AICc, BIC, and KIC) and three other statistics. Model selection criteria are tested with cross-validation experiments and insights for using alternative models to evaluate model structural adequacy are provided. The study is conducted using the computer codes UCODE_2005 and MMA (MultiModel Analysis). One recharge alternative is simulated using the TOPKAPI hydrological model. The predictions evaluated include eight heads and three flows located where ecological consequences and model precision are of concern. Cross-validation is used to obtain measures of prediction accuracy. Sixty-four models were designed deterministically and differ in representation of river, recharge, bedrock topography, and hydraulic conductivity. Results include: (1) What may seem like inconsequential choices in model construction may be important to predictions. Analysis of predictions from alternative models is advised. (2) None of the model selection criteria consistently identified models with more accurate predictions. This is a disturbing result that suggests to reconsider the utility of model selection criteria, and/or the cross-validation measures used in this work to measure model accuracy. (3) KIC displayed poor performance for the present regression problems; theoretical considerations suggest that difficulties are associated with wide variations in the sensitivity term of KIC resulting from the models being nonlinear and the problems being ill-posed due to parameter correlations and insensitivity. The other criteria performed somewhat better, and similarly to each other. (4) Quantities with high leverage are more difficult to predict. The results are expected to be generally applicable to models of environmental systems.

Developing a good practice model to evaluate the effectiveness of comprehensive primary health care in local communities

PubMed Central

2014-01-01

Background This paper describes the development of a model of Comprehensive Primary Health Care (CPHC) applicable to the Australian context. CPHC holds promise as an effective model of health system organization able to improve population health and increase health equity. However, there is little literature that describes and evaluates CPHC as a whole, with most evaluation focusing on specific programs. The lack of a consensus on what constitutes CPHC, and the complex and context-sensitive nature of CPHC are all barriers to evaluation. Methods The research was undertaken in partnership with six Australian primary health care services: four state government funded and managed services, one sexual health non-government organization, and one Aboriginal community controlled health service. A draft model was crafted combining program logic and theory-based approaches, drawing on relevant literature, 68 interviews with primary health care service staff, and researcher experience. The model was then refined through an iterative process involving two to three workshops at each of the six participating primary health care services, engaging health service staff, regional health executives and central health department staff. Results The resultant Southgate Model of CPHC in Australia model articulates the theory of change of how and why CPHC service components and activities, based on the theory, evidence and values which underpin a CPHC approach, are likely to lead to individual and population health outcomes and increased health equity. The model captures the importance of context, the mechanisms of CPHC, and the space for action services have to work within. The process of development engendered and supported collaborative relationships between researchers and stakeholders and the product provided a description of CPHC as a whole and a framework for evaluation. The model was endorsed at a research symposium involving investigators, service staff, and key stakeholders. Conclusions The development of a theory-based program logic model provided a framework for evaluation that allows the tracking of progress towards desired outcomes and exploration of the particular aspects of context and mechanisms that produce outcomes. This is important because there are no existing models which enable the evaluation of CPHC services in their entirety. PMID:24885812

A Conceptual Framework for Evaluating Higher Education Institutions

ERIC Educational Resources Information Center

Chinta, Ravi; Kebritchi, Mansureh; Ellias, Janelle

2016-01-01

Purpose: Performance evaluation is a topic that has been researched and practiced extensively in business organizations but has received scant attention in higher education institutions. A review of literature revealed that context, input, process, product (CIPP) model is an appropriate performance evaluation model for higher education…

Information and complexity measures for hydrologic model evaluation

USDA-ARS?s Scientific Manuscript database

Hydrological models are commonly evaluated through the residual-based performance measures such as the root-mean square error or efficiency criteria. Such measures, however, do not evaluate the degree of similarity of patterns in simulated and measured time series. The objective of this study was to...

Comprehensive evaluation system of intelligent urban growth

NASA Astrophysics Data System (ADS)

Li, Lian-Yan; Ren, Xiao-Bin

2017-06-01

With the rapid urbanization of the world, urban planning has become increasingly important and necessary to ensure people have access to equitable and sustainable homes, resources and jobs.This article is to talk about building an intelligent city evaluation system.First,using System Analysis Model(SAM) which concludes literature data analysis and stepwise regression analysis to describe intelligent growth scientifically and obtain the evaluation index. Then,using the improved entropy method to obtain the weight of the evaluation index.Afterwards, establishing a complete Smart Growth Comprehensive Evaluation Model(SGCEM).Finally,testing the correctness of the model.Choosing Otago(New Zealand )and Yumen(China) as research object by data mining and SGCEM model,then we get Yumen and Otago’s rational degree’s values are 0.3485 and 0.5376 respectively. It’s believed that the Otago’s smart level is higher,and it is found that the estimated value of rationality is consistent with the reality.

Evaluating gridded crop model simulations of evapotranspiration and irrigation using survey and remotely sensed data

NASA Astrophysics Data System (ADS)

Lopez Bobeda, J. R.

2017-12-01

The increasing use of groundwater for irrigation of crops has exacerbated groundwater sustainability issues faced by water limited regions. Gridded, process-based crop models have the potential to help farmers and policymakers asses the effects water shortages on yield and devise new strategies for sustainable water use. Gridded crop models are typically calibrated and evaluated using county-level survey data of yield, planting dates, and maturity dates. However, little is known about the ability of these models to reproduce observed crop evapotranspiration and water use at regional scales. The aim of this work is to evaluate a gridded version of the Decision Support System for Agrotechnology Transfer (DSSAT) crop model over the continental United States. We evaluated crop seasonal evapotranspiration over 5 arc-minute grids, and irrigation water use at the county level. Evapotranspiration was assessed only for rainfed agriculture to test the model evapotranspiration equations separate from the irrigation algorithm. Model evapotranspiration was evaluated against the Atmospheric Land Exchange Inverse (ALEXI) modeling product. Using a combination of the USDA crop land data layer (CDL) and the USGS Moderate Resolution Imaging Spectroradiometer Irrigated Agriculture Dataset for the United States (MIrAD-US), we selected only grids with more than 60% of their area planted with the simulated crops (corn, cotton, and soybean), and less than 20% of their area irrigated. Irrigation water use was compared against the USGS county level irrigated agriculture water use survey data. Simulated gridded data were aggregated to county level using USDA CDL and USGS MIrAD-US. Only counties where 70% or more of the irrigated land was corn, cotton, or soybean were selected for the evaluation. Our results suggest that gridded crop models can reasonably reproduce crop evapotranspiration at the country scale (RRMSE = 10%).

Evaluation of atmospheric density models and preliminary functional specifications for the Langley Atmospheric Information Retrieval System (LAIRS)

NASA Technical Reports Server (NTRS)

Lee, T.; Boland, D. F., Jr.

1980-01-01

This document presents the results of an extensive survey and comparative evaluation of current atmosphere and wind models for inclusion in the Langley Atmospheric Information Retrieval System (LAIRS). It includes recommended models for use in LAIRS, estimated accuracies for the recommended models, and functional specifications for the development of LAIRS.

Evaluation of habitat suitability index models for assessing biotic resources

Treesearch

John C. Rennie; Joseph D. Clark; James M. Sweeney

2000-01-01

Existing habitat suitability index (HSI) models are evaluated for assessing the biotic resources on Champion International Corporation (CIC) lands with data from a standard and an expanded timber inventory. Forty HSI models for 34 species that occur in the Southern Appalachians have been identified from the literature. All of the variables for 14 models are provided (...

Integrating distributional, spatial prioritization, and individual-based models to evaluate potential critical habitat networks: A case study using the Northern Spotted Owl

EPA Science Inventory

As part of the northern spotted owl recovery planning effort, we evaluated a series of alternative critical habitat scenarios using a species-distribution model (MaxEnt), a conservation-planning model (Zonation), and an individual-based population model (HexSim). With this suite ...

Catalog of Wargaming and Military Simulation Models

DTIC Science & Technology

1989-09-01

and newly developed software models. This system currently (and will in the near term) supports battle force architecture design and evaluation...aborted air refuelings, or replacement aircraft. PLANNED IMPROVEMENTS AND MODIFICATIONS: Completion of model. INPUT: Input fields are required to...vehicle mobility evaluation model). PROPONENT: Mobility Systems Division, Geotechnical Laboratory, U.S. Army Engineer Waterways Experiment Station

An Evaluation of Some Models for Culture-Fair Selection.

ERIC Educational Resources Information Center

Petersen, Nancy S.; Novick, Melvin R.

Models proposed by Cleary, Thorndike, Cole, Linn, Einhorn and Bass, Darlington, and Gross and Su for analyzing bias in the use of tests in a selection strategy are surveyed. Several additional models are also introduced. The purpose is to describe, compare, contrast, and evaluate these models while extracting such useful ideas as may be found in…

Dynamic evaluation of the CMAQv5.0 modeling system: Assessing the model’s ability to simulate ozone changes due to NOx emission reductions

EPA Science Inventory

Regional air quality models are frequently used for regulatory applications to predict changes in air quality due to changes in emissions or changes in meteorology. Dynamic model evaluation is thus an important step in establishing credibility in the model predicted pollutant re...

Satisfiers and Dissatisfiers: A Two-Factor Model for Website Design and Evaluation.

ERIC Educational Resources Information Center

Zhang, Ping; von Dran, Gisela M.

2000-01-01

Investigates Web site design factors and their impact from a theoretical perspective. Presents a two-factor model that can guide Web site design and evaluation. According to the model, there are two types of design factors: hygiene and motivator. Results showed that the two-factor model provides a means for Web-user interface studies. Provides…

ENSEMBLE and AMET: Two Systems and Approaches to a Harmonized, Simplified and Efficient Facility for Air Quality Models Development and Evaluation

EPA Science Inventory

The complexity of air quality modeling systems, air quality monitoring data make ad-hoc systems for model evaluation important aids to the modeling community. Among those are the ENSEMBLE system developed by the EC-Joint Research Center, and the AMET software developed by the US-...

Evaluation of operational online-coupled regional air quality models over Europe and North America in the context of AQMEII phase 2. Part II: Particulate Matter

EPA Science Inventory

The second phase of the Air Quality Model Evaluation International Initiative (AQMEII) brought together seventeen modeling groups from Europe and North America, running eight operational online-coupled air quality models over Europe and North America on common emissions and bound...

Evaluation of operational online-coupled regional air quality models over Europe and North America in the context of AQMEII phase 2. Part 1: Ozone”

EPA Science Inventory

The second phase of the Air Quality Model Evaluation International Initiative (AQMEII) brought together sixteen modeling groups from Europe and North America, running eight operational online-coupled air quality models over Europe and North America on common emissions and boundar...

Varicella infection modeling.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Jones, Katherine A.; Finley, Patrick D.; Moore, Thomas W.

2013-09-01

Infectious diseases can spread rapidly through healthcare facilities, resulting in widespread illness among vulnerable patients. Computational models of disease spread are useful for evaluating mitigation strategies under different scenarios. This report describes two infectious disease models built for the US Department of Veteran Affairs (VA) motivated by a Varicella outbreak in a VA facility. The first model simulates disease spread within a notional contact network representing staff and patients. Several interventions, along with initial infection counts and intervention delay, were evaluated for effectiveness at preventing disease spread. The second model adds staff categories, location, scheduling, and variable contact rates tomore » improve resolution. This model achieved more accurate infection counts and enabled a more rigorous evaluation of comparative effectiveness of interventions.« less

An Overview of Atmospheric Chemistry and Air Quality Modeling

NASA Technical Reports Server (NTRS)

Johnson, Matthew S.

2017-01-01

This presentation will include my personal research experience and an overview of atmospheric chemistry and air quality modeling to the participants of the NASA Student Airborne Research Program (SARP 2017). The presentation will also provide examples on ways to apply airborne observations for chemical transport (CTM) and air quality (AQ) model evaluation. CTM and AQ models are important tools in understanding tropospheric-stratospheric composition, atmospheric chemistry processes, meteorology, and air quality. This presentation will focus on how NASA scientist currently apply CTM and AQ models to better understand these topics. Finally, the importance of airborne observation in evaluating these topics and how in situ and remote sensing observations can be used to evaluate and improve CTM and AQ model predictions will be highlighted.

Mobility Models for Systems Evaluation

NASA Astrophysics Data System (ADS)

Musolesi, Mirco; Mascolo, Cecilia

Mobility models are used to simulate and evaluate the performance of mobile wireless systems and the algorithms and protocols at the basis of them. The definition of realistic mobility models is one of the most critical and, at the same time, difficult aspects of the simulation of applications and systems designed for mobile environments. There are essentially two possible types of mobility patterns that can be used to evaluate mobile network protocols and algorithms by means of simulations: traces and synthetic models [130]. Traces are obtained by means of measurements of deployed systems and usually consist of logs of connectivity or location information, whereas synthetic models are mathematical models, such as sets of equations, which try to capture the movement of the devices.

Depicting the logic of three evaluation theories.

PubMed

Hansen, Mark; Alkin, Marvin C; Wallace, Tanner Lebaron

2013-06-01

Here, we describe the development of logic models depicting three theories of evaluation practice: Practical Participatory (Cousins & Whitmore, 1998), Values-engaged (Greene, 2005a, 2005b), and Emergent Realist (Mark et al., 1998). We begin with a discussion of evaluation theory and the particular theories that were chosen for our analysis. We then outline the steps involved in constructing the models. The theoretical prescriptions and claims represented here follow a logic model template developed at the University Wisconsin-Extension (Taylor-Powell & Henert, 2008), which also closely aligns with Mark's (2008) framework for research on evaluation. Copyright © 2012 Elsevier Ltd. All rights reserved.

A holistic model for evaluating the impact of individual technology-enhanced learning resources.

PubMed

Pickering, James D; Joynes, Viktoria C T

2016-12-01

The use of technology within education has now crossed the Rubicon; student expectations, the increasing availability of both hardware and software and the push to fully blended learning environments mean that educational institutions cannot afford to turn their backs on technology-enhanced learning (TEL). The ability to meaningfully evaluate the impact of TEL resources nevertheless remains problematic. This paper aims to establish a robust means of evaluating individual resources and meaningfully measure their impact upon learning within the context of the program in which they are used. Based upon the experience of developing and evaluating a range of mobile and desktop based TEL resources, this paper outlines a new four-stage evaluation process, taking into account learner satisfaction, learner gain, and the impact of a resource on both the individual and the institution in which it has been adapted. A new multi-level model of TEL resource evaluation is proposed, which includes a preliminary evaluation of need, learner satisfaction and gain, learner impact and institutional impact. Each of these levels are discussed in detail, and in relation to existing TEL evaluation frameworks. This paper details a holistic, meaningful evaluation model for individual TEL resources within the specific context in which they are used. It is proposed that this model is adopted to ensure that TEL resources are evaluated in a more meaningful and robust manner than is currently undertaken.

Comprehensive Aspectual UML approach to support AspectJ.

PubMed

Magableh, Aws; Shukur, Zarina; Ali, Noorazean Mohd

2014-01-01

Unified Modeling Language is the most popular and widely used Object-Oriented modelling language in the IT industry. This study focuses on investigating the ability to expand UML to some extent to model crosscutting concerns (Aspects) to support AspectJ. Through a comprehensive literature review, we identify and extensively examine all the available Aspect-Oriented UML modelling approaches and find that the existing Aspect-Oriented Design Modelling approaches using UML cannot be considered to provide a framework for a comprehensive Aspectual UML modelling approach and also that there is a lack of adequate Aspect-Oriented tool support. This study also proposes a set of Aspectual UML semantic rules and attempts to generate AspectJ pseudocode from UML diagrams. The proposed Aspectual UML modelling approach is formally evaluated using a focus group to test six hypotheses regarding performance; a "good design" criteria-based evaluation to assess the quality of the design; and an AspectJ-based evaluation as a reference measurement-based evaluation. The results of the focus group evaluation confirm all the hypotheses put forward regarding the proposed approach. The proposed approach provides a comprehensive set of Aspectual UML structural and behavioral diagrams, which are designed and implemented based on a comprehensive and detailed set of AspectJ programming constructs.

The Triangle Model for evaluating the effect of health information technology on healthcare quality and safety

PubMed Central

Kern, Lisa M; Abramson, Erika; Kaushal, Rainu

2011-01-01

With the proliferation of relatively mature health information technology (IT) systems with large numbers of users, it becomes increasingly important to evaluate the effect of these systems on the quality and safety of healthcare. Previous research on the effectiveness of health IT has had mixed results, which may be in part attributable to the evaluation frameworks used. The authors propose a model for evaluation, the Triangle Model, developed for designing studies of quality and safety outcomes of health IT. This model identifies structure-level predictors, including characteristics of: (1) the technology itself; (2) the provider using the technology; (3) the organizational setting; and (4) the patient population. In addition, the model outlines process predictors, including (1) usage of the technology, (2) organizational support for and customization of the technology, and (3) organizational policies and procedures about quality and safety. The Triangle Model specifies the variables to be measured, but is flexible enough to accommodate both qualitative and quantitative approaches to capturing them. The authors illustrate this model, which integrates perspectives from both health services research and biomedical informatics, with examples from evaluations of electronic prescribing, but it is also applicable to a variety of types of health IT systems. PMID:21857023

Comprehensive Aspectual UML Approach to Support AspectJ

PubMed Central

Magableh, Aws; Shukur, Zarina; Mohd. Ali, Noorazean

2014-01-01

Unified Modeling Language is the most popular and widely used Object-Oriented modelling language in the IT industry. This study focuses on investigating the ability to expand UML to some extent to model crosscutting concerns (Aspects) to support AspectJ. Through a comprehensive literature review, we identify and extensively examine all the available Aspect-Oriented UML modelling approaches and find that the existing Aspect-Oriented Design Modelling approaches using UML cannot be considered to provide a framework for a comprehensive Aspectual UML modelling approach and also that there is a lack of adequate Aspect-Oriented tool support. This study also proposes a set of Aspectual UML semantic rules and attempts to generate AspectJ pseudocode from UML diagrams. The proposed Aspectual UML modelling approach is formally evaluated using a focus group to test six hypotheses regarding performance; a “good design” criteria-based evaluation to assess the quality of the design; and an AspectJ-based evaluation as a reference measurement-based evaluation. The results of the focus group evaluation confirm all the hypotheses put forward regarding the proposed approach. The proposed approach provides a comprehensive set of Aspectual UML structural and behavioral diagrams, which are designed and implemented based on a comprehensive and detailed set of AspectJ programming constructs. PMID:25136656

Evaluating the Theoretic Adequacy and Applied Potential of Computational Models of the Spacing Effect.

PubMed

Walsh, Matthew M; Gluck, Kevin A; Gunzelmann, Glenn; Jastrzembski, Tiffany; Krusmark, Michael

2018-06-01

The spacing effect is among the most widely replicated empirical phenomena in the learning sciences, and its relevance to education and training is readily apparent. Yet successful applications of spacing effect research to education and training is rare. Computational modeling can provide the crucial link between a century of accumulated experimental data on the spacing effect and the emerging interest in using that research to enable adaptive instruction. In this paper, we review relevant literature and identify 10 criteria for rigorously evaluating computational models of the spacing effect. Five relate to evaluating the theoretic adequacy of a model, and five relate to evaluating its application potential. We use these criteria to evaluate a novel computational model of the spacing effect called the Predictive Performance Equation (PPE). Predictive Performance Equation combines elements of earlier models of learning and memory including the General Performance Equation, Adaptive Control of Thought-Rational, and the New Theory of Disuse, giving rise to a novel computational account of the spacing effect that performs favorably across the complete sets of theoretic and applied criteria. We implemented two other previously published computational models of the spacing effect and compare them to PPE using the theoretic and applied criteria as guides. Copyright © 2018 Cognitive Science Society, Inc.

Why we do what we do: a theoretical evaluation of the integrated practice model for forensic nursing science.

PubMed

Valentine, Julie L

2014-01-01

An evaluation of the Integrated Practice Model for Forensic Nursing Science () is presented utilizing methods outlined by . A brief review of nursing theory basics and evaluation methods by Meleis is provided to enhance understanding of the ensuing theoretical evaluation and critique. The Integrated Practice Model for Forensic Nursing Science, created by forensic nursing pioneer Virginia Lynch, captures the theories, assumptions, concepts, and propositions inherent in forensic nursing practice and science. The historical background of the theory is explored as Lynch's model launched the role development of forensic nursing practice as both a nursing and forensic science specialty. It is derived from a combination of nursing, sociological, and philosophical theories to reflect the grounding of forensic nursing in the nursing, legal, psychological, and scientific communities. As Lynch's model is the first inception of forensic nursing theory, it is representative of a conceptual framework although the title implies a practice theory. The clarity and consistency displayed in the theory's structural components of assumptions, concepts, and propositions are analyzed. The model is described and evaluated. A summary of the strengths and limitations of the model is compiled followed by application to practice, education, and research with suggestions for ongoing theory development.

Uranium resource assessment through statistical analysis of exploration geochemical and other data. Final report. [Codes EVAL, SURE

DOE Office of Scientific and Technical Information (OSTI.GOV)

Koch, G.S. Jr.; Howarth, R.J.; Schuenemeyer, J.H.

1981-02-01

We have developed a procedure that can help quadrangle evaluators to systematically summarize and use hydrogeochemical and stream sediment reconnaissance (HSSR) and occurrence data. Although we have not provided an independent estimate of uranium endowment, we have devised a methodology that will provide this independent estimate when additional calibration is done by enlarging the study area. Our statistical model for evaluation (system EVAL) ranks uranium endowment for each quadrangle. Because using this model requires experience in geology, statistics, and data analysis, we have also devised a simplified model, presented in the package SURE, a System for Uranium Resource Evaluation. Wemore » have developed and tested these models for the four quadrangles in southern Colorado that comprise the study area; to investigate their generality, the models should be applied to other quandrangles. Once they are calibrated with accepted uranium endowments for several well-known quadrangles, the models can be used to give independent estimates for less-known quadrangles. The point-oriented models structure the objective comparison of the quandrangles on the bases of: (1) Anomalies (a) derived from stream sediments, (b) derived from waters (stream, well, pond, etc.), (2) Geology (a) source rocks, as defined by the evaluator, (b) host rocks, as defined by the evaluator, and (3) Aerial radiometric anomalies.« less

Towards systematic evaluation of crop model outputs for global land-use models

NASA Astrophysics Data System (ADS)

Leclere, David; Azevedo, Ligia B.; Skalský, Rastislav; Balkovič, Juraj; Havlík, Petr

2016-04-01

Land provides vital socioeconomic resources to the society, however at the cost of large environmental degradations. Global integrated models combining high resolution global gridded crop models (GGCMs) and global economic models (GEMs) are increasingly being used to inform sustainable solution for agricultural land-use. However, little effort has yet been done to evaluate and compare the accuracy of GGCM outputs. In addition, GGCM datasets require a large amount of parameters whose values and their variability across space are weakly constrained: increasing the accuracy of such dataset has a very high computing cost. Innovative evaluation methods are required both to ground credibility to the global integrated models, and to allow efficient parameter specification of GGCMs. We propose an evaluation strategy for GGCM datasets in the perspective of use in GEMs, illustrated with preliminary results from a novel dataset (the Hypercube) generated by the EPIC GGCM and used in the GLOBIOM land use GEM to inform on present-day crop yield, water and nutrient input needs for 16 crops x 15 management intensities, at a spatial resolution of 5 arc-minutes. We adopt the following principle: evaluation should provide a transparent diagnosis of model adequacy for its intended use. We briefly describe how the Hypercube data is generated and how it articulates with GLOBIOM in order to transparently identify the performances to be evaluated, as well as the main assumptions and data processing involved. Expected performances include adequately representing the sub-national heterogeneity in crop yield and input needs: i) in space, ii) across crop species, and iii) across management intensities. We will present and discuss measures of these expected performances and weight the relative contribution of crop model, input data and data processing steps in performances. We will also compare obtained yield gaps and main yield-limiting factors against the M3 dataset. Next steps include iterative improvement of parameter assumptions and evaluation of implications of GGCM performances for intended use in the IIASA EPIC-GLOBIOM model cluster. Our approach helps targeting future efforts at improving GGCM accuracy and would achieve highest efficiency if combined with traditional field-scale evaluation and sensitivity analysis.

Influence of boundary conditions to multi-model simulations of ozone and PM2.5 levels over Europe and North America in frame of AQMEII3

NASA Astrophysics Data System (ADS)

Im, Ulas; Hansen, Kaj M.; Geels, Camilla; Christensen, Jesper H.; Brandt, Jørgen; Hogrefe, Christian; Galmarini, Stefano

2016-04-01

AQMEII (Air Quality Model Evaluation International Initiative) promotes research on regional air quality model evaluation across the European and North American atmospheric modelling communities, providing the ideal platform for advancing the evaluation of air quality models at the regional scale. In frame of the AQMEII3 model evaluation exercise, thirteen regional chemistry and transport models have simulated the air pollutant levels over Europe and/or North America for the year 2010, along with various sensitivity simulations of reductions in anthropogenic emissions and boundary conditions. All participating groups have performed sensitivity simulation with 20% reductions in global (GLO) anthropogenic emissions. In addition, various groups simulated sensitivity scenarios of 20% reductions in anthropogenic emissions in different HTAP-defined regions such as North America (NAM), Europe (EUR) and East Asia (EAS). The boundary conditions for the base case and the perturbation scenarios were derived from the MOZART-IFS global chemical model. The present study will evaluate the impact of these emission perturbations on regional surface ozone and PM2.5 levels as well as over individual surface measurement stations over both continents and vertical profiles over the radiosonde stations from the World Ozone and Ultraviolet Radiation Data Centre (WOUDC) and the Aerosol Robotic Network (AERONET) stations for ozone and for PM2.5, respectively.

Description and initial evaluation of an educational and psychosocial support model for adults with congenitally malformed hearts.

PubMed

Rönning, Helén; Nielsen, Niels Erik; Swahn, Eva; Strömberg, Anna

2011-05-01

Various programmes for adults with congenitally malformed hearts have been developed, but detailed descriptions of content, rationale and goals are often missing. The aim of this study was to describe and make an initial evaluation of a follow-up model for adults with congenitally malformed hearts, focusing on education and psychosocial support by a multidisciplinary team (EPS). The model is described in steps and evaluated with regards to perceptions of knowledge, anxiety and satisfaction. The EPS model included a policlinic visit to the physician/nurse (medical consultation, computer-based and individual education face-to-face as well as psychosocial support) and a 1-month telephone follow-up. Fifty-five adults (mean age 34, 29 women) with the nine most common forms of congenitally malformed hearts participated in the EPS model as well as the 3-months follow-up. Knowledge about congenital heart malformation had increased in 40% of the participants at the 3-months follow-up. This study describes and evaluates a model that combines a multidisciplinary approach and computer-based education for follow-up of adults with congenitally malformed hearts. The EPS model was found to increase self-estimated knowledge, but further evaluations need to be conducted to prove patient-centred outcomes over time. The model is now ready to be implemented in adults with congenitally malformed hearts. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Disability Policy Evaluation: Combining Logic Models and Systems Thinking.

PubMed

Claes, Claudia; Ferket, Neelke; Vandevelde, Stijn; Verlet, Dries; De Maeyer, Jessica

2017-07-01

Policy evaluation focuses on the assessment of policy-related personal, family, and societal changes or benefits that follow as a result of the interventions, services, and supports provided to those persons to whom the policy is directed. This article describes a systematic approach to policy evaluation based on an evaluation framework and an evaluation process that combine the use of logic models and systems thinking. The article also includes an example of how the framework and process have recently been used in policy development and evaluation in Flanders (Belgium), as well as four policy evaluation guidelines based on relevant published literature.

Adaptation of Mesoscale Weather Models to Local Forecasting

NASA Technical Reports Server (NTRS)

Manobianco, John T.; Taylor, Gregory E.; Case, Jonathan L.; Dianic, Allan V.; Wheeler, Mark W.; Zack, John W.; Nutter, Paul A.

2003-01-01

Methodologies have been developed for (1) configuring mesoscale numerical weather-prediction models for execution on high-performance computer workstations to make short-range weather forecasts for the vicinity of the Kennedy Space Center (KSC) and the Cape Canaveral Air Force Station (CCAFS) and (2) evaluating the performances of the models as configured. These methodologies have been implemented as part of a continuing effort to improve weather forecasting in support of operations of the U.S. space program. The models, methodologies, and results of the evaluations also have potential value for commercial users who could benefit from tailoring their operations and/or marketing strategies based on accurate predictions of local weather. More specifically, the purpose of developing the methodologies for configuring the models to run on computers at KSC and CCAFS is to provide accurate forecasts of winds, temperature, and such specific thunderstorm-related phenomena as lightning and precipitation. The purpose of developing the evaluation methodologies is to maximize the utility of the models by providing users with assessments of the capabilities and limitations of the models. The models used in this effort thus far include the Mesoscale Atmospheric Simulation System (MASS), the Regional Atmospheric Modeling System (RAMS), and the National Centers for Environmental Prediction Eta Model ( Eta for short). The configuration of the MASS and RAMS is designed to run the models at very high spatial resolution and incorporate local data to resolve fine-scale weather features. Model preprocessors were modified to incorporate surface, ship, buoy, and rawinsonde data as well as data from local wind towers, wind profilers, and conventional or Doppler radars. The overall evaluation of the MASS, Eta, and RAMS was designed to assess the utility of these mesoscale models for satisfying the weather-forecasting needs of the U.S. space program. The evaluation methodology includes objective and subjective verification methodologies. Objective (e.g., statistical) verification of point forecasts is a stringent measure of model performance, but when used alone, it is not usually sufficient for quantifying the value of the overall contribution of the model to the weather-forecasting process. This is especially true for mesoscale models with enhanced spatial and temporal resolution that may be capable of predicting meteorologically consistent, though not necessarily accurate, fine-scale weather phenomena. Therefore, subjective (phenomenological) evaluation, focusing on selected case studies and specific weather features, such as sea breezes and precipitation, has been performed to help quantify the added value that cannot be inferred solely from objective evaluation.

A mixed integer bi-level DEA model for bank branch performance evaluation by Stackelberg approach

NASA Astrophysics Data System (ADS)

Shafiee, Morteza; Lotfi, Farhad Hosseinzadeh; Saleh, Hilda; Ghaderi, Mehdi

2016-03-01

One of the most complicated decision making problems for managers is the evaluation of bank performance, which involves various criteria. There are many studies about bank efficiency evaluation by network DEA in the literature review. These studies do not focus on multi-level network. Wu (Eur J Oper Res 207:856-864, 2010) proposed a bi-level structure for cost efficiency at the first time. In this model, multi-level programming and cost efficiency were used. He used a nonlinear programming to solve the model. In this paper, we have focused on multi-level structure and proposed a bi-level DEA model. We then used a liner programming to solve our model. In other hand, we significantly improved the way to achieve the optimum solution in comparison with the work by Wu (2010) by converting the NP-hard nonlinear programing into a mixed integer linear programming. This study uses a bi-level programming data envelopment analysis model that embodies internal structure with Stackelberg-game relationships to evaluate the performance of banking chain. The perspective of decentralized decisions is taken in this paper to cope with complex interactions in banking chain. The results derived from bi-level programming DEA can provide valuable insights and detailed information for managers to help them evaluate the performance of the banking chain as a whole using Stackelberg-game relationships. Finally, this model was applied in the Iranian bank to evaluate cost efficiency.

Evaluation of a 2 to 1 peer placement supervision model by physiotherapy students and their educators.

PubMed

Alpine, Lucy M; Caldas, Francieli Tanji; Barrett, Emer M

2018-04-02

The objective of the study was to investigate student and practice educator evaluations of practice placements using a structured 2 to 1 supervision and implementation model. Cross-sectional pilot study set in clinical sites providing placements for physiotherapy students in Ireland. Students and practice educators completing a 2.1 peer placement between 2013 and 2015 participated. A self-reported questionnaire which measured indicators linked to quality assured placements was used. Three open-ended questions captured comments on the benefits and challenges associated with the 2 to 1 model. Ten students (10/20; 50% response rate) and 10 practice educators (10/10; 100% response rate) responded to the questionnaire. Student responses included four pairs of students and one student from a further two pairs. There was generally positive agreement with the questionnaire indicating that placements using the 2 to 1 model were positively evaluated by participants. There were no significant differences between students and practice educators. The main benefits of the 2 to 1 model were shared learning experiences, a peer supported environment, and the development of peer evaluation and feedback skills by students. A key component of the model was the peer scripting process which provided time for reflection, self-evaluation, and peer review. 2 to 1 placements were positively evaluated by students and educators when supported by a structured supervision model. Clear guidance to students on the provision of peer feedback and support for educators providing feedback to two different students is recommended.

The role of affect and cognition in health decision making.

PubMed

Keer, Mario; van den Putte, Bas; Neijens, Peter

2010-03-01

Both affective and cognitive evaluations of behaviours have been allocated various positions in theoretical models of decision making. Most often, they have been studied as direct determinants of either intention or overall evaluation, but these two possible positions have never been compared. The aim of this study was to determine whether affective and cognitive evaluations influence intention directly, or whether their influence is mediated by overall evaluation. A sample of 300 university students filled in questionnaires on their affective, cognitive, and overall evaluations in respect of 20 health behaviours. The data were interpreted using mediation analyses with the application of path modelling. Both affective and cognitive evaluations were found to have significantly predicted intention. The influence of affective evaluation was largely direct for each of the behaviours studied, whereas that of cognitive evaluation was partially direct and partially mediated by overall evaluation. These results indicate that decisions regarding the content of persuasive communication (affective vs. cognitive) are highly dependent on the theoretical model chosen. It is suggested that affective evaluation should be included as a direct determinant of intention in theories of decision making when predicting health behaviours.

Defibrillator/monitor/pacemakers.

PubMed

2002-02-01

Defibrillator/monitors allow operators to assess and monitor a patient's ECG and, when necessary, deliver a defibrillating shock to the heart. When integral noninvasive pacing capability is added, the resulting device is referred to as a defibrillator/monitor/pacemaker. In this Update Evaluation, we present our findings for one newly evaluated model, the Philips Heartstream XL, and we summarize our findings for the seven previously evaluated models that are still on the market. (Our previous Evaluations were published in the May-June 1993, February 1998, and September 2000 issues of Health Devices.) Defibrillator/monitor/pacemakers are used for a variety of applications within the hospital, as well as by emergency medical services (EMS) personnel and others in the prehospital environment. To help both hospital-based and prehospital users select an appropriate model, we rate the models (1) for each of three in-hospital applications--general crash-cart use, in-hospital transport use, and in-hospital use by basic as well as advanced users--and (2) for prehospital (EMS) use. For in-hospital use, we recommend four of the evaluated models. These received either Preferred or Acceptable ratings for all the applications considered. For prehospital use, we found that five of the models will meet most organizations' needs.

A sound quality model for objective synthesis evaluation of vehicle interior noise based on artificial neural network

NASA Astrophysics Data System (ADS)

Wang, Y. S.; Shen, G. Q.; Xing, Y. F.

2014-03-01

Based on the artificial neural network (ANN) technique, an objective sound quality evaluation (SQE) model for synthesis annoyance of vehicle interior noises is presented in this paper. According to the standard named GB/T18697, firstly, the interior noises under different working conditions of a sample vehicle are measured and saved in a noise database. Some mathematical models for loudness, sharpness and roughness of the measured vehicle noises are established and performed by Matlab programming. Sound qualities of the vehicle interior noises are also estimated by jury tests following the anchored semantic differential (ASD) procedure. Using the objective and subjective evaluation results, furthermore, an ANN-based model for synthetical annoyance evaluation of vehicle noises, so-called ANN-SAE, is developed. Finally, the ANN-SAE model is proved by some verification tests with the leave-one-out algorithm. The results suggest that the proposed ANN-SAE model is accurate and effective and can be directly used to estimate sound quality of the vehicle interior noises, which is very helpful for vehicle acoustical designs and improvements. The ANN-SAE approach may be extended to deal with other sound-related fields for product quality evaluations in SQE engineering.

Toward a Trust Evaluation Mechanism in the Social Internet of Things.

PubMed

Truong, Nguyen Binh; Lee, Hyunwoo; Askwith, Bob; Lee, Gyu Myoung

2017-06-09

In the blooming era of the Internet of Things (IoT), trust has been accepted as a vital factor for provisioning secure, reliable, seamless communications and services. However, a large number of challenges still remain unsolved due to the ambiguity of the concept of trust as well as the variety of divergent trust models in different contexts. In this research, we augment the trust concept, the trust definition and provide a general conceptual model in the context of the Social IoT (SIoT) environment by breaking down all attributes influencing trust. Then, we propose a trust evaluation model called REK, comprised of the triad of trust indicators (TIs) Reputation, Experience and Knowledge. The REK model covers multi-dimensional aspects of trust by incorporating heterogeneous information from direct observation (as Knowledge TI), personal experiences (as Experience TI) to global opinions (as Reputation TI). The associated evaluation models for the three TIs are also proposed and provisioned. We then come up with an aggregation mechanism for deriving trust values as the final outcome of the REK evaluation model. We believe this article offers better understandings on trust as well as provides several prospective approaches for the trust evaluation in the SIoT environment.

Toward a Trust Evaluation Mechanism in the Social Internet of Things

PubMed Central

Truong, Nguyen Binh; Lee, Hyunwoo; Askwith, Bob; Lee, Gyu Myoung

2017-01-01

In the blooming era of the Internet of Things (IoT), trust has been accepted as a vital factor for provisioning secure, reliable, seamless communications and services. However, a large number of challenges still remain unsolved due to the ambiguity of the concept of trust as well as the variety of divergent trust models in different contexts. In this research, we augment the trust concept, the trust definition and provide a general conceptual model in the context of the Social IoT (SIoT) environment by breaking down all attributes influencing trust. Then, we propose a trust evaluation model called REK, comprised of the triad of trust indicators (TIs) Reputation, Experience and Knowledge. The REK model covers multi-dimensional aspects of trust by incorporating heterogeneous information from direct observation (as Knowledge TI), personal experiences (as Experience TI) to global opinions (as Reputation TI). The associated evaluation models for the three TIs are also proposed and provisioned. We then come up with an aggregation mechanism for deriving trust values as the final outcome of the REK evaluation model. We believe this article offers better understandings on trust as well as provides several prospective approaches for the trust evaluation in the SIoT environment. PMID:28598401

Quality of protection evaluation of security mechanisms.

PubMed

Ksiezopolski, Bogdan; Zurek, Tomasz; Mokkas, Michail

2014-01-01

Recent research indicates that during the design of teleinformatic system the tradeoff between the systems performance and the system protection should be made. The traditional approach assumes that the best way is to apply the strongest possible security measures. Unfortunately, the overestimation of security measures can lead to the unreasonable increase of system load. This is especially important in multimedia systems where the performance has critical character. In many cases determination of the required level of protection and adjustment of some security measures to these requirements increase system efficiency. Such an approach is achieved by means of the quality of protection models where the security measures are evaluated according to their influence on the system security. In the paper, we propose a model for QoP evaluation of security mechanisms. Owing to this model, one can quantify the influence of particular security mechanisms on ensuring security attributes. The methodology of our model preparation is described and based on it the case study analysis is presented. We support our method by the tool where the models can be defined and QoP evaluation can be performed. Finally, we have modelled TLS cryptographic protocol and presented the QoP security mechanisms evaluation for the selected versions of this protocol.

Is the Closet Door Still Closed in 2014? A CIPP Model Program Evaluation of Preservice Diversity Training Regarding LGBT Issues

ERIC Educational Resources Information Center

Woodruff, Joseph

2014-01-01

The purpose of this program evaluation was to examine the four components of the CIPP evaluation model (Context, Input, Process, and Product evaluations) in the diversity training program conceptualization and design delivered to College of Education K-12 preservice teachers at a large university in the southeastern United States (referred to in…

Evaluation of the Combined AERCOARE/AERMOD Modeling Approach for Offshore Sources

EPA Science Inventory

ENVIRON conducted an evaluation of the combined AERCOARE/AERMOD (AERCOARE-MOD) modeling approach for offshore sources using tracer data from four field studies. AERCOARE processes overwater meteorological data for use by the AERMOD air quality dispersion model (EPA, 2004a). AERC...

Ozone deposition modelling within the Air Quality Model Evaluation International Initiative (AQMEII)

EPA Science Inventory

This presentation provides an overview of the Air Quality Model Evaluation International Initiative (AQMEII). It contains a synopsis of the three phases of AQMEII, including objectives, logistics, and timelines. It also provides a number of examples of analyses conducted through ...

NEW CATEGORICAL METRICS FOR AIR QUALITY MODEL EVALUATION

EPA Science Inventory

Traditional categorical metrics used in model evaluations are "clear-cut" measures in that the model's ability to predict an exceedance is defined by a fixed threshold concentration and the metrics are defined by observation-forecast sets that are paired both in space and time. T...

Development and Evaluation of Land-Use Regression Models Using Modeled Air Quality Concentrations

EPA Science Inventory

Abstract Land-use regression (LUR) models have emerged as a preferred methodology for estimating individual exposure to ambient air pollution in epidemiologic studies in absence of subject-specific measurements. Although there is a growing literature focused on LUR evaluation, fu...

SENSITIVE PARAMETER EVALUATION FOR A VADOSE ZONE FATE AND TRANSPORT MODEL

EPA Science Inventory

This report presents information pertaining to quantitative evaluation of the potential impact of selected parameters on output of vadose zone transport and fate models used to describe the behavior of hazardous chemicals in soil. The Vadose 2one Interactive Processes (VIP) model...

On the quasi-steady aerodynamics of normal hovering flight part II: model implementation and evaluation

PubMed Central

Nabawy, Mostafa R. A.; Crowther, William J.

2014-01-01

This paper introduces a generic, transparent and compact model for the evaluation of the aerodynamic performance of insect-like flapping wings in hovering flight. The model is generic in that it can be applied to wings of arbitrary morphology and kinematics without the use of experimental data, is transparent in that the aerodynamic components of the model are linked directly to morphology and kinematics via physical relationships and is compact in the sense that it can be efficiently evaluated for use within a design optimization environment. An important aspect of the model is the method by which translational force coefficients for the aerodynamic model are obtained from first principles; however important insights are also provided for the morphological and kinematic treatments that improve the clarity and efficiency of the overall model. A thorough analysis of the leading-edge suction analogy model is provided and comparison of the aerodynamic model with results from application of the leading-edge suction analogy shows good agreement. The full model is evaluated against experimental data for revolving wings and good agreement is obtained for lift and drag up to 90° incidence. Comparison of the model output with data from computational fluid dynamics studies on a range of different insect species also shows good agreement with predicted weight support ratio and specific power. The validated model is used to evaluate the relative impact of different contributors to the induced power factor for the hoverfly and fruitfly. It is shown that the assumption of an ideal induced power factor (k = 1) for a normal hovering hoverfly leads to a 23% overestimation of the generated force owing to flapping. PMID:24554578

Model-Based Economic Evaluation of Treatments for Depression: A Systematic Literature Review.

PubMed

Kolovos, Spyros; Bosmans, Judith E; Riper, Heleen; Chevreul, Karine; Coupé, Veerle M H; van Tulder, Maurits W

2017-09-01

An increasing number of model-based studies that evaluate the cost effectiveness of treatments for depression are being published. These studies have different characteristics and use different simulation methods. We aimed to systematically review model-based studies evaluating the cost effectiveness of treatments for depression and examine which modelling technique is most appropriate for simulating the natural course of depression. The literature search was conducted in the databases PubMed, EMBASE and PsycInfo between 1 January 2002 and 1 October 2016. Studies were eligible if they used a health economic model with quality-adjusted life-years or disability-adjusted life-years as an outcome measure. Data related to various methodological characteristics were extracted from the included studies. The available modelling techniques were evaluated based on 11 predefined criteria. This methodological review included 41 model-based studies, of which 21 used decision trees (DTs), 15 used cohort-based state-transition Markov models (CMMs), two used individual-based state-transition models (ISMs), and three used discrete-event simulation (DES) models. Just over half of the studies (54%) evaluated antidepressants compared with a control condition. The data sources, time horizons, cycle lengths, perspectives adopted and number of health states/events all varied widely between the included studies. DTs scored positively in four of the 11 criteria, CMMs in five, ISMs in six, and DES models in seven. There were substantial methodological differences between the studies. Since the individual history of each patient is important for the prognosis of depression, DES and ISM simulation methods may be more appropriate than the others for a pragmatic representation of the course of depression. However, direct comparisons between the available modelling techniques are necessary to yield firm conclusions.

On the quasi-steady aerodynamics of normal hovering flight part II: model implementation and evaluation.

PubMed

Nabawy, Mostafa R A; Crowther, William J

2014-05-06

This paper introduces a generic, transparent and compact model for the evaluation of the aerodynamic performance of insect-like flapping wings in hovering flight. The model is generic in that it can be applied to wings of arbitrary morphology and kinematics without the use of experimental data, is transparent in that the aerodynamic components of the model are linked directly to morphology and kinematics via physical relationships and is compact in the sense that it can be efficiently evaluated for use within a design optimization environment. An important aspect of the model is the method by which translational force coefficients for the aerodynamic model are obtained from first principles; however important insights are also provided for the morphological and kinematic treatments that improve the clarity and efficiency of the overall model. A thorough analysis of the leading-edge suction analogy model is provided and comparison of the aerodynamic model with results from application of the leading-edge suction analogy shows good agreement. The full model is evaluated against experimental data for revolving wings and good agreement is obtained for lift and drag up to 90° incidence. Comparison of the model output with data from computational fluid dynamics studies on a range of different insect species also shows good agreement with predicted weight support ratio and specific power. The validated model is used to evaluate the relative impact of different contributors to the induced power factor for the hoverfly and fruitfly. It is shown that the assumption of an ideal induced power factor (k = 1) for a normal hovering hoverfly leads to a 23% overestimation of the generated force owing to flapping.

Evaluating the sources of water to wells: Three techniques for metamodeling of a groundwater flow model

USGS Publications Warehouse

Fienen, Michael N.; Nolan, Bernard T.; Feinstein, Daniel T.

2016-01-01

For decision support, the insights and predictive power of numerical process models can be hampered by insufficient expertise and computational resources required to evaluate system response to new stresses. An alternative is to emulate the process model with a statistical “metamodel.” Built on a dataset of collocated numerical model input and output, a groundwater flow model was emulated using a Bayesian Network, an Artificial neural network, and a Gradient Boosted Regression Tree. The response of interest was surface water depletion expressed as the source of water-to-wells. The results have application for managing allocation of groundwater. Each technique was tuned using cross validation and further evaluated using a held-out dataset. A numerical MODFLOW-USG model of the Lake Michigan Basin, USA, was used for the evaluation. The performance and interpretability of each technique was compared pointing to advantages of each technique. The metamodel can extend to unmodeled areas.

Modeling procedures for handling qualities evaluation of flexible aircraft

NASA Technical Reports Server (NTRS)

Govindaraj, K. S.; Eulrich, B. J.; Chalk, C. R.

1981-01-01

This paper presents simplified modeling procedures to evaluate the impact of flexible modes and the unsteady aerodynamic effects on the handling qualities of Supersonic Cruise Aircraft (SCR). The modeling procedures involve obtaining reduced order transfer function models of SCR vehicles, including the important flexible mode responses and unsteady aerodynamic effects, and conversion of the transfer function models to time domain equations for use in simulations. The use of the modeling procedures is illustrated by a simple example.

Switching performance of OBS network model under prefetched real traffic

NASA Astrophysics Data System (ADS)

Huang, Zhenhua; Xu, Du; Lei, Wen

2005-11-01

Optical Burst Switching (OBS) [1] is now widely considered as an efficient switching technique in building the next generation optical Internet .So it's very important to precisely evaluate the performance of the OBS network model. The performance of the OBS network model is variable in different condition, but the most important thing is that how it works under real traffic load. In the traditional simulation models, uniform traffics are usually generated by simulation software to imitate the data source of the edge node in the OBS network model, and through which the performance of the OBS network is evaluated. Unfortunately, without being simulated by real traffic, the traditional simulation models have several problems and their results are doubtable. To deal with this problem, we present a new simulation model for analysis and performance evaluation of the OBS network, which uses prefetched IP traffic to be data source of the OBS network model. The prefetched IP traffic can be considered as real IP source of the OBS edge node and the OBS network model has the same clock rate with a real OBS system. So it's easy to conclude that this model is closer to the real OBS system than the traditional ones. The simulation results also indicate that this model is more accurate to evaluate the performance of the OBS network system and the results of this model are closer to the actual situation.

Critical evaluation of mechanistic two-phase flow pipeline and well simulation models

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dhulesia, H.; Lopez, D.

1996-12-31

Mechanistic steady state simulation models, rather than empirical correlations, are used for a design of multiphase production system including well, pipeline and downstream installations. Among the available models, PEPITE, WELLSIM, OLGA, TACITE and TUFFP are widely used for this purpose and consequently, a critical evaluation of these models is needed. An extensive validation methodology is proposed which consists of two distinct steps: first to validate the hydrodynamic point model using the test loop data and, then to validate the over-all simulation model using the real pipelines and wells data. The test loop databank used in this analysis contains about 5952more » data sets originated from four different test loops and a majority of these data are obtained at high pressures (up to 90 bars) with real hydrocarbon fluids. Before performing the model evaluation, physical analysis of the test loops data is required to eliminate non-coherent data. The evaluation of these point models demonstrates that the TACITE and OLGA models can be applied to any configuration of pipes. The TACITE model performs better than the OLGA model because it uses the most appropriate closure laws from the literature validated on a large number of data. The comparison of predicted and measured pressure drop for various real pipelines and wells demonstrates that the TACITE model is a reliable tool.« less

A Multi-Model Assessment for the 2006 and 2010 Simulations under the AirQuality Model Evaluation International Initiative (AQMEII) Phase 2 over North America: Part I. Indicators of the Sensitivity of O3 and PM2.5 Formation Regimes

EPA Science Inventory

Under the Air Quality Model Evaluation International Initiative, Phase 2 (AQMEII-2), three online coupled air quality model simulations, with six different configurations, are analyzed for their performance, inter-model agreement, and responses to emission and meteorological chan...

Model Performance Evaluation and Scenario Analysis ...

EPA Pesticide Factsheets

This tool consists of two parts: model performance evaluation and scenario analysis (MPESA). The model performance evaluation consists of two components: model performance evaluation metrics and model diagnostics. These metrics provides modelers with statistical goodness-of-fit measures that capture magnitude only, sequence only, and combined magnitude and sequence errors. The performance measures include error analysis, coefficient of determination, Nash-Sutcliffe efficiency, and a new weighted rank method. These performance metrics only provide useful information about the overall model performance. Note that MPESA is based on the separation of observed and simulated time series into magnitude and sequence components. The separation of time series into magnitude and sequence components and the reconstruction back to time series provides diagnostic insights to modelers. For example, traditional approaches lack the capability to identify if the source of uncertainty in the simulated data is due to the quality of the input data or the way the analyst adjusted the model parameters. This report presents a suite of model diagnostics that identify if mismatches between observed and simulated data result from magnitude or sequence related errors. MPESA offers graphical and statistical options that allow HSPF users to compare observed and simulated time series and identify the parameter values to adjust or the input data to modify. The scenario analysis part of the too

ESEA Title I Evaluation and Reporting System: User's Guide.

ERIC Educational Resources Information Center

Tallmadge, G. Kasten; Wood, Christine T.

This guidebook concentrates primarily on describing the impact-assessment component of the Elementary and Secondary Education Act (ESEA) Title I evaluation and reporting system for users of the system. Three general evaluation models are presented, along with implementation information for each. The first model, a norm-referenced design, may be…

Evaluation of DeNitrification DeComposition model for estimating ammonia fluxes from chemical fertilizer application

USDA-ARS?s Scientific Manuscript database

DeNitrification DeComposition (DNDC) model predictions of NH3 fluxes following chemical fertilizer application were evaluated by comparison to relaxed eddy accumulation (REA) measurements, in Central Illinois, United States, over the 2014 growing season of corn. Practical issues for evaluating closu...

A Participatory Action Research Approach To Evaluating Inclusive School Programs.

ERIC Educational Resources Information Center

Dymond, Stacy K.

2001-01-01

This article proposes a model for evaluating inclusive schools. Key elements of the model are inclusion of stakeholders in the evaluation process through a participatory action research approach, analysis of program processes and outcomes, use of multiple methods and measures, and obtaining perceptions from diverse stakeholder groups. (Contains…

Classroom Factors Affecting Students: Self-Evaluation: An Interactional Model.

ERIC Educational Resources Information Center

Marshall, Hermine H.; Weinstein, Rhona S.

1984-01-01

A complex interactional model of classroom factors that contribute to the development of students' self-evaluations is presented. Factors described are: (1) task structure; (2) grouping practices; (3) feedback and evaluation procedures and information about ability; (4) motivational strategies; (5) locus of responsibility for learning; and (6) the…

An Evaluation Research Model for System-Wide Textbook Selection.

ERIC Educational Resources Information Center

Talmage, Harriet; Walberg, Herbert T.

One component of an evaluation research model for system-wide selection of curriculum materials is reported: implementation of an evaluation design for obtaining data that permits professional and lay persons to base curriculum materials decisions on a "best fit" principle. The design includes teacher characteristics, learning environment…

A BAYESIAN STATISTICAL APPROACHES FOR THE EVALUATION OF CMAQ

EPA Science Inventory

This research focuses on the application of spatial statistical techniques for the evaluation of the Community Multiscale Air Quality (CMAQ) model. The upcoming release version of the CMAQ model was run for the calendar year 2001 and is in the process of being evaluated by EPA an...

Multi-Dimensional Planning/Evaluation Schema for Community Education.

ERIC Educational Resources Information Center

Merkel-Keller, Claudia; Herr, Audrey

A model for planning and evaluating community education programs--Stufflebeam's context, input, process, product (CIPP) evaluation model--was described and field-tested with the community education programs in Lakewood, New Jersey. Community education was defined as a concern for everything that affects the well-being of all citizens within a…

An Analytical Hierarchy Process Model for the Evaluation of College Experimental Teaching Quality

ERIC Educational Resources Information Center

Yin, Qingli

2013-01-01

Taking into account the characteristics of college experimental teaching, through investigaton and analysis, evaluation indices and an Analytical Hierarchy Process (AHP) model of experimental teaching quality have been established following the analytical hierarchy process method, and the evaluation indices have been given reasonable weights. An…

An Information Search Model of Evaluative Concerns in Intergroup Interaction

ERIC Educational Resources Information Center

Vorauer, Jacquie D.

2006-01-01

In an information search model, evaluative concerns during intergroup interaction are conceptualized as a joint function of uncertainty regarding and importance attached to out-group members' views of oneself. High uncertainty generally fosters evaluative concerns during intergroup exchanges. Importance depends on whether out-group members'…

Space-Time Analysis of the Air Quality Model Evaluation International Initiative (AQMEII) Phase 1 Air Quality Simulations

EPA Science Inventory

This study presents an evaluation of summertime daily maximum ozone concentrations over North America (NA) and Europe (EU) using the database generated during Phase 1 of the Air Quality Model Evaluation International Initiative (AQMEII). The analysis focuses on identifying tempor...

Metrics for Evaluation of Student Models

ERIC Educational Resources Information Center

Pelanek, Radek

2015-01-01

Researchers use many different metrics for evaluation of performance of student models. The aim of this paper is to provide an overview of commonly used metrics, to discuss properties, advantages, and disadvantages of different metrics, to summarize current practice in educational data mining, and to provide guidance for evaluation of student…

Many-level multilevel structural equation modeling: An efficient evaluation strategy.

PubMed

Pritikin, Joshua N; Hunter, Michael D; von Oertzen, Timo; Brick, Timothy R; Boker, Steven M

2017-01-01

Structural equation models are increasingly used for clustered or multilevel data in cases where mixed regression is too inflexible. However, when there are many levels of nesting, these models can become difficult to estimate. We introduce a novel evaluation strategy, Rampart, that applies an orthogonal rotation to the parts of a model that conform to commonly met requirements. This rotation dramatically simplifies fit evaluation in a way that becomes more potent as the size of the data set increases. We validate and evaluate the implementation using a 3-level latent regression simulation study. Then we analyze data from a state-wide child behavioral health measure administered by the Oklahoma Department of Human Services. We demonstrate the efficiency of Rampart compared to other similar software using a latent factor model with a 5-level decomposition of latent variance. Rampart is implemented in OpenMx, a free and open source software.

Applying the natural disasters vulnerability evaluation model to the March 2011 north-east Japan earthquake and tsunami.

PubMed

Ruiz Estrada, Mario Arturo; Yap, Su Fei; Park, Donghyun

2014-07-01

Natural hazards have a potentially large impact on economic growth, but measuring their economic impact is subject to a great deal of uncertainty. The central objective of this paper is to demonstrate a model--the natural disasters vulnerability evaluation (NDVE) model--that can be used to evaluate the impact of natural hazards on gross national product growth. The model is based on five basic indicators-natural hazards growth rates (αi), the national natural hazards vulnerability rate (ΩT), the natural disaster devastation magnitude rate (Π), the economic desgrowth rate (i.e. shrinkage of the economy) (δ), and the NHV surface. In addition, we apply the NDVE model to the north-east Japan earthquake and tsunami of March 2011 to evaluate its impact on the Japanese economy. © 2014 The Author(s). Disasters © Overseas Development Institute, 2014.

Evaluating the compatibility of multi-functional and intensive urban land uses

NASA Astrophysics Data System (ADS)

Taleai, M.; Sharifi, A.; Sliuzas, R.; Mesgari, M.

2007-12-01

This research is aimed at developing a model for assessing land use compatibility in densely built-up urban areas. In this process, a new model was developed through the combination of a suite of existing methods and tools: geographical information system, Delphi methods and spatial decision support tools: namely multi-criteria evaluation analysis, analytical hierarchy process and ordered weighted average method. The developed model has the potential to calculate land use compatibility in both horizontal and vertical directions. Furthermore, the compatibility between the use of each floor in a building and its neighboring land uses can be evaluated. The method was tested in a built-up urban area located in Tehran, the capital city of Iran. The results show that the model is robust in clarifying different levels of physical compatibility between neighboring land uses. This paper describes the various steps and processes of developing the proposed land use compatibility evaluation model (CEM).

Evaluating the Ocean Component of the US Navy Earth System Model

NASA Astrophysics Data System (ADS)

Zamudio, L.

2017-12-01

Ocean currents, temperature, and salinity observations are used to evaluate the ocean component of the US Navy Earth System Model. The ocean and atmosphere components of the system are an eddy-resolving (1/12.5° equatorial resolution) version of the HYbrid Coordinate Ocean Model (HYCOM), and a T359L50 version of the NAVy Global Environmental Model (NAVGEM), respectively. The system was integrated in hindcast mode and the ocean results are compared against unassimilated observations, a stand-alone version of HYCOM, and the Generalized Digital Environment Model ocean climatology. The different observation types used in the system evaluation are: drifting buoys, temperature profiles, salinity profiles, and acoustical proxies (mixed layer depth, sonic layer depth, below layer gradient, and acoustical trapping). To evaluate the system's performance in each different metric, a scorecard is used to translate the system's errors into scores, which provide an indication of the system's skill in both space and time.

Inspiration or deflation? Feeling similar or dissimilar to slim and plus-size models affects self-evaluation of restrained eaters.

PubMed

Papies, Esther K; Nicolaije, Kim A H

2012-01-01

The present studies examined the effect of perceiving images of slim and plus-size models on restrained eaters' self-evaluation. While previous research has found that such images can lead to either inspiration or deflation, we argue that these inconsistencies can be explained by differences in perceived similarity with the presented model. The results of two studies (ns=52 and 99) confirmed this and revealed that restrained eaters with high (low) perceived similarity to the model showed more positive (negative) self-evaluations when they viewed a slim model, compared to a plus-size model. In addition, Study 2 showed that inducing in participants a similarities mindset led to more positive self-evaluations after viewing a slim compared to a plus-size model, but only among restrained eaters with a relatively high BMI. These results are discussed in the context of research on social comparison processes and with regard to interventions for protection against the possible detrimental effects of media images. Copyright © 2011 Elsevier Ltd. All rights reserved.

Statistical and Hydrological evaluation of precipitation forecasts from IMD MME and ECMWF numerical weather forecasts for Indian River basins

NASA Astrophysics Data System (ADS)

Mohite, A. R.; Beria, H.; Behera, A. K.; Chatterjee, C.; Singh, R.

2016-12-01

Flood forecasting using hydrological models is an important and cost-effective non-structural flood management measure. For forecasting at short lead times, empirical models using real-time precipitation estimates have proven to be reliable. However, their skill depreciates with increasing lead time. Coupling a hydrologic model with real-time rainfall forecasts issued from numerical weather prediction (NWP) systems could increase the lead time substantially. In this study, we compared 1-5 days precipitation forecasts from India Meteorological Department (IMD) Multi-Model Ensemble (MME) with European Center for Medium Weather forecast (ECMWF) NWP forecasts for over 86 major river basins in India. We then evaluated the hydrologic utility of these forecasts over Basantpur catchment (approx. 59,000 km2) of the Mahanadi River basin. Coupled MIKE 11 RR (NAM) and MIKE 11 hydrodynamic (HD) models were used for the development of flood forecast system (FFS). RR model was calibrated using IMD station rainfall data. Cross-sections extracted from SRTM 30 were used as input to the MIKE 11 HD model. IMD started issuing operational MME forecasts from the year 2008, and hence, both the statistical and hydrologic evaluation were carried out from 2008-2014. The performance of FFS was evaluated using both the NWP datasets separately for the year 2011, which was a large flood year in Mahanadi River basin. We will present figures and metrics for statistical (threshold based statistics, skill in terms of correlation and bias) and hydrologic (Nash Sutcliffe efficiency, mean and peak error statistics) evaluation. The statistical evaluation will be at pan-India scale for all the major river basins and the hydrologic evaluation will be for the Basantpur catchment of the Mahanadi River basin.

Advances in snow cover distributed modelling via ensemble simulations and assimilation of satellite data

NASA Astrophysics Data System (ADS)

Revuelto, J.; Dumont, M.; Tuzet, F.; Vionnet, V.; Lafaysse, M.; Lecourt, G.; Vernay, M.; Morin, S.; Cosme, E.; Six, D.; Rabatel, A.

2017-12-01

Nowadays snowpack models show a good capability in simulating the evolution of snow in mountain areas. However singular deviations of meteorological forcing and shortcomings in the modelling of snow physical processes, when accumulated on time along a snow season, could produce large deviations from real snowpack state. The evaluation of these deviations is usually assessed with on-site observations from automatic weather stations. Nevertheless the location of these stations could strongly influence the results of these evaluations since local topography may have a marked influence on snowpack evolution. Despite the evaluation of snowpack models with automatic weather stations usually reveal good results, there exist a lack of large scale evaluations of simulations results on heterogeneous alpine terrain subjected to local topographic effects.This work firstly presents a complete evaluation of the detailed snowpack model Crocus over an extended mountain area, the Arve upper catchment (western European Alps). This catchment has a wide elevation range with a large area above 2000m a.s.l. and/or glaciated. The evaluation compares results obtained with distributed and semi-distributed simulations (the latter nowadays used on the operational forecasting). Daily observations of the snow covered area from MODIS satellite sensor, seasonal glacier surface mass balance evolution measured in more than 65 locations and the galciers annual equilibrium line altitude from Landsat/Spot/Aster satellites, have been used for model evaluation. Additionally the latest advances in producing ensemble snowpack simulations for assimilating satellite reflectance data over extended areas will be presented. These advances comprises the generation of an ensemble of downscaled high-resolution meteorological forcing from meso-scale meteorological models and the application of a particle filter scheme for assimilating satellite observations. Despite the results are prefatory, they show a good potential improving snowpack forecasting capabilities.

Modeling and analysis of equipment managers in manufacturing execution systems for semiconductor packaging.

PubMed

Cheng, F T; Yang, H C; Luo, T L; Feng, C; Jeng, M

2000-01-01

Equipment Managers (EMs) play a major role in a Manufacturing Execution System (MES). They serve as the communication bridge between the components of an MES and the equipment. The purpose of this paper is to propose a novel methodology for developing analytical and simulation models for the EM such that the validity and performance of the EM can be evaluated. Domain knowledge and requirements are collected from a real semiconductor packaging factory. By using IDEFO and state diagrams, a static functional model and a dynamic state model of the EM are built. Next, these two models are translated into a Petri net model. This allows qualitative and quantitative analyses of the system. The EM net model is then expanded into the MES net model. Therefore, the performance of an EM in the MES environment can be evaluated. These evaluation results are good references for design and decision making.

Sensitivity analysis, calibration, and testing of a distributed hydrological model using error‐based weighting and one objective function

USGS Publications Warehouse

Foglia, L.; Hill, Mary C.; Mehl, Steffen W.; Burlando, P.

2009-01-01

We evaluate the utility of three interrelated means of using data to calibrate the fully distributed rainfall‐runoff model TOPKAPI as applied to the Maggia Valley drainage area in Switzerland. The use of error‐based weighting of observation and prior information data, local sensitivity analysis, and single‐objective function nonlinear regression provides quantitative evaluation of sensitivity of the 35 model parameters to the data, identification of data types most important to the calibration, and identification of correlations among parameters that contribute to nonuniqueness. Sensitivity analysis required only 71 model runs, and regression required about 50 model runs. The approach presented appears to be ideal for evaluation of models with long run times or as a preliminary step to more computationally demanding methods. The statistics used include composite scaled sensitivities, parameter correlation coefficients, leverage, Cook's D, and DFBETAS. Tests suggest predictive ability of the calibrated model typical of hydrologic models.

Modeling of dispersion near roadways based on the vehicle-induced turbulence concept

NASA Astrophysics Data System (ADS)

Sahlodin, Ali M.; Sotudeh-Gharebagh, Rahmat; Zhu, Yifang

A mathematical model is developed for dispersion near roadways by incorporating vehicle-induced turbulence (VIT) into Gaussian dispersion modeling using computational fluid dynamics (CFD). The model is based on the Gaussian plume equation in which roadway is regarded as a series of point sources. The Gaussian dispersion parameters are modified by simulation of the roadway using CFD in order to evaluate turbulent kinetic energy (TKE) as a measure of VIT. The model was evaluated against experimental carbon monoxide concentrations downwind of two major freeways reported in the literature. Good agreements were achieved between model results and the literature data. A significant difference was observed between the model results with and without considering VIT. The difference is rather high for data very close to the freeways. This model, after evaluation with additional data, may be used as a framework for predicting dispersion and deposition from any roadway for different traffic (vehicle type and speed) conditions.

Automation of reliability evaluation procedures through CARE - The computer-aided reliability estimation program.

NASA Technical Reports Server (NTRS)

Mathur, F. P.

1972-01-01

Description of an on-line interactive computer program called CARE (Computer-Aided Reliability Estimation) which can model self-repair and fault-tolerant organizations and perform certain other functions. Essentially CARE consists of a repository of mathematical equations defining the various basic redundancy schemes. These equations, under program control, are then interrelated to generate the desired mathematical model to fit the architecture of the system under evaluation. The mathematical model is then supplied with ground instances of its variables and is then evaluated to generate values for the reliability-theoretic functions applied to the model.

Literature Review on Modeling Cyber Networks and Evaluating Cyber Risks.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kelic, Andjelka; Campbell, Philip L

The National Infrastructure Simulations and Analysis Center (NISAC) conducted a literature review on modeling cyber networks and evaluating cyber risks. The literature review explores where modeling is used in the cyber regime and ways that consequence and risk are evaluated. The relevant literature clusters in three different spaces: network security, cyber-physical, and mission assurance. In all approaches, some form of modeling is utilized at varying levels of detail, while the ability to understand consequence varies, as do interpretations of risk. This document summarizes the different literature viewpoints and explores their applicability to securing enterprise networks.

The Acoustic Model Evaluation Committee (AMEC) Reports. Volume 3. Evaluation of the RAYMODE X Propagation Loss Model. Book 2. Appendices A-D

DTIC Science & Technology

1982-09-01

2 of 3 The Acasstic Model Evaluation Committee (AMEC) Reports Vodlme III, Appendices A-i) Elakiaie of Me RAYAODE X f*W#ton Loss Model (U) Prepared by...Richard B. Laer, NORDP Numerical Modefa Division 11N Pylos of RAYNONE X (g) ’T evy •u v ,ap,.,,.,.nDTiC hpWor 1984;& AELE TEfTE Aft 12 W~ LAJ...Activity NSTL Station, Mississippi 39529 84 04 06 511 CONFIDENTIAL .%.. CONFIDENTIAL Appendix IliA. Accuracy Assessment of RAYMODE X Compared to SUDS

Pre-fire and post-fire surface fuel and cover measurements collected in the southeastern United States for model evaluation and development - RxCADRE 2008, 2011 and 2012

Treesearch

Roger D. Ottmar; Andrew T. Hudak; Susan J. Prichard; Clinton S. Wright; Joseph C. Restaino; Maureen C. Kennedy; Robert E. Vihnanek

2016-01-01

A lack of independent, quality-assured data prevents scientists from effectively evaluating predictions and uncertainties in fire models used by land managers. This paper presents a summary of pre-fire and post-fire fuel, fuel moisture and surface cover fraction data that can be used for fire model evaluation and development. The data were collected in the...

Evidence used in model-based economic evaluations for evaluating pharmacogenetic and pharmacogenomic tests: a systematic review protocol

PubMed Central

Peters, Jaime L; Cooper, Chris; Buchanan, James

2015-01-01

Introduction Decision models can be used to conduct economic evaluations of new pharmacogenetic and pharmacogenomic tests to ensure they offer value for money to healthcare systems. These models require a great deal of evidence, yet research suggests the evidence used is diverse and of uncertain quality. By conducting a systematic review, we aim to investigate the test-related evidence used to inform decision models developed for the economic evaluation of genetic tests. Methods and analysis We will search electronic databases including MEDLINE, EMBASE and NHS EEDs to identify model-based economic evaluations of pharmacogenetic and pharmacogenomic tests. The search will not be limited by language or date. Title and abstract screening will be conducted independently by 2 reviewers, with screening of full texts and data extraction conducted by 1 reviewer, and checked by another. Characteristics of the decision problem, the decision model and the test evidence used to inform the model will be extracted. Specifically, we will identify the reported evidence sources for the test-related evidence used, describe the study design and how the evidence was identified. A checklist developed specifically for decision analytic models will be used to critically appraise the models described in these studies. Variations in the test evidence used in the decision models will be explored across the included studies, and we will identify gaps in the evidence in terms of both quantity and quality. Dissemination The findings of this work will be disseminated via a peer-reviewed journal publication and at national and international conferences. PMID:26560056