42 CFR 456.143 - Content of medical care evaluation studies.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 4 2010-10-01 2010-10-01 false Content of medical care evaluation studies. 456.143...: Medical Care Evaluation Studies § 456.143 Content of medical care evaluation studies. Each medical care... patient care; (b) Include analysis of at least the following: (1) Admissions; (2) Durations of stay; (3...

Evaluation methods used on health information systems (HISs) in Iran and the effects of HISs on Iranian healthcare: a systematic review.

PubMed

Ahmadian, Leila; Nejad, Simin Salehi; Khajouei, Reza

2015-06-01

The most important goal of a health information system (HIS) is improvement of quality, effectiveness and efficiency of health services. To achieve this goal, health care systems should be evaluated continuously. The aim of this paper was to study the impacts of HISs in Iran and the methods used for their evaluation. We systematically searched all English and Persian papers evaluating health information systems in Iran that were indexed in SID, Magiran, Iran medex, PubMed and Embase databases until June 2013. A data collection form was designed to extract required data such as types of systems evaluated, evaluation methods and tools. In this study, 53 out of 1103 retrieved articles were selected as relevant and reviewed by the authors. This study indicated that 28 studies used questionnaires to evaluate the system and in 27 studies the study instruments were distributed within a research population. In 26 papers the researchers collected the information by means of interviews, observations, heuristic evaluation and the review of documents and records. The main effects of the evaluated systems in health care settings were improving quality of services, reducing time, increasing accessibility to information, reducing costs and decreasing medical errors. Evaluation of health information systems is central to their development and enhancement, and to understanding their effect on health and health services. Despite numerous evaluation methods available, the reviewed studies used a limited number of methods to evaluate HIS. Additionally, the studies mainly discussed the positive effects of HIS on health care services. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Evaluation research in occupational health services: general principles and a systematic review of empirical studies.

PubMed

Hulshof, C T; Verbeek, J H; van Dijk, F J; van der Weide, W E; Braam, I T

1999-06-01

To study the nature and extent of evaluation research in occupational health services (OHSs). Literature review of evaluation research in OHSs. On the basis of a conceptual model of OHS evaluation, empirical studies are categorised into aspects of input, process, output, outcome, and OHS core activities. Many methods to evaluate OHSs or OHS activities exist, depending on the objective and object of evaluation. The amount of empirical studies on evaluation of OHSs or OHS activities that met the non-restrictive inclusion criteria, was remarkably limited. Most of the 52 studies were more descriptive than evaluative. The methodological quality of most studies was not high. A differentiated picture of the evidence of effectiveness of OHSs arises. Occupational health consultations and occupational rehabilitation are hardly studied despite much time spent on the consultation by occupational physicians in most countries. The lack of effectiveness and efficiency of the pre-employment examination should lead to its abandonment as a means of selection of personnel by OHSs. Periodic health monitoring or surveillance, and education on occupational health hazards can be carried out with reasonable process quality. Identification and evaluation of occupational health hazards by a workplace survey can be done with a high output quality, which, however, does not guarantee a favourable outcome. Although rigorous study designs are not always applicable or feasible in daily practice, much more effort should be directed at the scientific evaluation of OHSs and OHS instruments. To develop evidence-based occupational health care the quality of evaluation studies should be improved. In particular, process and outcome of consultation and rehabilitation activities of occupational physicians need to be studied more.

Evaluation and communication: using a communication audit to evaluate organizational communication.

PubMed

Hogard, Elaine; Ellis, Roger

2006-04-01

This article identifies a surprising dearth of studies that explicitly link communication and evaluation at substantive, theoretical, and methodological levels. A three-fold typology of evaluation studies referring to communication is proposed and examples given. The importance of organizational communication in program delivery is stressed and illustrative studies reviewed. It is proposed that organizational communication should be considered in all program evaluations and that this should be approached through communication audit. Communication audits are described with particular reference to established survey questionnaire instruments. Two case studies exemplify the use of such instruments in the evaluation of educational and social programs.

Study Sensitivity: Evaluating the Ability to Detect Effects in Systematic Reviews of Chemical Exposures

PubMed Central

Cooper, Glinda S.; Lunn, Ruth M.; Ågerstrand, Marlene; Glenn, Barbara S.; Kraft, Andrew D.; Luke, April M.; Ratcliffe, Jennifer M.

2016-01-01

A critical step in systematic reviews of potential health hazards is the structured evaluation of the strengths and weaknesses of the included studies; risk of bias is a term often used to represent this process, specifically with respect to the evaluation of systematic errors that can lead to inaccurate (biased) results (i.e. focusing on internal validity). Systematic review methods developed in the clinical medicine arena have been adapted for use in evaluating environmental health hazards; this expansion raises questions about the scope of risk of bias tools and the extent to which they capture the elements that can affect the interpretation of results from environmental and occupational epidemiology studies and in vivo animal toxicology studies, (the studies typically available for assessment of risk of chemicals). One such element, described here as “sensitivity”, is a measure of the ability of a study to detect a true effect or hazard. This concept is similar to the concept of the sensitivity of an assay; an insensitive study may fail to show a difference that truly exists, leading to a false conclusion of no effect. Factors relating to study sensitivity should be evaluated in a systematic manner with the same rigor as the evaluation of other elements within a risk of bias framework. We discuss the importance of this component for the interpretation of individual studies, examine approaches proposed or in use to address it, and describe how it relates to other evaluation components. The evaluation domains contained within a risk of bias tool can include, or can be modified to include, some features relating to study sensitivity; the explicit inclusion of these sensitivity criteria with the same rigor and at the same stage of study evaluation as other bias-related criteria can improve the evaluation process. In some cases, these and other features may be better addressed through a separate sensitivity domain. The combined evaluation of risk of bias and sensitivity can be used to identify the most informative studies, to evaluate the confidence of the findings from individual studies and to identify those study elements that may help to explain heterogeneity across the body of literature. PMID:27156196

Evaluating radiographers' diagnostic accuracy in screen-reading mammograms: what constitutes a quality study?

DOE Office of Scientific and Technical Information (OSTI.GOV)

Debono, Josephine C, E-mail: josephine.debono@bci.org.au; Poulos, Ann E; Westmead Breast Cancer Institute, Westmead, New South Wales

The aim of this study was to first evaluate the quality of studies investigating the diagnostic accuracy of radiographers as mammogram screen-readers and then to develop an adapted tool for determining the quality of screen-reading studies. A literature search was used to identify relevant studies and a quality evaluation tool constructed by combining the criteria for quality of Whiting, Rutjes, Dinnes et al. and Brealey and Westwood. This constructed tool was then applied to the studies and subsequently adapted specifically for use in evaluating quality in studies investigating diagnostic accuracy of screen-readers. Eleven studies were identified and the constructed toolmore » applied to evaluate quality. This evaluation resulted in the identification of quality issues with the studies such as potential for bias, applicability of results, study conduct, reporting of the study and observer characteristics. An assessment of the applicability and relevance of the tool for this area of research resulted in adaptations to the criteria and the development of a tool specifically for evaluating diagnostic accuracy in screen-reading. This tool, with further refinement and rigorous validation can make a significant contribution to promoting well-designed studies in this important area of research and practice.« less

Social Studies. MicroSIFT Courseware Evaluations.

ERIC Educational Resources Information Center

Northwest Regional Educational Lab., Portland, OR.

This compilation of 11 courseware evaluations gives a general overview of available social studies microcomputer courseware for students in grades 3-12. Each evaluation lists title, date, producer, date of evaluation, evaluating institution, cost, ability level, topic, medium of transfer, required hardware, required software, instructional…

Social Studies. Microsift Courseware Evaluations.

ERIC Educational Resources Information Center

Northwest Regional Educational Lab., Portland, OR.

This compilation of 17 courseware evaluations gives a general overview of available social studies microcomputer courseware for students in grades 1-12. Each evaluation lists title, date, producer, date of evaluation, evaluating institution, cost, ability level, topic, medium of transfer, required hardware, required software, instructional…

Challenges of teacher-based clinical evaluation from nursing students' point of view: Qualitative content analysis.

PubMed

Sadeghi, Tabandeh; Seyed Bagheri, Seyed Hamid

2017-01-01

Clinical evaluation is very important in the educational system of nursing. One of the most common methods of clinical evaluation is evaluation by the teacher, but the challenges that students would face in this evaluation method, have not been mentioned. Thus, this study aimed to explore the experiences and views of nursing students about the challenges of teacher-based clinical evaluation. This study was a descriptive qualitative study with a qualitative content analysis approach. Data were gathered through semi-structured focused group sessions with undergraduate nursing students who were passing their 8 th semester at Rafsanjan University of Medical Sciences. Date were analyzed using Graneheim and Lundman's proposed method. Data collection and analysis were concurrent. According to the findings, "factitious evaluation" was the main theme of study that consisted of three categories: "Personal preferences," "unfairness" and "shirking responsibility." These categories are explained using quotes derived from the data. According to the results of this study, teacher-based clinical evaluation would lead to factitious evaluation. Thus, changing this approach of evaluation toward modern methods of evaluation is suggested. The finding can help nursing instructors to get a better understanding of the nursing students' point of view toward this evaluation approach and as a result could be planning for changing of this approach.

Cautions on the Use of Investigative Case Studies in Meta-Evaluation.

ERIC Educational Resources Information Center

Smith, Nick L.

1990-01-01

A meta-analysis combining expert evaluation with naturalistic case study methods indicates that such investigations must use special methods to render evaluative judgments of worth. It is demonstrated that descriptive, interpretive, and evaluative aspects of such a study must be combined to yield justifiable conclusions. (TJH)

Formative and Summative Evaluation: Related Issues in Performance Measurement.

ERIC Educational Resources Information Center

Wholey, Joseph S.

1996-01-01

Performance measurement can serve both formative and summative evaluation functions. Formative evaluation is typically more useful for government purposes whereas performance measurement is more useful than one-shot evaluations of either formative or summative nature. Evaluators should study performance measurement through case studies and…

Economic evaluations of clinical pharmacy services: 2006-2010.

PubMed

Touchette, Daniel R; Doloresco, Fred; Suda, Katie J; Perez, Alexandra; Turner, Stuart; Jalundhwala, Yash; Tangonan, Maria C; Hoffman, James M

2014-08-01

Studies have consistently evidenced the positive clinical, economic, and humanistic benefits of pharmacist-directed patient care in a variety of settings. Given the vast differences in clinical outcomes associated with evaluated clinical pharmacy services (CPS), more detail as to the nature of the CPS is needed to better understand observed differences in economic outcomes. With the growing trend of outpatient pharmacy services, these economic evaluations serve as viable decision-making tools in choosing the most effective and cost-effective pharmacy programs. We previously conducted three systematic reviews to evaluate the economic impact of CPS from 1988 to 2005. In this systematic review, our objectives were to describe and evaluate the quality of economic evaluations of CPS published between 2006 and 2010, with the goal of informing administrators and practitioners as to their cost-effectiveness. We searched the scientific literature by using the Medline, International Pharmaceutical Abstracts, Embase, and Cumulative Index to Nursing and Allied Health Literature databases to identify studies describing CPS published from 2006 to 2010. Studies meeting our inclusion criteria (original research articles that evaluated CPS and described economic and clinical outcomes) were reviewed by two investigators. Methodology used, economic evaluation type, CPS setting and type, and clinical and economic outcome results were extracted. Results were informally compared with previous systematic reviews. Of 3587 potential studies identified, 25 met inclusion criteria. Common CPS settings were hospital (36%), community (32%), and clinic or hospital-based ambulatory practices (28%). CPS types were disease state management (48%), general pharmacotherapeutic monitoring (24%), target drug programs (8%), and patient education (4%). Two studies (8%) listed CPS as medication therapy management. Costs were evaluated in 24 studies (96%) and sufficiently described in 13 (52%). Clinical or humanistic outcomes were evaluated in 20 studies (80%) and were sufficiently described in 18 (72%). Control groups were included in 16 (70%) of 23 studies not involving modeling. Study assumptions and limitations were stated and justified in eight studies (32%). Conclusions and recommendations were considered justified and based on results in 24 studies (96%). Eighteen studies (72%) involved full economic evaluation. The mean ± SD study quality score for full economic evaluations (18 studies) was 60.4 ± 22.3 of a possible 100 points. Benefit-cost ratios from three studies ranged from 1.05:1 to 25.95:1, and incremental cost-effectiveness ratios of five studies were calculated and reported. Fewer studies documented the economic impact of CPS from 2006-2010 than from 2001-2005, although a higher proportion involved controlled designs and were full economic evaluations. Evaluations of ambulatory practices were increasingly common. CPS were generally considered cost-effective or provided a good benefit-cost ratio. © 2014 Pharmacotherapy Publications, Inc.

Arts Integration and Students' Reading Achievement: A Formative Evaluation Study

ERIC Educational Resources Information Center

Hosfelt, Patricia D.

2017-01-01

The purpose of this dissertation was to evaluate essential components of an arts-integration program that may contribute to improved student achievement in elementary reading at the school of study through a formative evaluation. Stufflebeam's CIPP model of program evaluation served as the conceptual framework for the study's findings. Creative…

Evaluation Policy in a Nonprofit Foundation: A Case Study Exploration of the Robert Wood Johnson Foundation

ERIC Educational Resources Information Center

Dillman, Lisa M.; Christie, Christina A.

2017-01-01

This study explores the relationship between evaluation policies and evaluation practice. Through document analysis, interviews, and a multiple case study, the research examined the explicit and implicit policies overarching the evaluation work commissioned by the Robert Wood Johnson Foundation (RWJF) and explored how these policies are…

Evaluation systems for clinical governance development: a comparative study.

PubMed

Hooshmand, Elaheh; Tourani, Sogand; Ravaghi, Hamid; Ebrahimipour, Hossein

2014-01-01

Lack of scientific and confirmed researches and expert knowledge about evaluation systems for clinical governance development in Iran have made studies on different evaluation systems for clinical governance development a necessity. These studies must provide applied strategies to design criteria of implementing clinical governance for hospital's accreditation. This is a descriptive and comparative study on development of clinical governance models all over the world. Data have been gathered by reviewing related articles. Models have been studied in comprehensive review method. The evaluated models of clinical governance development were Australian, NHS, SPOCK and OPTIGOV. The final aspects extracted from these models were Responsiveness, Policies and Strategies, Organizational Structure, Allocating Resources, Education and Occupational Development, Performance Evaluation, External Evaluation, Patient Oriented Approach, Risk Management, Personnel's Participation, Information Technology, Human Resources, Research and Development, Evidence Based Medicine, Clinical Audit, Health Technology Assessment and Quality. These results are applicable for completing the present criteria which evaluating clinical governance application and provide practical framework to evaluate country's hospital on the basis of clinical governance elements.

Students Evaluation of Faculty

ERIC Educational Resources Information Center

Thawabieh, Ahmad M.

2017-01-01

This study aimed to investigate how students evaluate their faculty and the effect of gender, expected grade, and college on students' evaluation. The study sample consisted of 5291 students from Tafila Technical University Faculty evaluation scale was used to collect data. The results indicated that student evaluation of faculty was high (mean =…

An Analysis of State and Local Alignment of Teacher Evaluation in Maryland

ERIC Educational Resources Information Center

Peterson, Serene N.

2014-01-01

This study explored the components of Maryland's newly-implemented teacher evaluation framework and compared state requirements with evaluations to three local school systems' evaluation procedures. The study sought to investigate the relationship between three evaluation protocols in comparison to the state requirements. Three local school…

Study sensitivity: Evaluating the ability to detect effects in systematic reviews of chemical exposures.

PubMed

Cooper, Glinda S; Lunn, Ruth M; Ågerstrand, Marlene; Glenn, Barbara S; Kraft, Andrew D; Luke, April M; Ratcliffe, Jennifer M

2016-01-01

A critical step in systematic reviews of potential health hazards is the structured evaluation of the strengths and weaknesses of the included studies; risk of bias is a term often used to represent this process, specifically with respect to the evaluation of systematic errors that can lead to inaccurate (biased) results (i.e. focusing on internal validity). Systematic review methods developed in the clinical medicine arena have been adapted for use in evaluating environmental health hazards; this expansion raises questions about the scope of risk of bias tools and the extent to which they capture the elements that can affect the interpretation of results from environmental and occupational epidemiology studies and in vivo animal toxicology studies, (the studies typically available for assessment of risk of chemicals). One such element, described here as "sensitivity", is a measure of the ability of a study to detect a true effect or hazard. This concept is similar to the concept of the sensitivity of an assay; an insensitive study may fail to show a difference that truly exists, leading to a false conclusion of no effect. Factors relating to study sensitivity should be evaluated in a systematic manner with the same rigor as the evaluation of other elements within a risk of bias framework. We discuss the importance of this component for the interpretation of individual studies, examine approaches proposed or in use to address it, and describe how it relates to other evaluation components. The evaluation domains contained within a risk of bias tool can include, or can be modified to include, some features relating to study sensitivity; the explicit inclusion of these sensitivity criteria with the same rigor and at the same stage of study evaluation as other bias-related criteria can improve the evaluation process. In some cases, these and other features may be better addressed through a separate sensitivity domain. The combined evaluation of risk of bias and sensitivity can be used to identify the most informative studies, to evaluate the confidence of the findings from individual studies and to identify those study elements that may help to explain heterogeneity across the body of literature. Copyright © 2016. Published by Elsevier Ltd.

Presenting an Evaluation Model for the Cancer Registry Software.

PubMed

Moghaddasi, Hamid; Asadi, Farkhondeh; Rabiei, Reza; Rahimi, Farough; Shahbodaghi, Reihaneh

2017-12-01

As cancer is increasingly growing, cancer registry is of great importance as the main core of cancer control programs, and many different software has been designed for this purpose. Therefore, establishing a comprehensive evaluation model is essential to evaluate and compare a wide range of such software. In this study, the criteria of the cancer registry software have been determined by studying the documents and two functional software of this field. The evaluation tool was a checklist and in order to validate the model, this checklist was presented to experts in the form of a questionnaire. To analyze the results of validation, an agreed coefficient of %75 was determined in order to apply changes. Finally, when the model was approved, the final version of the evaluation model for the cancer registry software was presented. The evaluation model of this study contains tool and method of evaluation. The evaluation tool is a checklist including the general and specific criteria of the cancer registry software along with their sub-criteria. The evaluation method of this study was chosen as a criteria-based evaluation method based on the findings. The model of this study encompasses various dimensions of cancer registry software and a proper method for evaluating it. The strong point of this evaluation model is the separation between general criteria and the specific ones, while trying to fulfill the comprehensiveness of the criteria. Since this model has been validated, it can be used as a standard to evaluate the cancer registry software.

Attitudes of affiliate faculty members toward medical student summative evaluation for clinical clerkships: a qualitative analysis.

PubMed

Wang, Karen E; Fitzpatrick, Caroline; George, David; Lane, Lindsey

2012-01-01

Summative evaluation of medical students is a critical component of the educational process. Despite extensive literature on evaluation, few studies have centered on affiliate faculty members' attitudes toward summative evaluation of students, though it has been suggested that these attitudes influence their effectiveness as evaluators. The objective is to examine affiliate faculty members' attitudes toward clinical clerkship evaluation using primarily qualitative research methods. The study used a nonexperimental research design and employed mixed methods. Data were collected through interviews, focus groups, and a questionnaire from 11 affiliate faculty members. Themes emerging from the data fell into three broad categories: (a) factors that influence grading, (b) consequences of negative evaluations, and (c) disconnections in the grading process. The quantitative portion of the study revealed important discrepancies supporting the use of qualitative methods. The study highlights faculty members' struggles with the evaluative process and emphasizes the need for improvements in evaluation tools and faculty development.

A systematic review of economic evaluations of health and health-related interventions in Bangladesh

PubMed Central

2011-01-01

Background Economic evaluation is used for effective resource allocation in health sector. Accumulated knowledge about economic evaluation of health programs in Bangladesh is not currently available. While a number of economic evaluation studies have been performed in Bangladesh, no systematic investigation of the studies has been done to our knowledge. The aim of this current study is to systematically review the published articles in peer-reviewed journals on economic evaluation of health and health-related interventions in Bangladesh. Methods Literature searches was carried out during November-December 2008 with a combination of key words, MeSH terms and other free text terms as suitable for the purpose. A comprehensive search strategy was developed to search Medline by the PubMed interface. The first specific interest was mapping the articles considering the areas of exploration by economic evaluation and the second interest was to scrutiny the methodological quality of studies. The methodological quality of economic evaluation of all articles has been scrutinized against the checklist developed by Evers Silvia and associates. Result Of 1784 potential articles 12 were accepted for inclusion. Ten studies described the competing alternatives clearly and only two articles stated the perspective of their articles clearly. All studies included direct cost, incurred by the providers. Only one study included the cost of community donated resources and volunteer costs. Two studies calculated the incremental cost effectiveness ratio (ICER). Six of the studies applied some sort of sensitivity analysis. Two of the studies discussed financial affordability of expected implementers and four studies discussed the issue of generalizability for application in different context. Conclusion Very few economic evaluation studies in Bangladesh are found in different areas of health and health-related interventions, which does not provide a strong basis of knowledge in the area. The most frequently applied economic evaluation is cost-effectiveness analysis. The majority of the studies did not follow the scientific method of economic evaluation process, which consequently resulted into lack of robustness of the analyses. Capacity building on economic evaluation of health and health-related programs should be enhanced. PMID:21771343

Impact of a Major National Evaluation Study: Israel's Van Leer Report.

ERIC Educational Resources Information Center

Alkin, Marvin C.; Lewy, Arieh

This investigation documents the impact of the Van Leer Study, a large-scale evaluation study of achievement in the primary schools of Israel. It is intended to increase understanding of the process of evaluation utilization, showing how evaluation findings and other kinds of information can work together, over time and in a variety of ways, to…

Strategies for Evaluating a Freshman Studies Program.

ERIC Educational Resources Information Center

Ketkar, Kusum; Bennett, Shelby D.

1989-01-01

The study developed an economic model for the evaluation of Seaton Hall University's freshman studies program. Two techniques used to evaluate the economic success of the program are break-even analysis and elasticity coefficient. (Author/MLW)

Rebooting Kirkpatrick: Integrating Information System Theory Into the Evaluation of Web-based Continuing Professional Development Interventions for Interprofessional Education.

PubMed

Shen, Nelson; Yufe, Shira; Saadatfard, Omid; Sockalingam, Sanjeev; Wiljer, David

2017-01-01

Information system research has stressed the importance of theory in understanding how user perceptions can motivate the use and adoption of technology such as web-based continuing professional development programs for interprofessional education (WCPD-IPE). A systematic review was conducted to provide an information system perspective on the current state of WCPD-IPE program evaluation and how current evaluations capture essential theoretical constructs in promoting technology adoption. Six databases were searched to identify studies evaluating WCPD-IPE. Three investigators determined eligibility of the articles. Evaluation items extracted from the studies were assessed using the Kirkpatrick-Barr framework and mapped to the Benefits Evaluation Framework. Thirty-seven eligible studies yielded 362 evaluation items for analysis. Most items (n = 252) were assessed as Kirkpatrick-Barr level 1 (reaction) and were mainly focused on the quality (information, service, and quality) and satisfaction dimensions of the Benefits Evaluation. System quality was the least evaluated quality dimension, accounting for 26 items across 13 studies. WCPD-IPE use was reported in 17 studies and its antecedent factors were evaluated in varying degrees of comprehensiveness. Although user reactions were commonly evaluated, greater focus on user perceptions of system quality (ie, functionality and performance), usefulness, and usability of the web-based platform is required. Surprisingly, WCPD-IPE use was reported in less than half of the studies. This is problematic as use is a prerequisite to realizing any individual, organizational, or societal benefit of WCPD-IPE. This review proposes an integrated framework which accounts for these factors and provides a theoretically grounded guide for future evaluations.

Insight into Evaluation Practice: A Content Analysis of Designs and Methods Used in Evaluation Studies Published in North American Evaluation-Focused Journals

ERIC Educational Resources Information Center

Christie, Christina A.; Fleischer, Dreolin Nesbitt

2010-01-01

To describe the recent practice of evaluation, specifically method and design choices, the authors performed a content analysis on 117 evaluation studies published in eight North American evaluation-focused journals for a 3-year period (2004-2006). The authors chose this time span because it follows the scientifically based research (SBR)…

Course Evaluation in Sweden--When, How, What and Why

ERIC Educational Resources Information Center

Cronholm, Stefan

2010-01-01

This study is about course evaluation in Swedish higher education. Performing course evaluation is regulated in Swedish law. Despite this, only half of the courses are evaluated. The aim of this study is to understand why satisfactory course evaluations not are performed. Problems are identified from a student perspective and the paper provides…

Textbook Evaluation for the Students of Speech Therapy

ERIC Educational Resources Information Center

Jamshidi, Tahereh; Soori, Afshin

2013-01-01

This study aimed to evaluate an ESP textbook in terms of McDonough and Shaw (2003) based on external and internal evaluation. The ESP textbook was "Special English for Computer Sciences" (2010) by Hojjat Baghban. This study also discussed the external evaluation and a detailed evaluation of a chapter of the ESP textbook. This ESP…

Economic evaluation of diagnostic methods used in dentistry. A systematic review.

PubMed

Christell, Helena; Birch, Stephen; Horner, Keith; Lindh, Christina; Rohlin, Madeleine

2014-11-01

To review the literature of economic evaluations regarding diagnostic methods used in dentistry. Four databases (MEDLINE, Web of Science, The Cochrane library, the NHS Economic Evaluation Database) were searched for studies, complemented by hand search, until February 2013. Two authors independently screened all titles or abstracts and then applied inclusion and exclusion criteria to select full-text publications published in English, which reported an economic evaluation comparing at least two alternative methods. Studies of diagnostic methods were assessed by four reviewers using a protocol based on the QUADAS tool regarding diagnostic methods and a check-list for economic evaluations. The results of the data extraction were summarized in a structured table and as a narrative description. From 476 identified full-text publications, 160 were considered to be economic evaluations. Only 12 studies (7%) were on diagnostic methods, whilst 78 studies (49%) were on prevention and 70 (40%) on treatment. Among studies on diagnostic methods, there was between-study heterogeneity methodologically, regarding the diagnostic method analysed and type of economic evaluation addressed. Generally, the choice of economic evaluation method was not justified and the perspective of the study not stated. Costing of diagnostic methods varied. A small body of literature addresses economic evaluation of diagnostic methods in dentistry. Thus, there is a need for studies from various perspectives with well defined research questions and measures of the cost and effectiveness. Economic resources in healthcare are finite. For diagnostic methods, an understanding of efficacy provides only part of the information needed for evidence-based practice. This study highlighted a paucity of economic evaluations of diagnostic methods used in dentistry, indicating that much of what we practise lacks sufficient evidence.

Simulation studies for the evaluation of health information technologies: experiences and results.

PubMed

Ammenwerth, Elske; Hackl, Werner O; Binzer, Kristine; Christoffersen, Tue E H; Jensen, Sanne; Lawton, Kitta; Skjoet, Peter; Nohr, Christian

It is essential for new health information technologies (IT) to undergo rigorous evaluations to ensure they are effective and safe for use in real-world situations. However, evaluation of new health IT is challenging, as field studies are often not feasible when the technology being evaluated is not sufficiently mature. Laboratory-based evaluations have also been shown to have insufficient external validity. Simulation studies seem to be a way to bridge this gap. The aim of this study was to evaluate, using a simulation methodology, the impact of a new prototype of an electronic medication management system on the appropriateness of prescriptions and drug-related activities, including laboratory test ordering or medication changes. This article presents the results of a controlled simulation study with 50 simulation runs, including ten doctors and five simulation patients, and discusses experiences and lessons learnt while conducting the study. Although the new electronic medication management system showed tendencies to improve medication safety when compared with the standard system, this tendency was not significant. Altogether, five distinct situations were identified where the new medication management system did help to improve medication safety. This simulation study provided a good compromise between internal validity and external validity. However, several challenges need to be addressed when undertaking simulation evaluations including: preparation of adequate test cases; training of participants before using unfamiliar applications; consideration of time, effort and costs of conducting the simulation; technical maturity of the evaluated system; and allowing adequate preparation of simulation scenarios and simulation setting. Simulation studies are an interesting but time-consuming approach, which can be used to evaluate newly developed health IT systems, particularly those systems that are not yet sufficiently mature to undergo field evaluation studies.

Revisioning the Process: A Case Study in Feminist Program Evaluation.

ERIC Educational Resources Information Center

Beardsley, Rebecca M.; Miller, Michelle Hughes

2002-01-01

Conducted a case study of the evaluation of a women's substance abuse prevention program and identified three key aspects of negotiated evaluation. Discusses the processes involved in feminist evaluation, including collaborative agenda setting and cooperative teamwork. (SLD)

Evaluating Voucher Programs: The Milwaukee Parental Choice Program

ERIC Educational Resources Information Center

Witte, John F.

2016-01-01

This paper is the first summary of two studies and 10 years of evaluating the Milwaukee Parental Choice (voucher) Program (MPCP). This paper discusses school voucher evaluations in general terms and how these studies are carried out. The paper outlines the types of studies completed in "Study I" and "Study II" and the results…

An Evaluative Case Study on Professional Competency of Preservice Information Technology Teachers

ERIC Educational Resources Information Center

Kabakci Yurdakul, Isil

2011-01-01

The purpose of this study is to evaluate preservice Information Technology (IT) teachers' professional competency in the teaching process. The study was designed on the basis of evaluative case study. The participants of the study consisted of seven preservice IT teachers attending the department of Computer Education and Instructional…

A systematic review of economic evaluations of population-based sodium reduction interventions.

PubMed

Hope, Silvia F; Webster, Jacqui; Trieu, Kathy; Pillay, Arti; Ieremia, Merina; Bell, Colin; Snowdon, Wendy; Neal, Bruce; Moodie, Marj

2017-01-01

To summarise evidence describing the cost-effectiveness of population-based interventions targeting sodium reduction. A systematic search of published and grey literature databases and websites was conducted using specified key words. Characteristics of identified economic evaluations were recorded, and included studies were appraised for reporting quality using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Twenty studies met the study inclusion criteria and received a full paper review. Fourteen studies were identified as full economic evaluations in that they included both costs and benefits associated with an intervention measured against a comparator. Most studies were modelling exercises based on scenarios for achieving salt reduction and assumed effects on health outcomes. All 14 studies concluded that their specified intervention(s) targeting reductions in population sodium consumption were cost-effective, and in the majority of cases, were cost saving. Just over half the studies (8/14) were assessed as being of 'excellent' reporting quality, five studies fell into the 'very good' quality category and one into the 'good' category. All of the identified evaluations were based on modelling, whereby inputs for all the key parameters including the effect size were either drawn from published datasets, existing literature or based on expert advice. Despite a clear increase in evaluations of salt reduction programs in recent years, this review identified relatively few economic evaluations of population salt reduction interventions. None of the studies were based on actual implementation of intervention(s) and the associated collection of new empirical data. The studies universally showed that population-based salt reduction strategies are likely to be cost effective or cost saving. However, given the reliance on modelling, there is a need for the effectiveness of new interventions to be evaluated in the field using strong study designs and parallel economic evaluations.

Michigan English Study of Structure for Curriculum Evaluation (MESSAGE).

ERIC Educational Resources Information Center

Taylor, Vi Marie

The Michigan English Study of Structure for Curriculum Evaluation (MESSAGE) training session involved 40 leaders of English curriculum study in a 7-day intensive program. They studied objective evaluation in terms of behavior, instruction, and institution; approaches to the English curriculum through instructional objectives that include…

How social context moderates the self-evaluative emotions experienced due to health risk behaviour.

PubMed

Grob, Judith D M; Dijkstra, Arie; de Groot, Carla

2011-10-01

When people are confronted with the potential negative physical outcomes of their own health risk behaviour, they experience a self-threat. This threat is felt as negative self-evaluative emotions. We hypothesise that the threat will lead to more private self-evaluative emotions (e.g. regret) in a private social context, whereas more public self-evaluative emotions (e.g. embarrassment) will be felt in a public social context with negative norms. Consistent with our hypotheses, we show that participants anticipate feeling more private self-evaluative emotions when confronted with the negative consequences of their unhealthy behaviour when alone, and more public self-evaluative emotions when in a group (Study 1). They further anticipate more public self-evaluative emotions in response to a health self-threat when the group norm is negative, and more private self-evaluative emotions when the group norm is lenient (Study 2). Finally, in a cross-sectional study amongst smokers, we show that private but not public negative self-evaluative emotions concerning their own smoking habits are positively correlated with the intent to quit smoking (Study 3). These studies show that a distinction needs to be made between public and private self-evaluative emotions, in terms of their antecedents and effects. Theoretical implications and further lines of research are discussed.

Head-camera video recordings of trauma core competency procedures can evaluate surgical resident's technical performance as well as colocated evaluators.

PubMed

Mackenzie, Colin F; Pasley, Jason; Garofalo, Evan; Shackelford, Stacy; Chen, Hegang; Longinaker, Nyaradzo; Granite, Guinevere; Pugh, Kristy; Hagegeorge, George; Tisherman, Samuel A

2017-07-01

Unbiased evaluation of trauma core competency procedures is necessary to determine if residency and predeployment training courses are useful. We tested whether a previously validated individual procedure score (IPS) for individual procedure vascular exposure and fasciotomy (FAS) performance skills could discriminate training status by comparing IPS of evaluators colocated with surgeons to blind video evaluations. Performance of axillary artery (AA), brachial artery (BA), and femoral artery (FA) vascular exposures and lower extremity FAS on fresh cadavers by 40 PGY-2 to PGY-6 residents was video-recorded from head-mounted cameras. Two colocated trained evaluators assessed IPS before and after training. One surgeon in each pretraining tertile of IPS for each procedure was randomly identified for blind video review. The same 12 surgeons were video-recorded repeating the procedures less than 4 weeks after training. Five evaluators independently reviewed all 96 randomly arranged deidentified videos. Inter-rater reliability/consistency, intraclass correlation coefficients were compared by colocated versus video review of IPS, and errors. Study methodology and bias were judged by Medical Education Research Study Quality Instrument and the Quality Assessment of Diagnostic Accuracy Studies criteria. There were no differences (p ≥ 0.5) in IPS for AA, FA, FAS, whether evaluators were colocated or reviewed video recordings. Evaluator consistency was 0.29 (BA) - 0.77 (FA). Video and colocated evaluators were in total agreement (p = 1.0) for error recognition. Intraclass correlation coefficient was 0.73 to 0.92, dependent on procedure. Correlations video versus colocated evaluations were 0.5 to 0.9. Except for BA, blinded video evaluators discriminated (p < 0.002) whether procedures were performed before training versus after training. Study methodology by Medical Education Research Study Quality Instrument criteria scored 15.5/19, Quality Assessment of Diagnostic Accuracy Studies 2 showed low bias risk. Video evaluations of AA, FA, and FAS procedures with IPS are unbiased, valid, and have potential for formative assessments of competency. Prognostic study, level II.

Developing Statistical Physics Course Handout on Distribution Function Materials Based on Science, Technology, Engineering, and Mathematics

NASA Astrophysics Data System (ADS)

Riandry, M. A.; Ismet, I.; Akhsan, H.

2017-09-01

This study aims to produce a valid and practical statistical physics course handout on distribution function materials based on STEM. Rowntree development model is used to produce this handout. The model consists of three stages: planning, development and evaluation stages. In this study, the evaluation stage used Tessmer formative evaluation. It consists of 5 stages: self-evaluation, expert review, one-to-one evaluation, small group evaluation and field test stages. However, the handout is limited to be tested on validity and practicality aspects, so the field test stage is not implemented. The data collection technique used walkthroughs and questionnaires. Subjects of this study are students of 6th and 8th semester of academic year 2016/2017 Physics Education Study Program of Sriwijaya University. The average result of expert review is 87.31% (very valid category). One-to-one evaluation obtained the average result is 89.42%. The result of small group evaluation is 85.92%. From one-to-one and small group evaluation stages, averagestudent response to this handout is 87,67% (very practical category). Based on the results of the study, it can be concluded that the handout is valid and practical.

Exploring Organizational Evaluation Capacity and Evaluation Capacity Building: A Delphi Study of Taiwanese Elementary and Junior High Schools

ERIC Educational Resources Information Center

Cheng, Shu-Huei; King, Jean A.

2017-01-01

Researchers have conducted numerous empirical studies on evaluation capacity (EC) and evaluation capacity building (ECB) in Western cultural settings. However, little is known about these practices in non-Western contexts. To that end, this study identified the major dimensions of EC and feasible ECB approaches in Taiwanese elementary and junior…

An Evaluation Study of Youth Participation in Youth Work: A Case Study in Southern Italy

ERIC Educational Resources Information Center

Morciano, Daniele; Scardigno, Anna Fausta; Manuti, Amelia; Pastore, Serafina

2014-01-01

In this paper an evaluation study of a public programme financing a regional network of 157 youth centres in the South of Italy is presented. A theory-based evaluation model was adopted to explore the causal links between different types of participation experience. Evaluation questions focused on three main issues are: the perception of…

Evaluating Work-Based Learning: Insights from an Illuminative Evaluation Study of Work-Based Learning in a Vocational Qualification

ERIC Educational Resources Information Center

van Rensburg, Estelle

2008-01-01

This article outlines an illuminative evaluation study of the work-based module in a vocational qualification in Animal Health offered for the paraveterinary industry by a distance education institution in South Africa. In illuminative evaluation, a programme is studied by qualitative methods to gain an in-depth understanding of its…

Critical appraisal of published economic evaluations of home care for the elderly.

PubMed

Ramos, Maria Lucia Teixeira; Ferraz, Marcos Bosi; Sesso, Ricardo

2004-01-01

The goal of the study was to appraise the economic evaluations published between 1980 and 2004 of "home care" for the elderly, focusing on the methodological aspects. MEDLINE was searched to identify and assess economic evaluations (defined as an analysis comparing two or more strategies, involving the assessment of both costs and consequences) related to "home care" exclusively for the elderly (65 years or more) and to critically appraise the methodology using five accepted principles used worldwide for conducting economic evaluations. Twenty-four economic evaluations of "home care" for the elderly were identified and the articles were assessed. All five principles were satisfactorily addressed in two studies (8.3%), four principles in four studies (16.7%), three principles in five studies (20.8%), two principles in eight studies (33.3%) and only one principle in five studies (20.8%). A disparity in the methodology of writing economic evaluations compromises the comparisons among outcomes and lately jeopardizes decisions on the choice of the most appropriate healthcare interventions. The methodological principles represent important guidelines but the discussion of the context of the economic evaluation and the special characteristics of some services and populations should be considered for the appropriate use of economic evaluations.

Performing well in an evaluative situation: the roles of perceived competence and task-irrelevant interfering thoughts.

PubMed

Van Yperen, Nico W

2007-12-01

This research expands on previous research by arguing and demonstrating that high perceived competence buffers the detrimental effects of an evaluative situation. In Study 1 (n=75, 38.7% male), the situation (evaluative vs. non-evaluative) and perceived competence (high vs. low) were manipulated, whereas in Study 2 (n=42, 33.3% male), perceived competence relied on naturally occurring differences in perceived competence. The results of Study 1 indicate that people may underachieve in an evaluative situation. More importantly, in Study 2 it was demonstrated that such an evaluative situation had only a negative effect on test performance among individuals low in perceived competence. The occurrence of task-irrelevant interfering thoughts during task completion accounted for this inimical effect of an evaluative situation on test performance among these individuals.

Economic analysis of the health impacts of housing improvement studies: a systematic review.

PubMed

Fenwick, Elisabeth; Macdonald, Catriona; Thomson, Hilary

2013-10-01

Economic evaluation of public policies has been advocated but rarely performed. Studies from a systematic review of the health impacts of housing improvement included data on costs and some economic analysis. Examination of these data provides an opportunity to explore the difficulties and the potential for economic evaluation of housing. Data were extracted from all studies included in the systematic review of housing improvement which had reported costs and economic analysis (n=29/45). The reported data were assessed for their suitability to economic evaluation. Where an economic analysis was reported the analysis was described according to pre-set definitions of various types of economic analysis used in the field of health economics. 25 studies reported cost data on the intervention and/or benefits to the recipients. Of these, 11 studies reported data which was considered amenable to economic evaluation. A further four studies reported conducting an economic evaluation. Three of these studies presented a hybrid 'balance sheet' approach and indicated a net economic benefit associated with the intervention. One cost-effectiveness evaluation was identified but the data were unclearly reported; the cost-effectiveness plane suggested that the intervention was more costly and less effective than the status quo. Future studies planning an economic evaluation need to (i) make best use of available data and (ii) ensure that all relevant data are collected. To facilitate this, economic evaluations should be planned alongside the intervention with input from health economists from the outset of the study. When undertaken appropriately, economic evaluation provides the potential to make significant contributions to housing policy.

Analyzing the School Evaluation Use Process To Make Evaluation Worth the Effort.

ERIC Educational Resources Information Center

Pechman, Ellen M.; King, Jean A.

This paper describes a structure for assessing the school evaluation use process developed from a longitudinal case study of districtwide and school level evaluation procedures in a large urban school district. Two fundamental questions guided the study: (1) Why isn't the evaluation process more useful to decision-makers and practitioners? and (2)…

Learning from Evaluation by Peer Team: A Case Study of a Family Counselling Organization

ERIC Educational Resources Information Center

Muniute-Cobb, Eivina I.; Alfred, Mary V.

2010-01-01

This qualitative study explores how employees learn from Team Primacy Concept-based employee evaluation and how they use the feedback in performing their jobs. Team Primacy Concept-based evaluation is a type of multirater evaluation. The distinctive characteristic of such evaluation is its peer feedback component during which the employee's…

Products for Improving Educational Evaluation. Center for the Study of Evaluation. Fifth Annual Report.

ERIC Educational Resources Information Center

Alkin, Marvin C.

This publication provides background information on the functions and operations of the Center for the Study of Evaluation and reports on such center products as Insructional Objectives Exchange (IOX), CSE Elementary School Test Evaluations, and Evaluation Workshop I. Appendixes include: a summary of center accomplishments; a list of the center's…

Examining Teacher Evaluation Validity and Leadership Decision Making within a Standards-Based Evaluation System

ERIC Educational Resources Information Center

Kimball, Steven M.; Milanowski, Anthony

2009-01-01

Purpose: The article reports on a study of school leader decision making that examined variation in the validity of teacher evaluation ratings in a school district that has implemented a standards-based teacher evaluation system. Research Methods: Applying mixed methods, the study used teacher evaluation ratings and value-added student achievement…

User Experience Evaluations in Rehabilitation Video Games for Children: A Systematic Mapping of the Literature.

PubMed

Rico-Olarte, Carolina; López, Diego M; Blobel, Bernd; Kepplinger, Sara

2017-01-01

In recent years, the interest in user experience (UX) evaluation methods for assessing technology solutions, especially in health systems for children with special needs like cognitive disabilities, has increased. Conduct a systematic mapping study to provide an overview in the field of UX evaluations in rehabilitation video games for children. The definition of research questions, the search for primary studies and the extraction of those studies by inclusion and exclusion criteria lead to the mapping of primary papers according to a classification scheme. Main findings from this study include the detection of the target population of the selected studies, the recognition of two different ways of evaluating UX: (i) user evaluation and (ii) system evaluation, and UX measurements and devices used. This systematic mapping specifies the research gaps identified for future research works in the area.

On the importance of cognitive evaluation as a determinant of interpersonal attraction.

PubMed

Montoya, R Matthew; Horton, Robert S

2004-05-01

Three studies examined a model of attraction in which the cognitive evaluation of the target individual was the primary determinant of interpersonal attraction. In Study 1, the cognitive evaluation of the target individual mediated the influence of attitude similarity on interpersonal attraction. In Study 2, a path analysis revealed significant indirect effects of (a) similarity on cognitive evaluation via the valence of information implied by attitudes and (b) the valence of information implied by attitudes on attraction via cognitive evaluation of the target. Study 3 provided empirical and theoretical support for the uniqueness of interpersonal attraction from cognitive evaluation. The implications of these data for existing attraction theory are discussed, and a new model of interpersonal attraction is described.

Economic evaluation of vaccines in Canada: A systematic review.

PubMed

Chit, Ayman; Lee, Jason K H; Shim, Minsup; Nguyen, Van Hai; Grootendorst, Paul; Wu, Jianhong; Van Exan, Robert; Langley, Joanne M

2016-05-03

Economic evaluations should form part of the basis for public health decision making on new vaccine programs. While Canada's national immunization advisory committee does not systematically include economic evaluations in immunization decision making, there is increasing interest in adopting them. We therefore sought to examine the extent and quality of economic evaluations of vaccines in Canada. We conducted a systematic review of economic evaluations of vaccines in Canada to determine and summarize: comprehensiveness across jurisdictions, studied vaccines, funding sources, study designs, research quality, and changes over time. Searches in multiple databases were conducted using the terms "vaccine," "economics" and "Canada." Descriptive data from eligible manuscripts was abstracted and three authors independently evaluated manuscript quality using a 7-point Likert-type scale scoring tool based on criteria from the International Society for Pharmacoeconomics and Outcomes Research (ISPOR). 42/175 articles met the search criteria. Of these, Canada-wide studies were most common (25/42), while provincial studies largely focused on the three populous provinces of Ontario, Quebec and British Columbia. The most common funding source was industry (17/42), followed by government (7/42). 38 studies used mathematical models estimating expected economic benefit while 4 studies examined post-hoc data on established programs. Studies covered 10 diseases, with 28/42 addressing pediatric vaccines. Many studies considered cost-utility (22/42) and the majority of these studies reported favorable economic results (16/22). The mean quality score was 5.9/7 and was consistent over publication date, funding sources, and disease areas. We observed diverse approaches to evaluate vaccine economics in Canada. Given the increased complexity of economic studies evaluating vaccines and the impact of results on public health practice, Canada needs improved, transparent and consistent processes to review and assess the findings of the economic evaluations of vaccines.

Objective evaluation of acute adverse events and image quality of gadolinium-based contrast agents (gadobutrol and gadobenate dimeglumine) by blinded evaluation. Pilot study.

PubMed

Semelka, Richard C; Hernandes, Mateus de A; Stallings, Clifton G; Castillo, Mauricio

2013-01-01

The purpose was to objectively evaluate a recently FDA-approved gadolinium-based contrast agent (GBCA) in comparison to our standard GBCA for acute adverse events and image quality by blinded evaluation. Evaluation was made of a recently FDA-approved GBCA, gadobutrol (Gadavist; Bayer), in comparison to our standard GBCA, gadobenate dimeglumine (MultiHance; Bracco), in an IRB- and HIPAA-compliant study. Both the imaging technologist and patient were not aware of the brand of the GBCA used. A total of 59 magnetic resonance studies were evaluated (59 patients, 31 men, 28 women, age range of 5-85 years, mean age of 52 years). Twenty-nine studies were performed with gadobutrol (22 abdominal and 7 brain studies), and 30 studies were performed with gadobenate dimeglumine (22 abdominal and 8 brain studies). Assessment was made of acute adverse events focusing on objective observations of vomiting, hives, and moderate and severe reactions. Adequacy of enhancement was rated as poor, fair and good by one of two experienced radiologists who were blinded to the type of agent evaluated. No patient experienced acute adverse events with either agent. The target minor adverse events of vomiting or hives, and moderate and severe reactions were not observed in any patient. Adequacy of enhancement was rated as good for both agents in all patients. Objective, blinded evaluation is feasible and readily performable for the evaluation of GBCAs. This proof-of-concept study showed that both GBCAs evaluated exhibited consistent good image quality and no noteworthy adverse events. Copyright © 2013 Elsevier Inc. All rights reserved.

Unintended outcomes evaluation approach: A plausible way to evaluate unintended outcomes of social development programmes.

PubMed

Jabeen, Sumera

2018-06-01

Social development programmes are deliberate attempts to bring about change and unintended outcomes can be considered as inherent to any such intervention. There is now a solid consensus among the international evaluation community regarding the need to consider unintended outcomes as a key aspect in any evaluative study. However, this concern often equates to nothing more than false piety. Exiting evaluation theory suffers from overlap of terminology, inadequate categorisation of unintended outcomes and lack of guidance on how to study them. To advance the knowledge of evaluation theory, methods and practice, the author has developed an evaluation approach to study unintended effects using a theory building, testing and refinement process. A comprehensive classification of unintended outcomes on the basis of knowability, value, distribution and temporality helped specify various type of unintended outcomes for programme evaluation. Corresponding to this classification, a three-step evaluation process was proposed including a) outlining programme intentions b) forecasting likely unintended effects c) mapping the anticipated and understanding unanticipated unintended outcomes. This unintended outcomes evaluation approach (UOEA) was then trialled by undertaking a multi-site and multi-method case study of a poverty alleviation programme in Pakistan and refinements were made to the approach.The case study revealed that this programme was producing a number of unintended effects, mostly negative, affecting those already disadvantaged such as the poorest, women and children. The trialling process demonstrated the effectiveness of the UOEA and suggests that this can serve as a useful guide for future evaluation practice. It also provides the discipline of evaluation with an empirically-based reference point for further theoretical developments in the study of unintended outcomes. Copyright © 2017 Elsevier Ltd. All rights reserved.

Barriers to and Facilitators of the Evaluation of Integrated Community-Wide Overweight Intervention Approaches: A Qualitative Case Study in Two Dutch Municipalities

PubMed Central

van Koperen, Tessa M.; de Kruif, Anja; van Antwerpen, Lisa; Hendriks, Anna-Marie; Seidell, Jacob C.; Schuit, Albertine J.; Renders, Carry M.

2016-01-01

To prevent overweight and obesity the implementation of an integrated community-wide intervention approach (ICIA) is often advocated. Evaluation can enhance implementation of such an approach and demonstrate the extent of effectiveness. To be able to support professionals in the evaluation of ICIAs we studied barriers to and facilitators of ICIA evaluation. In this study ten professionals of two Dutch municipalities involved in the evaluation of an ICIA participated. We conducted semi-structured interviews (n = 12), observed programme meetings (n = 4) and carried out document analysis. Data were analyzed using a thematic content approach. We learned that evaluation is hampered when it is perceived as unfeasible due to limited time and budget, a lack of evaluation knowledge or a negative evaluation attitude. Other barriers are a poor understanding of the evaluation process and its added value to optimizing the programme. Sufficient communication between involved professionals on evaluation can facilitate evaluation, as does support for evaluation of ICIAs together with stakeholders at a strategic and tactical level. To stimulate the evaluation of ICIAs, we recommend supporting professionals in securing evaluation resources, providing tailored training and tools to enhance evaluation competences and stimulating strategic communication on evaluation. PMID:27043600

Barriers to and Facilitators of the Evaluation of Integrated Community-Wide Overweight Intervention Approaches: A Qualitative Case Study in Two Dutch Municipalities.

PubMed

van Koperen, Tessa M; de Kruif, Anja; van Antwerpen, Lisa; Hendriks, Anna-Marie; Seidell, Jacob C; Schuit, Albertine J; Renders, Carry M

2016-03-31

To prevent overweight and obesity the implementation of an integrated community-wide intervention approach (ICIA) is often advocated. Evaluation can enhance implementation of such an approach and demonstrate the extent of effectiveness. To be able to support professionals in the evaluation of ICIAs we studied barriers to and facilitators of ICIA evaluation. In this study ten professionals of two Dutch municipalities involved in the evaluation of an ICIA participated. We conducted semi-structured interviews (n = 12), observed programme meetings (n = 4) and carried out document analysis. Data were analyzed using a thematic content approach. We learned that evaluation is hampered when it is perceived as unfeasible due to limited time and budget, a lack of evaluation knowledge or a negative evaluation attitude. Other barriers are a poor understanding of the evaluation process and its added value to optimizing the programme. Sufficient communication between involved professionals on evaluation can facilitate evaluation, as does support for evaluation of ICIAs together with stakeholders at a strategic and tactical level. To stimulate the evaluation of ICIAs, we recommend supporting professionals in securing evaluation resources, providing tailored training and tools to enhance evaluation competences and stimulating strategic communication on evaluation.

A Study of the Correlation of the Improvement of Teaching Evaluation Scores Based on Student Performance Grades

ERIC Educational Resources Information Center

Chen, Chi Yuan; Wang, Shu-Yin; Yang, Yi-Fang

2017-01-01

The purpose of the study is to explore the influence of teaching evaluations on teachers in that they might try to please their students by giving higher grades in order to get higher teaching evaluation scores. To achieve this purpose, the study analyzed the correlations between teaching evaluation scores, student's final grades and course fail…

An Interprofessional Program Evaluation Case Study: Utilizing Multiple Measures To Assess What Matters. AIR 1997 Annual Forum Paper.

ERIC Educational Resources Information Center

Delaney, Anne Marie

This paper reviews the first two years of a model program-evaluation case study which is intended to show: (1) how program evaluation can contribute to academic and professional degree programs; (2) how qualitative and quantitative techniques can be used to produce reliable measures for evaluation studies; and (3) how the role of the institutional…

Early economic evaluation of emerging health technologies: protocol of a systematic review

PubMed Central

2014-01-01

Background The concept of early health technology assessment, discussed well over a decade, has now been collaboratively implemented by industry, government, and academia to select and expedite the development of emerging technologies that may address the needs of patients and health systems. Early economic evaluation is essential to assess the value of emerging technologies, but empirical data to inform the current practice of early evaluation is limited. We propose a systematic review of early economic evaluation studies in order to better understand the current practice. Methods/design This protocol describes a systematic review of economic evaluation studies of regulated health technologies in which the evaluation is conducted prior to regulatory approval and when the technology effectiveness is not well established. Included studies must report an economic evaluation, defined as the comparative analysis of alternatives with respect to their associated costs and health consequences, and must evaluate some regulated health technology such as pharmaceuticals, biologics, high-risk medical devices, or biomarkers. We will conduct the literature search on multiple databases, including MEDLINE, EMBASE, the Centre for Reviews and Dissemination Databases, and EconLit. Additional citations will be identified via scanning reference lists and author searching. We suspect that many early economic evaluation studies are unpublished, especially those conducted for internal use only. Additionally, we use a chain-referral sampling approach to identify authors of unpublished studies who work in technology discovery and development, starting out with our contact lists and authors who published relevant studies. Citation screening and full-text review will be conducted by pairs of reviewers. Abstracted data will include those related to the decision context and decision problem of the early evaluation, evaluation methods (e.g., data sources, methods, and assumptions used to identify, measure, and value the likely effectiveness and the costs and consequences of the new technology, handling of uncertainty), and whether the study results adequately address the main study question or objective. Data will be summarized overall and stratified by publication status. Discussion This study is timely to inform early economic evaluation practice, given the international trend in early health technology assessment initiatives. PMID:25055987

Early economic evaluation of emerging health technologies: protocol of a systematic review.

PubMed

Pham, Ba'; Tu, Hong Anh Thi; Han, Dolly; Pechlivanoglou, Petros; Miller, Fiona; Rac, Valeria; Chin, Warren; Tricco, Andrea C; Paulden, Mike; Bielecki, Joanna; Krahn, Murray

2014-07-23

The concept of early health technology assessment, discussed well over a decade, has now been collaboratively implemented by industry, government, and academia to select and expedite the development of emerging technologies that may address the needs of patients and health systems. Early economic evaluation is essential to assess the value of emerging technologies, but empirical data to inform the current practice of early evaluation is limited. We propose a systematic review of early economic evaluation studies in order to better understand the current practice. This protocol describes a systematic review of economic evaluation studies of regulated health technologies in which the evaluation is conducted prior to regulatory approval and when the technology effectiveness is not well established. Included studies must report an economic evaluation, defined as the comparative analysis of alternatives with respect to their associated costs and health consequences, and must evaluate some regulated health technology such as pharmaceuticals, biologics, high-risk medical devices, or biomarkers. We will conduct the literature search on multiple databases, including MEDLINE, EMBASE, the Centre for Reviews and Dissemination Databases, and EconLit. Additional citations will be identified via scanning reference lists and author searching. We suspect that many early economic evaluation studies are unpublished, especially those conducted for internal use only. Additionally, we use a chain-referral sampling approach to identify authors of unpublished studies who work in technology discovery and development, starting out with our contact lists and authors who published relevant studies. Citation screening and full-text review will be conducted by pairs of reviewers. Abstracted data will include those related to the decision context and decision problem of the early evaluation, evaluation methods (e.g., data sources, methods, and assumptions used to identify, measure, and value the likely effectiveness and the costs and consequences of the new technology, handling of uncertainty), and whether the study results adequately address the main study question or objective. Data will be summarized overall and stratified by publication status. This study is timely to inform early economic evaluation practice, given the international trend in early health technology assessment initiatives.

Teacher Evaluation Models: Compliance or Growth Oriented?

ERIC Educational Resources Information Center

Clenchy, Kelly R.

2017-01-01

This research study reviewed literature specific to the evolution of teacher evaluation models and explored the effectiveness of standards-based evaluation models' potential to facilitate professional growth. The researcher employed descriptive phenomenology to conduct a study of teachers' perceptions of a standard-based evaluation model's…

Factors Underlying the Public's Credibility Perceptions in Educational Evaluations

ERIC Educational Resources Information Center

Jacobson, Miriam R.

2016-01-01

Understanding how evaluation audiences perceive credibility can help evaluators design evaluations and reports that support appropriate use. While researchers have studied credibility assessment among various educational stakeholders, little research has been conducted with the broader public. This study explored potential factors affecting the…

Do State Pre-K Programs Improve Children's Pre-Literacy and Math Learning? Evaluation Science Brief

ERIC Educational Resources Information Center

National Forum on Early Childhood Program Evaluation, 2008

2008-01-01

"Evaluation Science Briefs" summarize the findings and implications of a recent study evaluating the effects of an early childhood program or environment. This Brief evaluates the study "An Effectiveness-Based Evaluation of Five State Pre-Kindergarten Programs" (V. C. Wong, T. D. Cook, W. S. Barnett, and K. Jung.) States have high aspirations for…

Empirically evaluating decision-analytic models.

PubMed

Goldhaber-Fiebert, Jeremy D; Stout, Natasha K; Goldie, Sue J

2010-08-01

Model-based cost-effectiveness analyses support decision-making. To augment model credibility, evaluation via comparison to independent, empirical studies is recommended. We developed a structured reporting format for model evaluation and conducted a structured literature review to characterize current model evaluation recommendations and practices. As an illustration, we applied the reporting format to evaluate a microsimulation of human papillomavirus and cervical cancer. The model's outputs and uncertainty ranges were compared with multiple outcomes from a study of long-term progression from high-grade precancer (cervical intraepithelial neoplasia [CIN]) to cancer. Outcomes included 5 to 30-year cumulative cancer risk among women with and without appropriate CIN treatment. Consistency was measured by model ranges overlapping study confidence intervals. The structured reporting format included: matching baseline characteristics and follow-up, reporting model and study uncertainty, and stating metrics of consistency for model and study results. Structured searches yielded 2963 articles with 67 meeting inclusion criteria and found variation in how current model evaluations are reported. Evaluation of the cervical cancer microsimulation, reported using the proposed format, showed a modeled cumulative risk of invasive cancer for inadequately treated women of 39.6% (30.9-49.7) at 30 years, compared with the study: 37.5% (28.4-48.3). For appropriately treated women, modeled risks were 1.0% (0.7-1.3) at 30 years, study: 1.5% (0.4-3.3). To support external and projective validity, cost-effectiveness models should be iteratively evaluated as new studies become available, with reporting standardized to facilitate assessment. Such evaluations are particularly relevant for models used to conduct comparative effectiveness analyses.

Student Self-evaluation After Nursing Examinations: That's a Wrap.

PubMed

Butzlaff, Alice; Gaylle, Debrayh; O'Leary Kelley, Colleen

2018-04-13

Examination wrappers are a self-evaluation tool that uses metacognition to help students reflect on test performance. After examinations, rather than focus on points earned, students learn to self-identify study strategies and recognize methods of test preparation. The purpose of the study was to determine if the use of an examination wrapper after each test would encourage students to self-evaluate performance and adjust study strategies. A total of 120 undergraduate nursing students completed self-evaluations after each examination, which were analyzed using content analysis. Three general patterns emerged from student self-evaluation: effective and ineffective study strategies, understanding versus memorization of content, and nurse educator assistance.

Nuclear density evaluation on asphaltic concrete: final report.

DOT National Transportation Integrated Search

1967-04-01

This study was one of two studies designated as Research Project 62-1SB to evaluate the use of nuclear devices in highway construction. (1) The primary objective of this study was to evaluate a nuclear density device for obtaining densities on asphal...

An Evaluation System for the Online Training Programs in Meteorology and Hydrology

ERIC Educational Resources Information Center

Wang, Yong; Zhi, Xiefei

2009-01-01

This paper studies the current evaluation system for the online training program in meteorology and hydrology. CIPP model that includes context evaluation, input evaluation, process evaluation and product evaluation differs from Kirkpatrick model including reactions evaluation, learning evaluation, transfer evaluation and results evaluation in…

A Collection of Papers on Self-Study and Institutional Improvement, 2003. Volume 4: The Self-Study Process for Commission Evaluation. 2003 Edition.

ERIC Educational Resources Information Center

Van Kollenburg, Susan E., Ed.

Papers in this collection were prepared for the annual meeting of the North Central Association of Colleges and Schools. This volume contains papers related to the self-study process for Commission evaluation. Papers in chapters 1, "Self-Study and Commission Evaluation: Coordinating the Self-Study," are: (1) "Initiating the…

Evaluating Creative Thinking of Rn-Bsn Students in the Course of Clinical Case Study and Practicum

ERIC Educational Resources Information Center

Ku, Ya-Lie

2015-01-01

This case study evaluated creative thinking of RN-BSN students in the course of clinical case study and practicum. Study design used quantitative and qualitative evaluations of creative thinking of RN-BSN students by triangulation method in the course of clinical case study and practicum. Sixty RN-BSN students self-perceived the changing levels of…

Issues in evaluation: evaluating assessments of elderly people using a combination of methods.

PubMed

McEwan, R T

1989-02-01

In evaluating a health service, individuals will give differing accounts of its performance, according to their experiences of the service, and the evaluative perspective they adopt. The value of a service may also change through time, and according to the particular part of the service studied. Traditional health care evaluations have generally not accounted for this variability because of the approaches used. Studies evaluating screening or assessment programmes for the elderly have focused on programme effectiveness and efficiency, using relatively inflexible quantitative methods. Evaluative approaches must reflect the complexity of health service provision, and methods must vary to suit the particular research objective. Under these circumstances, this paper presents the case for the use of multiple triangulation in evaluative research, where differing methods and perspectives are combined in one study. Emphasis is placed on the applications and benefits of subjectivist approaches in evaluation. An example of combined methods is provided in the form of an evaluation of the Newcastle Care Plan for the Elderly.

Status and the evaluation of workplace deviance.

PubMed

Bowles, Hannah Riley; Gelfand, Michele

2010-01-01

Bias in the evaluation of workplace misbehavior is hotly debated in courts and corporations, but it has received little empirical attention. Classic sociological literature suggests that deviance by lower-status actors will be evaluated more harshly than deviance by higher-status actors. However, more recent psychological literature suggests that discrimination in the evaluation of misbehavior may be moderated by the relative status of the evaluator because status influences both rule observance and attitudes toward social hierarchy. In Study 1, the psychological experience of higher status decreased rule observance and increased preferences for social hierarchy, as we theorized. In three subsequent experiments, we tested the hypothesis that higher-status evaluators would be more discriminating in their evaluations of workplace misbehavior, evaluating fellow higher-status deviants more leniently than lower-status deviants. Results supported the hypothesized interactive effect of evaluator status and target status on the evaluation of workplace deviance, when both achieved status characteristics (Studies 2a and 2b) and ascribed status characteristics (i.e., race and gender in Study 3) were manipulated.

[Self-acceptance as adaptively resigning the self to low self-evaluation].

PubMed

Ueda, T

1996-10-01

In past studies, the concept of self-acceptance has often been confused with self-evaluation or self-esteem. The purpose of this study was to distinguish these concepts, and operationally define self-acceptance as Carl Rogers proposed: feeling all right toward the self when self-evaluation was low. Self-acceptance as adaptive resignation, a moderating variable, therefore should raise self-esteem of only those people with low self-evaluation. Self-acceptance was measured in the study as affirmative evaluation of own self-evaluation. Two hundred and forty college students, 120 each for men and women, completed a questionnaire of self-evaluative consciousness and self-esteem scales. Results of statistical analyses showed that among subjects with low self-evaluation, the higher self-acceptance, the higher the person's self-esteem. The same relation was not observed among those with high self-evaluation. Thus, it may be concluded that self-acceptance was adaptive resignation, and therefore meaningful to only those with low self-evaluation.

Principals' Experiences of Being Evaluated: A Phenomenological Study

ERIC Educational Resources Information Center

Parylo, Oksana; Zepeda, Sally J.; Bengtson, Ed

2012-01-01

This phenomenological study sought to understand principals' lived experiences of being evaluated with reliance on the principles of developmental supervision and adult learning theory. Analysis of interview data from 16 principals revealed 3 major constructs in principal evaluation: evaluation is a complex, constantly changing system; principal…

Economic analysis of the health impacts of housing improvement studies: a systematic review

PubMed Central

Fenwick, Elisabeth; Macdonald, Catriona; Thomson, Hilary

2013-01-01

Background Economic evaluation of public policies has been advocated but rarely performed. Studies from a systematic review of the health impacts of housing improvement included data on costs and some economic analysis. Examination of these data provides an opportunity to explore the difficulties and the potential for economic evaluation of housing. Methods Data were extracted from all studies included in the systematic review of housing improvement which had reported costs and economic analysis (n=29/45). The reported data were assessed for their suitability to economic evaluation. Where an economic analysis was reported the analysis was described according to pre-set definitions of various types of economic analysis used in the field of health economics. Results 25 studies reported cost data on the intervention and/or benefits to the recipients. Of these, 11 studies reported data which was considered amenable to economic evaluation. A further four studies reported conducting an economic evaluation. Three of these studies presented a hybrid ‘balance sheet’ approach and indicated a net economic benefit associated with the intervention. One cost-effectiveness evaluation was identified but the data were unclearly reported; the cost-effectiveness plane suggested that the intervention was more costly and less effective than the status quo. Conclusions Future studies planning an economic evaluation need to (i) make best use of available data and (ii) ensure that all relevant data are collected. To facilitate this, economic evaluations should be planned alongside the intervention with input from health economists from the outset of the study. When undertaken appropriately, economic evaluation provides the potential to make significant contributions to housing policy. PMID:23929616

Review of health information technology usability study methodologies

PubMed Central

Bakken, Suzanne

2011-01-01

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user–task interaction, n=42; stage 2, system–task interaction, n=310; stage 3, user–task–system interaction, n=69; stage 4, user–task–system–environment interaction, n=54; and stage 5, user–task–system–environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation. PMID:21828224

What We Do and Do Not Know about Teaching Medical Image Interpretation.

PubMed

Kok, Ellen M; van Geel, Koos; van Merriënboer, Jeroen J G; Robben, Simon G F

2017-01-01

Educators in medical image interpretation have difficulty finding scientific evidence as to how they should design their instruction. We review and comment on 81 papers that investigated instructional design in medical image interpretation. We distinguish between studies that evaluated complete offline courses and curricula, studies that evaluated e-learning modules, and studies that evaluated specific educational interventions. Twenty-three percent of all studies evaluated the implementation of complete courses or curricula, and 44% of the studies evaluated the implementation of e-learning modules. We argue that these studies have encouraging results but provide little information for educators: too many differences exist between conditions to unambiguously attribute the learning effects to specific instructional techniques. Moreover, concepts are not uniformly defined and methodological weaknesses further limit the usefulness of evidence provided by these studies. Thirty-two percent of the studies evaluated a specific interventional technique. We discuss three theoretical frameworks that informed these studies: diagnostic reasoning, cognitive schemas and study strategies. Research on diagnostic reasoning suggests teaching students to start with non-analytic reasoning and subsequently applying analytic reasoning, but little is known on how to train non-analytic reasoning. Research on cognitive schemas investigated activities that help the development of appropriate cognitive schemas. Finally, research on study strategies supports the effectiveness of practice testing, but more study strategies could be applicable to learning medical image interpretation. Our commentary highlights the value of evaluating specific instructional techniques, but further evidence is required to optimally inform educators in medical image interpretation.

[The methods within the evaluation of disease management programmes in control-group designs using the example of diabetes mellitus - a systematic literature review].

PubMed

Drabik, A; Sawicki, P T; Müller, D; Passon, A; Stock, S

2012-08-01

Disease management programmes (DMPs) were implemented in Germany in 2002. Their evaluation is required by law. Beyond the mandatory evaluation, a growing number of published studies evaluate the DMP for diabetes mellitus type 2 in a control-group design. As patients opt into the programme on a voluntary basis it is necessary to adjust the inherent selection bias between groups. The aim of this study is to review published studies which evaluate the diabetes DMP using a control-group design with respect to the methods used. A systematic literature review of electronic databases (PUBMED, Cochrane Library, EMBASE, MEDPILOT) and a hand search of reference lists of the relevant publications was conducted to identify studies evaluating the DMP diabetes mellitus in a control-group design. 8 studies were included in the systematic literature review. 4 studies gathered retrospective claims data from sickness funds, one from physician's records, one study used prospective data from ambulatory care, and 2 studies were based on one patient survey. Methods used for adjustment of selection bias included exact matching, matching using propensity score methods, age-adjusted and sex-separated analysis, and adjustment in a regression model/analysis of covariance. One study did not apply adjustment methods. The intervention period ranged from 1 day to 4 years. Considered outcomes of studies (surrogate parameter, diabetes complications, mortality, quality of life, and claim data) depended on the database. In the evaluation of the DMP diabetes mellitus based on a control-group design neither the database nor the methods used for selection bias adjustment were consistent in the available studies. Effectiveness of DMPs cannot be judged based on this review due to heterogeneity of study designs. To allow for a comprehensive programme evaluation standardised minimum requirements for the evaluation of DMPs in the control group design are required. © Georg Thieme Verlag KG Stuttgart · New York.

A Tentative Study on the Evaluation of Community Health Service Quality*

NASA Astrophysics Data System (ADS)

Ma, Zhi-qiang; Zhu, Yong-yue

Community health service is the key point of health reform in China. Based on pertinent studies, this paper constructed an indicator system for the community health service quality evaluation from such five perspectives as visible image, reliability, responsiveness, assurance and sympathy, according to service quality evaluation scale designed by Parasuraman, Zeithaml and Berry. A multilevel fuzzy synthetical evaluation model was constructed to evaluate community health service by fuzzy mathematics theory. The applicability and maneuverability of the evaluation indicator system and evaluation model were verified by empirical analysis.

The evaluability bias in charitable giving: Saving administration costs or saving lives?

PubMed Central

Caviola, Lucius; Faulmüller, Nadira; Everett, Jim. A. C.; Savulescu, Julian; Kahane, Guy

2014-01-01

We describe the “evaluability bias”: the tendency to weight the importance of an attribute in proportion to its ease of evaluation. We propose that the evaluability bias influences decision making in the context of charitable giving: people tend to have a strong preference for charities with low overhead ratios (lower administrative expenses) but not for charities with high cost-effectiveness (greater number of saved lives per dollar), because the former attribute is easier to evaluate than the latter. In line with this hypothesis, we report the results of four studies showing that, when presented with a single charity, people are willing to donate more to a charity with low overhead ratio, regardless of cost-effectiveness. However, when people are presented with two charities simultaneously—thereby enabling comparative evaluation—they base their donation behavior on cost-effectiveness (Study 1). This suggests that people primarily value cost-effectiveness but manifest the evaluability bias in cases where they find it difficult to evaluate. However, people seem also to value a low overhead ratio for its own sake (Study 2). The evaluability bias effect applies to charities of different domains (Study 3). We also show that overhead ratio is easier to evaluate when its presentation format is a ratio, suggesting an inherent reference point that allows meaningful interpretation (Study 4). PMID:25279024

Veterinary and human vaccine evaluation methods

PubMed Central

Knight-Jones, T. J. D.; Edmond, K.; Gubbins, S.; Paton, D. J.

2014-01-01

Despite the universal importance of vaccines, approaches to human and veterinary vaccine evaluation differ markedly. For human vaccines, vaccine efficacy is the proportion of vaccinated individuals protected by the vaccine against a defined outcome under ideal conditions, whereas for veterinary vaccines the term is used for a range of measures of vaccine protection. The evaluation of vaccine effectiveness, vaccine protection assessed under routine programme conditions, is largely limited to human vaccines. Challenge studies under controlled conditions and sero-conversion studies are widely used when evaluating veterinary vaccines, whereas human vaccines are generally evaluated in terms of protection against natural challenge assessed in trials or post-marketing observational studies. Although challenge studies provide a standardized platform on which to compare different vaccines, they do not capture the variation that occurs under field conditions. Field studies of vaccine effectiveness are needed to assess the performance of a vaccination programme. However, if vaccination is performed without central co-ordination, as is often the case for veterinary vaccines, evaluation will be limited. This paper reviews approaches to veterinary vaccine evaluation in comparison to evaluation methods used for human vaccines. Foot-and-mouth disease has been used to illustrate the veterinary approach. Recommendations are made for standardization of terminology and for rigorous evaluation of veterinary vaccines. PMID:24741009

Economic evaluations in gastroenterology in Brazil: A systematic review.

PubMed

de Paiva Haddad, Luciana Bertocco; Decimoni, Tassia Cristina; Turri, Jose Antonio; Leandro, Roseli; de Soárez, Patrícia Coelho

2016-02-06

To systematically review economic evaluations in gastroenterology, relating to Brazil, published between 1980 and 2013. We selected full and partial economic evaluations from among those retrieved by searching the following databases: MEDLINE (PubMed); Excerpta Medica; the Latin American and Caribbean Health Sciences Literature database; the Scientific Electronic Library Online; the database of the Centre for Reviews and Dissemination; the National Health Service (NHS) Economic Evaluation Database; the NHS Health Technology Assessment database; the Health Economics database of the Brazilian Virtual Library of Health; Scopus; Web of Science; and the Brazilian Network for the Evaluation of Health Technologies. Two researchers, working independently, selected the studies and extracted the data. We identified 535 health economic evaluations relating to Brazil and published in the 1980-2013 period. Of those 535 articles, only 40 dealt with gastroenterology. Full and partial economic evaluations respectively accounted for 23 (57.5%) and 17 (42.5%) of the 40 studies included. Among the 23 full economic evaluations, there were 11 cost-utility analyses, seven cost-effectiveness analyses, four cost-consequence analyses, and one cost-minimization analysis. Of the 40 studies, 25 (62.5%) evaluated medications; 7 (17.5%) evaluated procedures; and 3 (7.5%) evaluated equipment. Most (55%) of the studies were related to viral hepatitis, and most (63.4%) were published after 2010. Other topics included gastrointestinal cancer, liver transplantation, digestive diseases and hernias. Over the 33-year period examined, the number of such economic evaluations relating to Brazil, especially of those evaluating medications for the treatment of hepatitis, increased considerably. Further studies are needed in order to ensure that expenditures on health care in Brazil are made as fairly and efficiently as possible.

Economic evaluations in gastroenterology in Brazil: A systematic review

PubMed Central

de Paiva Haddad, Luciana Bertocco; Decimoni, Tassia Cristina; Turri, Jose Antonio; Leandro, Roseli; de Soárez, Patrícia Coelho

2016-01-01

AIM: To systematically review economic evaluations in gastroenterology, relating to Brazil, published between 1980 and 2013. METHODS: We selected full and partial economic evaluations from among those retrieved by searching the following databases: MEDLINE (PubMed); Excerpta Medica; the Latin American and Caribbean Health Sciences Literature database; the Scientific Electronic Library Online; the database of the Centre for Reviews and Dissemination; the National Health Service (NHS) Economic Evaluation Database; the NHS Health Technology Assessment database; the Health Economics database of the Brazilian Virtual Library of Health; Scopus; Web of Science; and the Brazilian Network for the Evaluation of Health Technologies. Two researchers, working independently, selected the studies and extracted the data. RESULTS: We identified 535 health economic evaluations relating to Brazil and published in the 1980-2013 period. Of those 535 articles, only 40 dealt with gastroenterology. Full and partial economic evaluations respectively accounted for 23 (57.5%) and 17 (42.5%) of the 40 studies included. Among the 23 full economic evaluations, there were 11 cost-utility analyses, seven cost-effectiveness analyses, four cost-consequence analyses, and one cost-minimization analysis. Of the 40 studies, 25 (62.5%) evaluated medications; 7 (17.5%) evaluated procedures; and 3 (7.5%) evaluated equipment. Most (55%) of the studies were related to viral hepatitis, and most (63.4%) were published after 2010. Other topics included gastrointestinal cancer, liver transplantation, digestive diseases and hernias. Over the 33-year period examined, the number of such economic evaluations relating to Brazil, especially of those evaluating medications for the treatment of hepatitis, increased considerably. CONCLUSION: Further studies are needed in order to ensure that expenditures on health care in Brazil are made as fairly and efficiently as possible. PMID:26855823

78 FR 41926 - Proposed Information Collection Request; Comment Request; Performance Evaluation Studies on...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-07-12

... ENVIRONMENTAL PROTECTION AGENCY [EPA-HQ-OECA-2013-0413; FRL--9832-8] Proposed Information Collection Request; Comment Request; Performance Evaluation Studies on Wastewater Laboratories (Renewal); EPA... request (ICR), ``Performance Evaluation Studies on Wastewater Laboratories'' (EPA ICR No. 0234.11, OMB...

Economic Evaluation of Obesity Prevention in Early Childhood: Methods, Limitations and Recommendations.

PubMed

Döring, Nora; Mayer, Susanne; Rasmussen, Finn; Sonntag, Diana

2016-09-13

Despite methodological advances in the field of economic evaluations of interventions, economic evaluations of obesity prevention programmes in early childhood are seldom conducted. The aim of the present study was to explore existing methods and applications of economic evaluations, examining their limitations and making recommendations for future cost-effectiveness assessments. A systematic literature search was conducted using PubMed, Cochrane Library, the British National Health Service Economic Evaluation Databases and EconLit. Eligible studies included trial-based or simulation-based cost-effectiveness analyses of obesity prevention programmes targeting preschool children and/or their parents. The quality of included studies was assessed. Of the six studies included, five were intervention studies and one was based on a simulation approach conducted on secondary data. We identified three main conceptual and methodological limitations of their economic evaluations: Insufficient conceptual approach considering the complexity of childhood obesity, inadequate measurement of effects of interventions, and lack of valid instruments to measure child-related quality of life and costs. Despite the need for economic evaluations of obesity prevention programmes in early childhood, only a few studies of varying quality have been conducted. Moreover, due to methodological and conceptual weaknesses, they offer only limited information for policy makers and intervention providers. We elaborate reasons for the limitations of these studies and offer guidance for designing better economic evaluations of early obesity prevention.

Economic Evaluation of Obesity Prevention in Early Childhood: Methods, Limitations and Recommendations

PubMed Central

Döring, Nora; Mayer, Susanne; Rasmussen, Finn; Sonntag, Diana

2016-01-01

Despite methodological advances in the field of economic evaluations of interventions, economic evaluations of obesity prevention programmes in early childhood are seldom conducted. The aim of the present study was to explore existing methods and applications of economic evaluations, examining their limitations and making recommendations for future cost-effectiveness assessments. A systematic literature search was conducted using PubMed, Cochrane Library, the British National Health Service Economic Evaluation Databases and EconLit. Eligible studies included trial-based or simulation-based cost-effectiveness analyses of obesity prevention programmes targeting preschool children and/or their parents. The quality of included studies was assessed. Of the six studies included, five were intervention studies and one was based on a simulation approach conducted on secondary data. We identified three main conceptual and methodological limitations of their economic evaluations: Insufficient conceptual approach considering the complexity of childhood obesity, inadequate measurement of effects of interventions, and lack of valid instruments to measure child-related quality of life and costs. Despite the need for economic evaluations of obesity prevention programmes in early childhood, only a few studies of varying quality have been conducted. Moreover, due to methodological and conceptual weaknesses, they offer only limited information for policy makers and intervention providers. We elaborate reasons for the limitations of these studies and offer guidance for designing better economic evaluations of early obesity prevention. PMID:27649218

Understanding of evaluation capacity building in practice: a case study of a national medical education organization.

PubMed

Sarti, Aimee J; Sutherland, Stephanie; Landriault, Angele; DesRosier, Kirk; Brien, Susan; Cardinal, Pierre

2017-01-01

Evaluation capacity building (ECB) is a topic of great interest to many organizations as they face increasing demands for accountability and evidence-based practices. ECB is about building the knowledge, skills, and attitudes of organizational members, the sustainability of rigorous evaluative practices, and providing the resources and motivations to engage in ongoing evaluative work. There exists a solid foundation of theoretical research on ECB, however, understanding what ECB looks like in practice is relatively thin. Our purpose was to investigate what ECB looks like firsthand within a national medical educational organization. The context for this study was the Acute Critical Events Simulation (ACES) organization in Canada, which has successfully evolved into a national educational program, driven by physicians. We conducted an exploratory qualitative study to better understand and describe ECB in practice. In doing so, interviews were conducted with program leaders and instructors so as to gain a richer understanding of evaluative processes and practices. A total of 21 individuals participated in the semistructured interviews. Themes from our qualitative data analysis included the following: evaluation knowledge, skills, and attitudes, use of evaluation findings, shared evaluation beliefs and commitment, evaluation frameworks and processes, and resources dedicated to evaluation. The national ACES organization was a useful case study to explore ECB in practice. The ECB literature provided a solid foundation to understand the purpose and nuances of ECB. This study added to the paucity of studies focused on examining ECB in practice. The most important lesson learned was that the organization must have leadership who are intrinsically motivated to employ and use evaluation data to drive ongoing improvements within the organization. Leaders who are intrinsically motivated will employ risk taking when evaluation practices and processes may be somewhat unfamiliar. Creating and maintaining a culture of data use and ongoing inquiry have enabled national ACES to achieve a sustainable evaluation practice.

Addressing Enterprise-Level Information System Deficiencies

DTIC Science & Technology

2015-03-26

goals and effectiveness. This case study evaluates the Depot Source of Repair (DSOR) team and how it has addressed the USAF’s enterprise- level IS...deficiencies. A framework created from the literature review is used to evaluate the DSOR team’s IS called DSOR II. The case study evaluation ...7 IS Design Evaluation

Teacher Evaluation Reform: Focus, Feedback, and Fear

ERIC Educational Resources Information Center

Donaldson, Morgaen L.

2016-01-01

How are teachers experiencing the more rigorous teacher evaluation systems that many states have mandated in recent years? Donaldson, who has studied teacher evaluation reform over the past eight years, shares insights from a study of 14 Connecticut districts that have implemented the state's 2012 teacher evaluation reforms. In surveys and…

A Database Evaluation Based on Information Needs of Academic Social Scientists.

ERIC Educational Resources Information Center

Buterbaugh, Nancy Toth

This study evaluates two databases, "Historical Abstracts" and REESWeb, to determine their effectiveness in supporting academic social science research. While many performance evaluations gather quantitative data from isolated query and response transactions, this study is a qualitative evaluation of the databases in the context of…

Evaluation and Communication: Using a Communication Audit to Evaluate Organizational Communication

ERIC Educational Resources Information Center

Hogard, Elaine; Ellis, Roger

2006-01-01

This article identifies a surprising dearth of studies that explicitly link communication and evaluation at substantive, theoretical, and methodological levels. A three-fold typology of evaluation studies referring to communication is proposed and examples given. The importance of organizational communication in program delivery is stressed and…

Inside the Black Box--An Implementation Evaluation Case Study

ERIC Educational Resources Information Center

Rector, Patricia; Bakacs, Michele; Rowe, Amy; Barbour, Bruce

2016-01-01

The case study presented in this article is an example of an implementation evaluation. The evaluation investigated significant components of the implementation of a long-term environmental educational program. Direct observation, evaluation-specific survey data, and historical data were used to determine program integrity as identified by…

Research Governance and the Role of Evaluation: A Comparative Study

ERIC Educational Resources Information Center

Molas-Gallart, Jordi

2012-01-01

Through a comparative study of the United Kingdom and Spain, this article addresses the effect of different research governance structures on the functioning and uses of research evaluation. It distinguishes three main evaluation uses: distributive, improvement, and controlling. Research evaluation in the United Kingdom plays important…

Practical Considerations in Evaluating Patient/Consumer Health Education Programs.

ERIC Educational Resources Information Center

Bryant, Nancy H.

This report contains brief descriptions of seven evaluative efforts and outcomes of health education programs, some considerations of problems encountered in evaluating the programs, and detailed descriptions of two case studies: (1) a process evaluation of preoperative teaching and (2) a retrospective study of visiting nurse association use by…

The Effect of Evaluation on Performance: Evidence from Longitudinal Student Achievement Data of Mid-Career Teachers. NBER Working Paper No. 16877

ERIC Educational Resources Information Center

Taylor, Eric S.; Tyler, John H.

2011-01-01

The effect of evaluation on employee performance is traditionally studied in the context of the principal-agent problem. Evaluation can, however, also be characterized as an investment in the evaluated employee's human capital. We study a sample of mid-career public school teachers where we can consider these two types of evaluation effect…

A Comparison of the Autism Treatment Evaluation Checklist (ATEC) and the Childhood Autism Rating Scale (CARS) for the Quantitative Evaluation of Autism

ERIC Educational Resources Information Center

Geier, David A.; Kern, Janet K.; Geier, Mark R.

2013-01-01

The purpose of this study was to evaluate scores generated from the Autism Treatment Evaluation Checklist (ATEC), a parent-rated measure, and those derived from professionally completed Childhood Autism Rating Scale (CARS) evaluations. A cohort of 56 participants diagnosed with an autism spectrum disorder was used for the study, and each child was…

The scope of costs in alcohol studies: Cost-of-illness studies differ from economic evaluations.

PubMed

van Gils, Paul F; Hamberg-van Reenen, Heleen H; van den Berg, Matthijs; Tariq, Luqman; de Wit, G Ardine

2010-07-06

Alcohol abuse results in problems on various levels in society. In terms of health, alcohol abuse is not only an important risk factor for chronic disease, but it is also related to injuries. Social harms which can be related to drinking include interpersonal problems, work problems, violent and other crimes. The scope of societal costs related to alcohol abuse in principle should be the same for both economic evaluations and cost-of-illness studies. In general, economic evaluations report a small part of all societal costs. To determine the cost- effectiveness of an intervention it is necessary that all costs and benefits are included. The purpose of this study is to describe and quantify the difference in societal costs incorporated in economic evaluations and cost-of-illness studies on alcohol abuse. To investigate the economic costs attributable to alcohol in cost-of-illness studies we used the results of a recent systematic review (June 2009). We performed a PubMed search to identify economic evaluations on alcohol interventions. Only economic evaluations in which two or more interventions were compared from a societal perspective were included. The proportion of health care costs and the proportion of societal costs were estimated in both type of studies. The proportion of healthcare costs in cost-of-illness studies was 17% and the proportion of societal costs 83%. In economic evaluations, the proportion of healthcare costs was 57%, and the proportion of societal costs was 43%. The costs included in economic evaluations performed from a societal perspective do not correspond with those included in cost-of-illness studies. Economic evaluations on alcohol abuse underreport true societal cost of alcohol abuse. When considering implementation of alcohol abuse interventions, policy makers should take into account that economic evaluations from the societal perspective might underestimate the total effects and costs of interventions.

Automatic Evaluations and Exercising: Systematic Review and Implications for Future Research.

PubMed

Schinkoeth, Michaela; Antoniewicz, Franziska

2017-01-01

The general purpose of this systematic review was to summarize, structure and evaluate the findings on automatic evaluations of exercising. Studies were eligible for inclusion if they reported measuring automatic evaluations of exercising with an implicit measure and assessed some kind of exercise variable. Fourteen nonexperimental and six experimental studies (out of a total N = 1,928) were identified and rated by two independent reviewers. The main study characteristics were extracted and the grade of evidence for each study evaluated. First, results revealed a large heterogeneity in the applied measures to assess automatic evaluations of exercising and the exercise variables. Generally, small to large-sized significant relations between automatic evaluations of exercising and exercise variables were identified in the vast majority of studies. The review offers a systematization of the various examined exercise variables and prompts to differentiate more carefully between actually observed exercise behavior (proximal exercise indicator) and associated physiological or psychological variables (distal exercise indicator). Second, a lack of transparent reported reflections on the differing theoretical basis leading to the use of specific implicit measures was observed. Implicit measures should be applied purposefully, taking into consideration the individual advantages or disadvantages of the measures. Third, 12 studies were rated as providing first-grade evidence (lowest grade of evidence), five represent second-grade and three were rated as third-grade evidence. There is a dramatic lack of experimental studies, which are essential for illustrating the cause-effect relation between automatic evaluations of exercising and exercise and investigating under which conditions automatic evaluations of exercising influence behavior. Conclusions about the necessity of exercise interventions targeted at the alteration of automatic evaluations of exercising should therefore not be drawn too hastily.

Automatic Evaluations and Exercising: Systematic Review and Implications for Future Research

PubMed Central

Schinkoeth, Michaela; Antoniewicz, Franziska

2017-01-01

The general purpose of this systematic review was to summarize, structure and evaluate the findings on automatic evaluations of exercising. Studies were eligible for inclusion if they reported measuring automatic evaluations of exercising with an implicit measure and assessed some kind of exercise variable. Fourteen nonexperimental and six experimental studies (out of a total N = 1,928) were identified and rated by two independent reviewers. The main study characteristics were extracted and the grade of evidence for each study evaluated. First, results revealed a large heterogeneity in the applied measures to assess automatic evaluations of exercising and the exercise variables. Generally, small to large-sized significant relations between automatic evaluations of exercising and exercise variables were identified in the vast majority of studies. The review offers a systematization of the various examined exercise variables and prompts to differentiate more carefully between actually observed exercise behavior (proximal exercise indicator) and associated physiological or psychological variables (distal exercise indicator). Second, a lack of transparent reported reflections on the differing theoretical basis leading to the use of specific implicit measures was observed. Implicit measures should be applied purposefully, taking into consideration the individual advantages or disadvantages of the measures. Third, 12 studies were rated as providing first-grade evidence (lowest grade of evidence), five represent second-grade and three were rated as third-grade evidence. There is a dramatic lack of experimental studies, which are essential for illustrating the cause-effect relation between automatic evaluations of exercising and exercise and investigating under which conditions automatic evaluations of exercising influence behavior. Conclusions about the necessity of exercise interventions targeted at the alteration of automatic evaluations of exercising should therefore not be drawn too hastily. PMID:29250022

Evaluating For-Profit Higher Education: Evidence from the Education Longitudinal Study. A CAPSEE Working Paper

ERIC Educational Resources Information Center

Liu, Yuen Ting; Belfield, Clive

2014-01-01

This study evaluates the postsecondary and labor market outcomes of students who attended for-profit colleges. The evaluation complements a similar study by Deming, Goldin, and Katz (2012) that found significant differences in outcomes between students in for-profit colleges and those in other sectors. In this study we use the Education…

Study design considerations in evaluating environmental impacts

Treesearch

Stan T. Lebow; Paul A. Cooper; Patricia Lebow

2006-01-01

The purpose of this chapter is to make the reader aware of how choices in study parameters may influence the outcome of treated-wood environmental impact evaluations. Evaluation of the leaching and environmental accumulation of preservatives from treated wood is a complex process. and many factors can influence the results of such studies. In laboratory studies, the...

Evaluation and Assessment Processes in Nebraska Public Schools. A Staff Report. Education Interim Study LR 181.

ERIC Educational Resources Information Center

Nebraska Legislative Council, Lincoln. Legislative Research Div.

During the spring of 1987, the National Conference of State Legislatures awarded the Nebraska Legislature a cost-sharing award to study local school evaluation processes. Embodied in Legislative Resolution 181, which has the purpose of studying school evaluation procedures, the study attempts to provide legislators, educators, local school…

Design and Implementation of a Multi-Strategy, Collegewide Program of Evaluation and Planning: The Mercy College Self-Study Project.

ERIC Educational Resources Information Center

Kraetzer, Mary C.; And Others

The rationale, strategies, and methods of The Mercy College Self-Study Project are considered, and evaluation instruments are provided. This program of institutional evaluation and planning was initiated in 1980 and consists of: standardized surveys, a 10-year longitudinal (panel) study, and academic department self-studies. Questionnaires…

Evaluation Studies of Robotic Rollators by the User Perspective: A Systematic Review.

PubMed

Werner, Christian; Ullrich, Phoebe; Geravand, Milad; Peer, Angelika; Hauer, Klaus

2016-01-01

Robotic rollators enhance the basic functions of established devices by technically advanced physical, cognitive, or sensory support to increase autonomy in persons with severe impairment. In the evaluation of such ambient assisted living solutions, both the technical and user perspectives are important to prove usability, effectiveness and safety, and to ensure adequate device application. The aim of this systematic review is to summarize the methodology of studies evaluating robotic rollators with focus on the user perspective and to give recommendations for future evaluation studies. A systematic literature search up to December 31, 2014, was conducted based on the Cochrane Review methodology using the electronic databases PubMed and IEEE Xplore. Articles were selected according to the following inclusion criteria: evaluation studies of robotic rollators documenting human-robot interaction, no case reports, published in English language. Twenty-eight studies were identified that met the predefined inclusion criteria. Large heterogeneity in the definitions of the target user group, study populations, study designs and assessment methods was found across the included studies. No generic methodology to evaluate robotic rollators could be identified. We found major methodological shortcomings related to insufficient sample descriptions and sample sizes, and lack of appropriate, standardized and validated assessment methods. Long-term use in habitual environment was also not evaluated. Apart from the heterogeneity, methodological deficits in most of the identified studies became apparent. Recommendations for future evaluation studies include: clear definition of target user group, adequate selection of subjects, inclusion of other assistive mobility devices for comparison, evaluation of the habitual use of advanced prototypes, adequate assessment strategy with established, standardized and validated methods, and statistical analysis of study results. Assessment strategies may additionally focus on specific functionalities of the robotic rollators allowing an individually tailored assessment of innovative features to document their added value. © 2016 S. Karger AG, Basel.

A systematic review of economic evaluations of population-based sodium reduction interventions

PubMed Central

Hope, Silvia F.; Webster, Jacqui; Trieu, Kathy; Pillay, Arti; Ieremia, Merina; Bell, Colin; Snowdon, Wendy; Neal, Bruce; Moodie, Marj

2017-01-01

Objective To summarise evidence describing the cost-effectiveness of population-based interventions targeting sodium reduction. Methods A systematic search of published and grey literature databases and websites was conducted using specified key words. Characteristics of identified economic evaluations were recorded, and included studies were appraised for reporting quality using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Results Twenty studies met the study inclusion criteria and received a full paper review. Fourteen studies were identified as full economic evaluations in that they included both costs and benefits associated with an intervention measured against a comparator. Most studies were modelling exercises based on scenarios for achieving salt reduction and assumed effects on health outcomes. All 14 studies concluded that their specified intervention(s) targeting reductions in population sodium consumption were cost-effective, and in the majority of cases, were cost saving. Just over half the studies (8/14) were assessed as being of ‘excellent’ reporting quality, five studies fell into the ‘very good’ quality category and one into the ‘good’ category. All of the identified evaluations were based on modelling, whereby inputs for all the key parameters including the effect size were either drawn from published datasets, existing literature or based on expert advice. Conclusion Despite a clear increase in evaluations of salt reduction programs in recent years, this review identified relatively few economic evaluations of population salt reduction interventions. None of the studies were based on actual implementation of intervention(s) and the associated collection of new empirical data. The studies universally showed that population-based salt reduction strategies are likely to be cost effective or cost saving. However, given the reliance on modelling, there is a need for the effectiveness of new interventions to be evaluated in the field using strong study designs and parallel economic evaluations. PMID:28355231

Women's Studies Collections: A Checklist Evaluation

ERIC Educational Resources Information Center

Bolton, Brooke A.

2009-01-01

A checklist evaluation on thirty-seven Women's Studies programs conducted using the individual institutions' online public access catalogs (OPACs) is presented. Although Women's Studies collections are very difficult to build, an evaluation of existing programs shows that collections, for the most part, have managed substantial coverage of the…

Online Continuing Professional Education: An Evaluative Case Study

ERIC Educational Resources Information Center

Pullen, Darren L.

2005-01-01

An evaluation study, assessing the pedagogical and instructional design (e-pedagogy) effectiveness of online continuing professional education (CPE) courses offered by a large Australasian CPE provider to health care professionals (HCPs). The study used a naturalistic theory approach in conjunction with a multilevel evaluation to examine the…

77 FR 15370 - Agency Information Collection Request. 60-Day Public Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2012-03-15

.... Proposed Project: Teen Pregnancy Prevention Replication Evaluation Study: Follow-up Data Collection--OMB No... Evaluation (ASPE), the Centers for Disease Control and Prevention (CDC), and the Administration for Children... with ASPE the Teen Pregnancy Prevention Replication Evaluation Study (TPP Replication Study). The TPP...

7 CFR 295.4 - Program evaluation status reports.

Code of Federal Regulations, 2011 CFR

2011-01-01

... completed studies may be obtained by writing the Director, Office of Analysis and Evaluation, Food and... evaluation status reports. FNS also publishes summaries of objectives and findings of completed studies and...

Evaluation of a proposed expert system development methodology: Two case studies

NASA Technical Reports Server (NTRS)

Gilstrap, Lewey

1990-01-01

Two expert system development projects were studied to evaluate a proposed Expert Systems Development Methodology (ESDM). The ESDM was developed to provide guidance to managers and technical personnel and serve as a standard in the development of expert systems. It was agreed that the proposed ESDM must be evaluated before it could be adopted; therefore a study was planned for its evaluation. This detailed study is now underway. Before the study began, however, two ongoing projects were selected for a retrospective evaluation. They were the Ranging Equipment Diagnostic Expert System (REDEX) and the Backup Control Mode Analysis and Utility System (BCAUS). Both projects were approximately 1 year into development. Interviews of project personnel were conducted, and the resulting data was used to prepare the retrospective evaluation. Decision models of the two projects were constructed and used to evaluate the completeness and accuracy of key provisions of ESDM. A major conclusion reached from these case studies is that suitability and risk analysis should be required for all AI projects, large and small. Further, the objectives of each stage of development during a project should be selected to reduce the next largest area of risk or uncertainty on the project.

Usability Study and Heuristic Evaluation of the Applied Robotics for Installations and Base Operations (ARIBO) Driverless Vehicle Reservation Application ARIBO Mobile

DTIC Science & Technology

2017-03-01

ARL-TN-0814 ● MAR 2017 US Army Research Laboratory Usability Study and Heuristic Evaluation of the Applied Robotics for...ARL-TN-0814 ● MAR 2017 US Army Research Laboratory Usability Study and Heuristic Evaluation of the Applied Robotics for...Heuristic Evaluation of the Applied Robotics for Installations and Base Operations (ARIBO) Driverless Vehicle Reservation Application ARIBO Mobile 5a

Evaluation of STD/AIDS prevention programs: a review of approaches and methodologies.

PubMed

da Cruz, Marly Marques; dos Santos, Elizabeth Moreira; Monteiro, Simone

2007-05-01

The article presents a review of approaches and methodologies in the evaluation of STD/AIDS prevention programs, searching for theoretical and methodological support for the institutionalization of evaluation and decision-making. The review included the MEDLINE, SciELO, and ISI Web of Science databases and other sources like textbooks and congress abstracts from 1990 to 2005, with the key words: "evaluation", "programs", "prevention", "STD/AIDS", and similar terms. The papers showed a predominance of quantitative outcome or impact evaluative studies with an experimental or quasi-experimental design. The main use of evaluation is accountability, although knowledge output and program improvement were also identified in the studies. Only a few evaluative studies contemplate process evaluation and its relationship to the contexts. The review aimed to contribute to the debate on STD/AIDS, which requires more effective, consistent, and sustainable decisions in the field of prevention.

75 FR 68829 - Agency Information Collection Activities: Comment Request; Generic Survey Clearance for the...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-11-09

... evaluation studies. In order to conduct program- or portfolio-level evaluations, however, both experimental and quasi-experimental evaluation research studies on STEM education interventions require researchers...

Evaluation and impact of cardiotocography training programmes: a systematic review.

PubMed

Pehrson, C; Sorensen, J L; Amer-Wåhlin, I

2011-07-01

The interpretation and management of cardiotocography (CTG) tracings are often criticised in obstetric malpractice cases. As a consequence, regular CTG training has been recommended, even though little is known about the effect of CTG training. To perform a systematic review of the existing literature on studies on CTG training in order to assess educational strategies, evaluation of training programmes, and impact of training programmes. The Medline database was searched to identify studies describing and/or evaluating CTG training programmes. The literature search resulted in 409 citations. Twenty studies describing and evaluating CTG training programmes were included. There was no restriction on study design. Data regarding study design, study quality, educational strategies used for training in CTG interpretation and decision making, target groups, number of participants, methods used for evaluation, quality of evaluation, level of evaluation and results of training was extracted from 20 articles, and analysed using Kirkpatrick's four-level model for the evaluation of education. Training was associated with improvements on all Kirkpatrick levels, resulting in increased CTG knowledge and interpretive skills, higher interobserver agreement, better management of intrapartum CTG, and improved quality of care. Computer-based training (CBT) might be less time-consuming than classroom teaching. Clinical skills seem to decrease faster than theoretical knowledge. Training can improve CTG competence and clinical practise. Further research on CBT, test-enhanced learning and long-term retention, evaluation of training and impact on clinical outcomes is recommended. © 2011 The Authors BJOG An International Journal of Obstetrics and Gynaecology © 2011 RCOG.

Health Science students' evaluation of courses and Instructors: the effect of response rate and class size interaction.

PubMed

Kuwaiti, Ahmed Al

2015-01-01

This study aims at investigating the effect of response rate and class size interaction on students' evaluation of instructors and the courses offered at heath science colleges in Saudi Arabia. A retrospective study design was adapted to ascertain Course Evaluation Surveys (CES) conducted at the health science colleges of the University of Dammam [UOD] in the academic year 2013-2014. Accordingly, the CES data which was downloaded from an exclusive online application 'UDQUEST' which includes 337 different courses and 15,264 surveys were utilized in this study. Two-way analysis of variance was utilized to test whether there is any significant interaction between the class size and the response rate on the students' evaluation of courses and instructors. The study showed that high response rate is required for student evaluation of instructors at Health Science colleges when the class size is small whereas a medium response rate is required for students' evaluation of courses. On the other hand, when the class size is medium, a medium or high response rate is needed for students' evaluation of both instructors and courses. The results of this study recommend that the administrators of the health science colleges to be aware of the interpretation of students' evaluations of courses and instructors. The study also suggests that the interaction between response rate and class size is a very important factor that needs to be taken into consideration while interpreting the findings of the students' evaluation of instructors and courses.

42 CFR 456.242 - UR plan requirements for medical care evaluation studies.

Code of Federal Regulations, 2010 CFR

2010-10-01

... HEALTH AND HUMAN SERVICES (CONTINUED) MEDICAL ASSISTANCE PROGRAMS UTILIZATION CONTROL Utilization Control: Mental Hospitals Ur Plan: Medical Care Evaluation Studies § 456.242 UR plan requirements for medical care... evaluation studies in the mental hospital; (2) Documents for each study— (i) Its results; and (ii) How the...

Evaluation of Virtual Laboratory Package on Nigerian Secondary School Physics Concepts

ERIC Educational Resources Information Center

Falode, Oluwole Caleb; Gambari, Amosa Isiaka

2017-01-01

The study evaluated accessibility, flexibility, cost and learning effectiveness of researchers-developed virtual laboratory package for Nigerian secondary school physics. Based on these issues, four research questions were raised and answered. The study was a quantitative-based evaluation research. Sample for the study included 24 physics…

77 FR 61789 - Agency Information Collection Activities; Submission for OMB Review; Comment Request; Workforce...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-10-11

... Programs Gold Standard Evaluation Follow-Up Surveys, Veterans Study, and Cost Data,'' to the Office of... Standard Evaluation Follow-Up Surveys, Veterans Study, and Cost Data ACTION: Notice. SUMMARY: The... Worker Programs Gold Standard Evaluation Follow-Up Surveys, Veterans Study, and Cost Data. OMB ICR...

Handbook of Evaluated Materials for Secondary Social Studies.

ERIC Educational Resources Information Center

Mount Diablo Unified School District, Concord, CA.

The handbook contains teacher evaluations of secondary social studies materials used in the Mt. Diablo Unified School District (MDUSD), Orinda, California. The Materials Testing Program of MDUSD is an effort to involve secondary teachers in the process of evaluating social studies materials in certain concept areas: conflict, social change, man's…

Teacher Evaluations in Leisure Studies Programs: An Old Issue with a New Slant.

ERIC Educational Resources Information Center

Butts, Frank B.; Swearingen, Tommy

1994-01-01

This paper examines teacher evaluation practices in leisure studies programs, noting the perceived effectiveness of rating instruments. Surveys of leisure studies professors nationwide indicated many institutions used evaluation instruments and processes that were not statistically validated; key decisions were often made on the basis of these…

An Evaluation of Digital Stories Created for Social Studies Teaching

ERIC Educational Resources Information Center

Seker, Burcu Sezginsoy

2016-01-01

Digital stories are useful tools for combining technology with education, in terms of the preparation stage, practicality, availability and usability as an evaluation instrument. In this study, digital stories created in a social studies teaching class were evaluated and the opinions of primary school teacher candidates were obtained concerning…

The Principal's Role in the Post-Evaluation Process.--How Does the Principal Engage in the Work Carried out after the Schools Self-Evaluation?

ERIC Educational Resources Information Center

Emstad, Anne Berit

2011-01-01

This article refers to a study on how the school principal engaged in the process after a school self-evaluation. The study examined how two primary schools followed up the evaluation. Although they both used the same evaluation tool, the schools' understanding and application of results differed greatly. This paper describes and discusses the…

A qualitative case study of evaluation use in the context of a collaborative program evaluation strategy in Burkina Faso.

PubMed

D'Ostie-Racine, Léna; Dagenais, Christian; Ridde, Valéry

2016-05-26

Program evaluation is widely recognized in the international humanitarian sector as a means to make interventions and policies more evidence based, equitable, and accountable. Yet, little is known about the way humanitarian non-governmental organizations (NGOs) actually use evaluations. The current qualitative evaluation employed an instrumental case study design to examine evaluation use (EU) by a humanitarian NGO based in Burkina Faso. This organization developed an evaluation strategy in 2008 to document the implementation and effects of its maternal and child healthcare user fee exemption program. Program evaluations have been undertaken ever since, and the present study examined the discourses of evaluation partners in 2009 (n = 15) and 2011 (n = 17). Semi-structured individual interviews and one group interview were conducted to identify instances of EU over time. Alkin and Taut's (Stud Educ Eval 29:1-12, 2003) conceptualization of EU was used as the basis for thematic qualitative analyses of the different forms of EU identified by stakeholders of the exemption program in the two data collection periods. Results demonstrated that stakeholders began to understand and value the utility of program evaluations once they were exposed to evaluation findings and then progressively used evaluations over time. EU was manifested in a variety of ways, including instrumental and conceptual use of evaluation processes and findings, as well as the persuasive use of findings. Such EU supported planning, decision-making, program practices, evaluation capacity, and advocacy. The study sheds light on the many ways evaluations can be used by different actors in the humanitarian sector. Conceptualizations of EU are also critically discussed.

Evaluating Teachers' Professional Development Initiatives: Towards an Extended Evaluative Framework

ERIC Educational Resources Information Center

Merchie, Emmelien; Tuytens, Melissa; Devos, Geert; Vanderlinde, Ruben

2018-01-01

Evaluating teachers' professional development initiatives (PDI) is one of the main challenges for the teacher professionalisation field. Although different studies have focused on the effectiveness of PDI, the obtained effects and evaluative methods have been found to be widely divergent. By means of a narrative review, this study provides an…

Can Principals Promote Teacher Development as Evaluators? A Case Study of Principals' Views and Experiences

ERIC Educational Resources Information Center

Kraft, Matthew A.; Gilmour, Allison F.

2016-01-01

Purpose: New teacher evaluation systems have expanded the role of principals as instructional leaders, but little is known about principals' ability to promote teacher development through the evaluation process. We conducted a case study of principals' perspectives on evaluation and their experiences implementing observation and feedback cycles to…

76 FR 78670 - Draft Guidance for Industry and Food and Drug Administration Staff; Evaluation of Sex Differences...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-19

...] Draft Guidance for Industry and Food and Drug Administration Staff; Evaluation of Sex Differences in... entitled ``Evaluation of Sex Differences in Medical Device Clinical Studies.'' This document provides guidance on the study and evaluation of sex differences in medical device clinical trials, with a specific...

Summative Evaluation on the Hospital Wards. What Do Faculty Say to Learners?

ERIC Educational Resources Information Center

Hasley, Peggy B.; Arnold, Robert M.

2009-01-01

No previous studies have described how faculty give summative evaluations to learners on the medical wards. The aim of this study was to describe summative evaluations on the medical wards. Participants were students, house staff and faculty at the University of Pittsburgh. Ward rotation evaluative sessions were tape recorded. Feedback was…

Evaluating the Assessment Models for Young Children with Special Needs in Taiwan

ERIC Educational Resources Information Center

Ho, Hua-Kuo

2009-01-01

The purpose of this study was intended to evaluate the assessment models of two representative centers of team evaluation for children's development in Taiwan. Documentary analysis and phone interview were employed in the study to collect the research data needed. Two centers of team evaluation for children's development were selected and…

Evaluation of Turkish and Mathematics Curricula According to Value-Based Evaluation Model

ERIC Educational Resources Information Center

Duman, Serap Nur; Akbas, Oktay

2017-01-01

This study evaluated secondary school seventh-grade Turkish and mathematics programs using the Context-Input-Process-Product Evaluation Model based on student, teacher, and inspector views. The convergent parallel mixed method design was used in the study. Student values were identified using the scales for socio-level identification, traditional…

A Study To Evaluate the Effectiveness of the Montana Functional Vision Assessment for Multihandicapped Persons.

ERIC Educational Resources Information Center

Marks, Ron

The study evaluated the validity, reliability, and effectiveness in providing needed information of the Montana Functional Vision Assessment (MFVA) Instrument by comparing it with the Texas Education Agency Functional Vision Assessment and field testing the instrument in evaluations of nine multihandicapped students (ages 3-11). Evaluation led to…

Questionnaire Evaluating Teaching Competencies in the University Environment. Evaluation of Teaching Competencies in the University

ERIC Educational Resources Information Center

Moreno-Murcia, Juan Antonio; Silveira Torregrosa, Yolanda; Belando Pedreño, Noelia

2015-01-01

The objective of this study was to design and validate a measuring instrument to evaluate the performance of university professors. The Evaluation of Teaching Performance (CEID [Centro de Estudios e Investigaciones Docentes (Center for Teaching Studies and Research)]) questionnaire was administered to 1297 university students. Various factor…

An Analysis of Critical Issues in Korean Teacher Evaluation Systems

ERIC Educational Resources Information Center

Choi, Hee Jun; Park, Ji-Hye

2016-01-01

Korea has used three different teacher evaluation systems since the 1960s: teacher performance rating, teacher performance-based pay and teacher evaluation for professional development. A number of studies have focused on an analysis of each evaluation system in terms of its advent, development, advantages and disadvantages, but these studies have…

Measuring Spontaneous and Instructed Evaluation Processes during Web Search: Integrating Concurrent Thinking-Aloud Protocols and Eye-Tracking Data

ERIC Educational Resources Information Center

Gerjets, Peter; Kammerer, Yvonne; Werner, Benita

2011-01-01

Web searching for complex information requires to appropriately evaluating diverse sources of information. Information science studies identified different criteria applied by searchers to evaluate Web information. However, the explicit evaluation instructions used in these studies might have resulted in a distortion of spontaneous evaluation…

An Evaluation of the Private High School Curriculum in Turkey

ERIC Educational Resources Information Center

Aslan, Dolgun

2016-01-01

This study aims at evaluating curricula of private high schools in line with opinions of teachers working at the related high schools, and identifying any related problems. Screening model is used as a quantitative research method in the study. The "element-based curriculum evaluation model" is taken as basis for evaluation of the…

Understanding Expertise-Based Training Effects on the Software Evaluation Process of Mathematics Education Teachers

ERIC Educational Resources Information Center

Incikabi, Lutfi; Sancar Tokmak, Hatice

2012-01-01

This case study examined the educational software evaluation processes of pre-service teachers who attended either expertise-based training (XBT) or traditional training in conjunction with a Software-Evaluation checklist. Forty-three mathematics teacher candidates and three experts participated in the study. All participants evaluated educational…

[The development of evaluation capacity in primary healthcare management: a case study in Santa Catarina State, Brazil, 2008-2011].

PubMed

Nickel, Daniela Alba; Calvo, Maria Cristina Marino; Natal, Sonia; Freitas, Sérgio Fernando Torres de; Hartz, Zulmira Maria de Araújo

2014-04-01

This article analyzes evaluation capacity-building based on the case study of a State Health Secretariat participating in the Project to Strengthen the Technical Capacity of State Health Secretariats in Monitoring and Evaluating Primary Healthcare. The case study adopted a mixed design with information from documents, semi-structured interviews, and evaluation of primary care by the State Health Secretariat in 2008-2011. Process analysis was used to identify the logical events that contributed to evaluation capacity-building, with two categories: evaluation capacity-building events and events for building organizational structure. The logical chain of events was formed by negotiation and agreement on the decision-making levels for the continuity of evaluation, data collection and analysis by the State Health Secretariat, a change in key indicators, restructuring of the evaluation matrix, and communication of the results to the municipalities. The three-way analysis showed that the aim of developing evaluation capacity was achieved.

User Interface Problems of a Nationwide Inpatient Information System: A Heuristic Evaluation.

PubMed

Atashi, Alireza; Khajouei, Reza; Azizi, Amirabbas; Dadashi, Ali

2016-01-01

While studies have shown that usability evaluation could uncover many design problems of health information systems, the usability of health information systems in developing countries using their native language is poorly studied. The objective of this study was to evaluate the usability of a nationwide inpatient information system used in many academic hospitals in Iran. Three trained usability evaluators independently evaluated the system using Nielsen's 10 usability heuristics. The evaluators combined identified problems in a single list and independently rated the severity of the problems. We statistically compared the number and severity of problems identified by HIS experienced and non-experienced evaluators. A total of 158 usability problems were identified. After removing duplications 99 unique problems were left. The highest mismatch with usability principles was related to "Consistency and standards" heuristic (25%) and the lowest related to "Flexibility and efficiency of use" (4%). The average severity of problems ranged from 2.4 (Major problem) to 3.3 (Catastrophe problem). The experienced evaluator with HIS identified significantly more problems and gave higher severities to problems (p<0.02). Heuristic Evaluation identified a high number of usability problems in a widely used inpatient information system in many academic hospitals. These problems, if remain unsolved, may waste users' and patients' time, increase errors and finally threaten patient's safety. Many of them can be fixed with simple redesign solutions such as using clear labels and better layouts. This study suggests conducting further studies to confirm the findings concerning effect of evaluator experience on the results of Heuristic Evaluation.

User Interface Problems of a Nationwide Inpatient Information System: A Heuristic Evaluation

PubMed Central

Atashi, Alireza; Azizi, Amirabbas; Dadashi, Ali

2016-01-01

Summary Introduction While studies have shown that usability evaluation could uncover many design problems of health information systems, the usability of health information systems in developing countries using their native language is poorly studied. The objective of this study was to evaluate the usability of a nationwide inpatient information system used in many academic hospitals in Iran. Material and Methods Three trained usability evaluators independently evaluated the system using Nielsen’s 10 usability heuristics. The evaluators combined identified problems in a single list and independently rated the severity of the problems. We statistically compared the number and severity of problems identified by HIS experienced and non-experienced evaluators. Results A total of 158 usability problems were identified. After removing duplications 99 unique problems were left. The highest mismatch with usability principles was related to “Consistency and standards” heuristic (25%) and the lowest related to “Flexibility and efficiency of use” (4%). The average severity of problems ranged from 2.4 (Major problem) to 3.3 (Catastrophe problem). The experienced evaluator with HIS identified significantly more problems and gave higher severities to problems (p<0.02). Discussion Heuristic Evaluation identified a high number of usability problems in a widely used inpatient information system in many academic hospitals. These problems, if remain unsolved, may waste users’ and patients’ time, increase errors and finally threaten patient’s safety. Many of them can be fixed with simple redesign solutions such as using clear labels and better layouts. This study suggests conducting further studies to confirm the findings concerning effect of evaluator experience on the results of Heuristic Evaluation. PMID:27081409

Evaluation Criteria of Noninvasive Telemonitoring for Patients With Heart Failure: Systematic Review.

PubMed

Farnia, Troskah; Jaulent, Marie-Christine; Steichen, Olivier

2018-01-16

Telemonitoring can improve heart failure (HF) management, but there is no standardized evaluation framework to comprehensively evaluate its impact. Our objectives were to list the criteria used in published evaluations of noninvasive HF telemonitoring projects, describe how they are used in the evaluation studies, and organize them into a consistent scheme. Articles published from January 1990 to August 2015 were obtained through MEDLINE, Web of Science, and EMBASE. Articles were eligible if they were original reports of a noninvasive HF telemonitoring evaluation study in the English language. Studies of implantable telemonitoring devices were excluded. Each selected article was screened to extract the description of the telemonitoring project and the evaluation process and criteria. A qualitative synthesis was performed. We identified and reviewed 128 articles leading to 52 evaluation criteria classified into 6 dimensions: clinical, economic, user perspective, educational, organizational, and technical. The clinical and economic impacts were evaluated in more than 70% of studies, whereas the educational, organizational, and technical impacts were studied in fewer than 15%. User perspective was the most frequently covered dimension in the development phase of telemonitoring projects, whereas clinical and economic impacts were the focus of later phases. Telemonitoring evaluation frameworks should cover all 6 dimensions appropriately distributed along the telemonitoring project lifecycle. Our next goal is to build such a comprehensive evaluation framework for telemonitoring and test it on an ongoing noninvasive HF telemonitoring project. ©Troskah Farnia, Marie-Christine Jaulent, Olivier Steichen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.01.2018.

A Model and Method of Evaluative Accounts: Development Impact of the National Literacy Mission (NLM of India).

ERIC Educational Resources Information Center

Bhola, H. S.

2002-01-01

Studied the developmental impact of the National Literacy Mission of India, providing an evaluative account based on 97 evaluation studies. Compared findings with those from a 27-study synthesis of studies of effects of adult literacy efforts in Africa. Findings show the impact of literacy on the development of nations. (SLD)

Evaluation of the Professional Development Program on Web Based Content Development

ERIC Educational Resources Information Center

Yurdakul, Bünyamin; Uslu, Öner; Çakar, Esra; Yildiz, Derya G.

2014-01-01

The aim of this study is to evaluate the professional development program on web based content development (WBCD) designed by the Ministry of National Education (MoNE). Based on the theoretical CIPP model by Stufflebeam and Guskey's levels of evaluation, the study was carried out as a case study. The study group consisted of the courses that…

40 CFR Appendix A to Subpart H of... - Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations

Code of Federal Regulations, 2010 CFR

2010-07-01

... Studies and Environmental Evaluations A Appendix A to Subpart H of Part 35 Protection of Environment... to Subpart H of Part 35—Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations... officer as specified in §§ 35.1650-3(b)(1) and 35.1650-3(c)(1). (a) A diagnostic study consisting of: (1...

40 CFR Appendix A to Subpart H of... - Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations

Code of Federal Regulations, 2012 CFR

2012-07-01

... Studies and Environmental Evaluations A Appendix A to Subpart H of Part 35 Protection of Environment... to Subpart H of Part 35—Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations... officer as specified in §§ 35.1650-3(b)(1) and 35.1650-3(c)(1). (a) A diagnostic study consisting of: (1...

40 CFR Appendix A to Subpart H of... - Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations

Code of Federal Regulations, 2014 CFR

2014-07-01

... Studies and Environmental Evaluations A Appendix A to Subpart H of Part 35 Protection of Environment... to Subpart H of Part 35—Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations... officer as specified in §§ 35.1650-3(b)(1) and 35.1650-3(c)(1). (a) A diagnostic study consisting of: (1...

40 CFR Appendix A to Subpart H of... - Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations

Code of Federal Regulations, 2013 CFR

2013-07-01

... Studies and Environmental Evaluations A Appendix A to Subpart H of Part 35 Protection of Environment... to Subpart H of Part 35—Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations... officer as specified in §§ 35.1650-3(b)(1) and 35.1650-3(c)(1). (a) A diagnostic study consisting of: (1...

40 CFR Appendix A to Subpart H of... - Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations

Code of Federal Regulations, 2011 CFR

2011-07-01

... Studies and Environmental Evaluations A Appendix A to Subpart H of Part 35 Protection of Environment... to Subpart H of Part 35—Requirements for Diagnostic-Feasibility Studies and Environmental Evaluations... officer as specified in §§ 35.1650-3(b)(1) and 35.1650-3(c)(1). (a) A diagnostic study consisting of: (1...

Evaluating the utility of two gestural discomfort evaluation methods

PubMed Central

Son, Minseok; Jung, Jaemoon; Park, Woojin

2017-01-01

Evaluating physical discomfort of designed gestures is important for creating safe and usable gesture-based interaction systems; yet, gestural discomfort evaluation has not been extensively studied in HCI, and few evaluation methods seem currently available whose utility has been experimentally confirmed. To address this, this study empirically demonstrated the utility of the subjective rating method after a small number of gesture repetitions (a maximum of four repetitions) in evaluating designed gestures in terms of physical discomfort resulting from prolonged, repetitive gesture use. The subjective rating method has been widely used in previous gesture studies but without empirical evidence on its utility. This study also proposed a gesture discomfort evaluation method based on an existing ergonomics posture evaluation tool (Rapid Upper Limb Assessment) and demonstrated its utility in evaluating designed gestures in terms of physical discomfort resulting from prolonged, repetitive gesture use. Rapid Upper Limb Assessment is an ergonomics postural analysis tool that quantifies the work-related musculoskeletal disorders risks for manual tasks, and has been hypothesized to be capable of correctly determining discomfort resulting from prolonged, repetitive gesture use. The two methods were evaluated through comparisons against a baseline method involving discomfort rating after actual prolonged, repetitive gesture use. Correlation analyses indicated that both methods were in good agreement with the baseline. The methods proposed in this study seem useful for predicting discomfort resulting from prolonged, repetitive gesture use, and are expected to help interaction designers create safe and usable gesture-based interaction systems. PMID:28423016

Economic evaluation of vaccines in Canada: A systematic review

PubMed Central

Chit, Ayman; Lee, Jason K. H.; Shim, Minsup; Nguyen, Van Hai; Grootendorst, Paul; Wu, Jianhong; Van Exan, Robert; Langley, Joanne M.

2016-01-01

ABSTRACT Background: Economic evaluations should form part of the basis for public health decision making on new vaccine programs. While Canada's national immunization advisory committee does not systematically include economic evaluations in immunization decision making, there is increasing interest in adopting them. We therefore sought to examine the extent and quality of economic evaluations of vaccines in Canada. Objective: We conducted a systematic review of economic evaluations of vaccines in Canada to determine and summarize: comprehensiveness across jurisdictions, studied vaccines, funding sources, study designs, research quality, and changes over time. Methods: Searches in multiple databases were conducted using the terms “vaccine,” “economics” and “Canada.” Descriptive data from eligible manuscripts was abstracted and three authors independently evaluated manuscript quality using a 7-point Likert-type scale scoring tool based on criteria from the International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Results: 42/175 articles met the search criteria. Of these, Canada-wide studies were most common (25/42), while provincial studies largely focused on the three populous provinces of Ontario, Quebec and British Columbia. The most common funding source was industry (17/42), followed by government (7/42). 38 studies used mathematical models estimating expected economic benefit while 4 studies examined post-hoc data on established programs. Studies covered 10 diseases, with 28/42 addressing pediatric vaccines. Many studies considered cost-utility (22/42) and the majority of these studies reported favorable economic results (16/22). The mean quality score was 5.9/7 and was consistent over publication date, funding sources, and disease areas. Conclusions: We observed diverse approaches to evaluate vaccine economics in Canada. Given the increased complexity of economic studies evaluating vaccines and the impact of results on public health practice, Canada needs improved, transparent and consistent processes to review and assess the findings of the economic evaluations of vaccines. PMID:26890128

Student perceptions of evaluation in undergraduate medical education: A qualitative study from one medical school.

PubMed

Schiekirka, Sarah; Reinhardt, Deborah; Heim, Susanne; Fabry, Götz; Pukrop, Tobias; Anders, Sven; Raupach, Tobias

2012-06-22

Evaluation is an integral part of medical education. Despite a wide use of various evaluation tools, little is known about student perceptions regarding the purpose and desired consequences of evaluation. Such knowledge is important to facilitate interpretation of evaluation results. The aims of this study were to elicit student views on the purpose of evaluation, indicators of teaching quality, evaluation tools and possible consequences drawn from evaluation data. This qualitative study involved 17 undergraduate medical students in Years 3 and 4 participating in 3 focus group interviews. Content analysis was conducted by two different researchers. Evaluation was viewed as a means to facilitate improvements within medical education. Teaching quality was believed to be dependent on content, process, teacher and student characteristics as well as learning outcome, with an emphasis on the latter. Students preferred online evaluations over paper-and-pencil forms and suggested circulating results among all faculty and students. Students strongly favoured the allocation of rewards and incentives for good teaching to individual teachers. In addition to assessing structural aspects of teaching, evaluation tools need to adequately address learning outcome. The use of reliable and valid evaluation methods is a prerequisite for resource allocation to individual teachers based on evaluation results.

Student perceptions of evaluation in undergraduate medical education: A qualitative study from one medical school

PubMed Central

2012-01-01

Background Evaluation is an integral part of medical education. Despite a wide use of various evaluation tools, little is known about student perceptions regarding the purpose and desired consequences of evaluation. Such knowledge is important to facilitate interpretation of evaluation results. The aims of this study were to elicit student views on the purpose of evaluation, indicators of teaching quality, evaluation tools and possible consequences drawn from evaluation data. Methods This qualitative study involved 17 undergraduate medical students in Years 3 and 4 participating in 3 focus group interviews. Content analysis was conducted by two different researchers. Results Evaluation was viewed as a means to facilitate improvements within medical education. Teaching quality was believed to be dependent on content, process, teacher and student characteristics as well as learning outcome, with an emphasis on the latter. Students preferred online evaluations over paper-and-pencil forms and suggested circulating results among all faculty and students. Students strongly favoured the allocation of rewards and incentives for good teaching to individual teachers. Conclusions In addition to assessing structural aspects of teaching, evaluation tools need to adequately address learning outcome. The use of reliable and valid evaluation methods is a prerequisite for resource allocation to individual teachers based on evaluation results. PMID:22726271

Guidelines for reporting evaluations based on observational methodology.

PubMed

Portell, Mariona; Anguera, M Teresa; Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana

2015-01-01

Observational methodology is one of the most suitable research designs for evaluating fidelity of implementation, especially in complex interventions. However, the conduct and reporting of observational studies is hampered by the absence of specific guidelines, such as those that exist for other evaluation designs. This lack of specific guidance poses a threat to the quality and transparency of these studies and also constitutes a considerable publication hurdle. The aim of this study thus was to draw up a set of proposed guidelines for reporting evaluations based on observational methodology. The guidelines were developed by triangulating three sources of information: observational studies performed in different fields by experts in observational methodology, reporting guidelines for general studies and studies with similar designs to observational studies, and proposals from experts in observational methodology at scientific meetings. We produced a list of guidelines grouped into three domains: intervention and expected outcomes, methods, and results. The result is a useful, carefully crafted set of simple guidelines for conducting and reporting observational studies in the field of program evaluation.

The role of affect and cognition in health decision making.

PubMed

Keer, Mario; van den Putte, Bas; Neijens, Peter

2010-03-01

Both affective and cognitive evaluations of behaviours have been allocated various positions in theoretical models of decision making. Most often, they have been studied as direct determinants of either intention or overall evaluation, but these two possible positions have never been compared. The aim of this study was to determine whether affective and cognitive evaluations influence intention directly, or whether their influence is mediated by overall evaluation. A sample of 300 university students filled in questionnaires on their affective, cognitive, and overall evaluations in respect of 20 health behaviours. The data were interpreted using mediation analyses with the application of path modelling. Both affective and cognitive evaluations were found to have significantly predicted intention. The influence of affective evaluation was largely direct for each of the behaviours studied, whereas that of cognitive evaluation was partially direct and partially mediated by overall evaluation. These results indicate that decisions regarding the content of persuasive communication (affective vs. cognitive) are highly dependent on the theoretical model chosen. It is suggested that affective evaluation should be included as a direct determinant of intention in theories of decision making when predicting health behaviours.

Escalator Design Features Evaluation

DOT National Transportation Integrated Search

1982-05-01

This study provides an evaluation of the effectiveness of several special design features associated with escalators in rail transit systems. The objective of the study was to evaluate the effectiveness of three escalator design features: (1) mat ope...

STARE-HI – Statement on Reporting of Evaluation Studies in Health Informatics

PubMed Central

Brender, J.; Talmon, J.; de Keizer, N.; Nykänen, P.; Rigby, M.; Ammenwerth, E.

2013-01-01

Summary Background Improving the quality of reporting of evaluation studies in health informatics is an important requirement towards the vision of evidence-based health informatics. The STARE-HI – Statement on Reporting of Evaluation Studies in health informatics, published in 2009, provides guidelines on the elements to be contained in an evaluation study report. Objectives To elaborate on and provide a rationale for the principles of STARE-HI and to guide authors and readers of evaluation studies in health informatics by providing explanatory examples of reporting. Methods A group of methodologists, researchers and editors prepared the present elaboration of the STARE-HI statement and selected examples from the literature. Results The 35 STARE-HI items to be addressed in evaluation papers describing health informatics interventions are discussed one by one and each is extended with examples and elaborations. Conclusion The STARE-HI statement and this elaboration document should be helpful resources to improve reporting of both quantitative and qualitative evaluation studies. Evaluation manuscripts adhering to the principles will enable readers of such papers to better place the studies in a proper context and judge their validity and generalizability, and thus in turn optimize the exploitation of the evidence contained therein. Limitations This paper is based on experiences of a group of editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the details of these principles has to evolve as a function of their use in practice. PMID:24155788

Evaluating the Feasibility of RESCUE: An Adjunctive HAI-Based Intervention for Veterans with PTSD

DTIC Science & Technology

2017-05-01

with clinicians on study 9/14/16 – 9/16/16. c) Independent evaluators completed 1-day Clinician Administered PTSD Scale (CAPS) training 10/31/16. d...therapists Recruit, facilitate hiring, and train study independent evaluators (IEs) 0-6 100% Facilitate and coordinate training and PE certification...developed referral stream for study 6 - 8 100% Major Task 3: Conduct pre-treatment evaluations for case series Screen, obtain consent, assess, and

Consensus-based cross-European recommendations for the identification, measurement and valuation of costs in health economic evaluations: a European Delphi study.

PubMed

van Lier, Lisanne I; Bosmans, Judith E; van Hout, Hein P J; Mokkink, Lidwine B; van den Hout, Wilbert B; de Wit, G Ardine; Dirksen, Carmen D; Nies, Henk L G R; Hertogh, Cees M P M; van der Roest, Henriëtte G

2017-12-19

Differences between country-specific guidelines for economic evaluations complicate the execution of international economic evaluations. The aim of this study was to develop cross-European recommendations for the identification, measurement and valuation of resource use and lost productivity in economic evaluations using a Delphi procedure. A comprehensive literature search was conducted to identify European guidelines on the execution of economic evaluations or costing studies as part of economic evaluations. Guideline recommendations were extracted by two independent reviewers and formed the basis for the first round of the Delphi study, which was conducted among European health economic experts. During three written rounds, consensus (agreement of 67% or higher) was sought on items concerning the identification, measurement and valuation of costs. Recommendations from 18 guidelines were extracted. Consensus among 26 panellists from 17 European countries was reached on 61 of 68 items. The recommendations from the Delphi study are to adopt a societal perspective, to use patient report for measuring resource use and lost productivity, to value both constructs with use of country-specific standardized/unit costs and to use country-specific discounting rates. This study provides consensus-based cross-European recommendations on how to measure and value resource use and lost productivity in economic evaluations. These recommendations are expected to support researchers, healthcare professionals, and policymakers in executing and appraising economic evaluations performed in international contexts.

Methodological convergence of program evaluation designs.

PubMed

Chacón-Moscoso, Salvador; Anguera, M Teresa; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2014-01-01

Nowadays, the confronting dichotomous view between experimental/quasi-experimental and non-experimental/ethnographic studies still exists but, despite the extensive use of non-experimental/ethnographic studies, the most systematic work on methodological quality has been developed based on experimental and quasi-experimental studies. This hinders evaluators and planners' practice of empirical program evaluation, a sphere in which the distinction between types of study is changing continually and is less clear. Based on the classical validity framework of experimental/quasi-experimental studies, we carry out a review of the literature in order to analyze the convergence of design elements in methodological quality in primary studies in systematic reviews and ethnographic research. We specify the relevant design elements that should be taken into account in order to improve validity and generalization in program evaluation practice in different methodologies from a practical methodological and complementary view. We recommend ways to improve design elements so as to enhance validity and generalization in program evaluation practice.

Non-formal educator use of evaluation results.

PubMed

Baughman, Sarah; Boyd, Heather H; Franz, Nancy K

2012-08-01

Increasing demands for accountability in educational programming have resulted in increasing calls for program evaluation in educational organizations. Many organizations include conducting program evaluations as part of the job responsibilities of program staff. Cooperative Extension is a complex organization offering non-formal educational programs through land grant universities. Many Extension services require non-formal educational program evaluations be conducted by field-based Extension educators. Evaluation research has focused primarily on the efforts of professional, external evaluators. The work of program staff with many responsibilities including program evaluation has received little attention. This study examined how field based Extension educators (i.e. program staff) in four Extension services use the results of evaluations of programs that they have conducted themselves. Four types of evaluation use are measured and explored; instrumental use, conceptual use, persuasive use and process use. Results indicate that there are few programmatic changes as a result of evaluation findings among the non-formal educators surveyed in this study. Extension educators tend to use evaluation results to persuade others about the value of their programs and learn from the evaluation process. Evaluation use is driven by accountability measures with very little program improvement use as measured in this study. Practical implications include delineating accountability and program improvement tasks within complex organizations in order to align evaluation efforts and to improve the results of both. There is some evidence that evaluation capacity building efforts may be increasing instrumental use by educators evaluating their own programs. Copyright © 2011 Elsevier Ltd. All rights reserved.

Case study of evaluations that go beyond clinical outcomes to assess quality improvement diabetes programmes using the Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE).

PubMed

Paquette-Warren, Jann; Harris, Stewart B; Naqshbandi Hayward, Mariam; Tompkins, Jordan W

2016-10-01

Investments in efforts to reduce the burden of diabetes on patients and health care are critical; however, more evaluation is needed to provide evidence that informs and supports future policies and programmes. The newly developed Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE) incorporates the theoretical concepts needed to facilitate the capture of critical information to guide investments, policy and programmatic decision making. The aim of the study is to assess the applicability and value of DEFINE in comprehensive real-world evaluation. Using a critical and positivist approach, this intrinsic and collective case study retrospectively examines two naturalistic evaluations to demonstrate how DEFINE could be used when conducting real-world comprehensive evaluations in health care settings. The variability between the cases and the evaluation designs are described and aligned to the DEFINE goals, steps and sub-steps. The majority of the theoretical steps of DEFINE were exemplified in both cases, although limited for knowledge translation efforts. Application of DEFINE to evaluate diverse programmes that target various chronic diseases is needed to further test the inclusivity and built-in flexibility of DEFINE and its role in encouraging more comprehensive knowledge translation. This case study shows how DEFINE could be used to structure or guide comprehensive evaluations of programmes and initiatives implemented in health care settings and support scale-up of successful innovations. Future use of the framework will continue to strengthen its value in guiding programme evaluation and informing health policy to reduce the burden of diabetes and other chronic diseases. © 2016 The Authors. Journal of Evaluation in Clinical Practice published by John Wiley & Sons, Ltd.

Outcome Reporting Bias in Government-Sponsored Policy Evaluations: A Qualitative Content Analysis of 13 Studies

PubMed Central

2016-01-01

The reporting of evaluation outcomes can be a point of contention between evaluators and policy-makers when a given reform fails to fulfil its promises. Whereas evaluators are required to report outcomes in full, policy-makers have a vested interest in framing these outcomes in a positive light–especially when they previously expressed a commitment to the reform. The current evidence base is limited to a survey of policy evaluators, a study on reporting bias in education research and several studies investigating the influence of industry sponsorship on the reporting of clinical trials. The objective of this study was twofold. Firstly, it aimed to assess the risk of outcome reporting bias (ORB or ‘spin’) in pilot evaluation reports, using seven indicators developed by clinicians. Secondly, it sought to examine how the government’s commitment to a given reform may affect the level of ORB found in the corresponding evaluation report. To answer these questions, 13 evaluation reports were content-analysed, all of which found a non-significant effect of the intervention on its stated primary outcome. These reports were systematically selected from a dataset of 233 pilot and experimental evaluations spanning three policy areas and 13 years of government-commissioned research in the UK. The results show that the risk of ORB is real. Indeed, all studies reviewed here resorted to at least one of the presentational strategies associated with a risk of spin. This study also found a small, negative association between the seniority of the reform’s champion and the risk of ORB in the evaluation of that reform. The publication of protocols and the use of reporting guidelines are recommended. PMID:27690131

Helping Students Evaluate the Validity of a Research Study.

ERIC Educational Resources Information Center

Morgan, George A.; Gliner, Jeffrey A.

Students often have difficulty in evaluating the validity of a study. A conceptually and linguistically meaningful framework for evaluating research studies is proposed that is based on the discussion of internal and external validity of T. D. Cook and D. T. Campbell (1979). The proposal includes six key dimensions, three related to internal…

Evaluation of Core Vocabulary Intervention for Treatment of Inconsistent Phonological Disorder: Three Treatment Case Studies

ERIC Educational Resources Information Center

McIntosh, Beth; Dodd, Barbara

2009-01-01

Children with unintelligible speech differ in severity, underlying deficit, type of surface error patterns and response to treatment. Detailed treatment case studies, evaluating specific intervention protocols for particular diagnostic groups, can identify best practice for children with speech disorder. Three treatment case studies evaluated the…

How to Critically Evaluate Case Studies in Social Work

ERIC Educational Resources Information Center

Lee, Eunjung; Mishna, Faye; Brennenstuhl, Sarah

2010-01-01

The purpose of this article is to develop guidelines to assist practitioners and researchers in evaluating and developing rigorous case studies. The main concern in evaluating a case study is to accurately assess its quality and ultimately to offer clients social work interventions informed by the best available evidence. To assess the quality of…

The evaluation of complex interventions in palliative care: an exploration of the potential of case study research strategies.

PubMed

Walshe, Catherine

2011-12-01

Complex, incrementally changing, context dependent and variable palliative care services are difficult to evaluate. Case study research strategies may have potential to contribute to evaluating such complex interventions, and to develop this field of evaluation research. This paper explores definitions of case study (as a unit of study, a process, and a product) and examines the features of case study research strategies which are thought to confer benefits for the evaluation of complex interventions in palliative care settings. Ten features of case study that are thought to be beneficial in evaluating complex interventions in palliative care are discussed, drawing from exemplars of research in this field. Important features are related to a longitudinal approach, triangulation, purposive instance selection, comprehensive approach, multiple data sources, flexibility, concurrent data collection and analysis, search for proving-disproving evidence, pattern matching techniques and an engaging narrative. The limitations of case study approaches are discussed including the potential for subjectivity and their complex, time consuming and potentially expensive nature. Case study research strategies have great potential in evaluating complex interventions in palliative care settings. Three key features need to be exploited to develop this field: case selection, longitudinal designs, and the use of rival hypotheses. In particular, case study should be used in situations where there is interplay and interdependency between the intervention and its context, such that it is difficult to define or find relevant comparisons.

Application of an organizational evaluation capacity self-assessment instrument to different organizations: similarities and lessons learned.

PubMed

Bourgeois, Isabelle; Whynot, Jane; Thériault, Étienne

2015-06-01

Organizational evaluation capacity (EC) has received significant attention in the evaluation research literature in the past decade. Much of the focus has been on defining organizational evaluation capacity, which can be thought of as the competencies and structures required to conduct high-quality evaluation studies (capacity to do), as well as the organization's ability to integrate evaluation findings into its decision-making processes (capacity to use). This paper seeks to contribute to this growing body of knowledge through a multiple case study of EC across three different organizations (e.g., non-profit, provincial government and federal government, herein named sectors); the novelty of this particular study is that each case study is based on the use of a common measurement tool developed by Bourgeois, Toews, Whynot and Lamarche (2013). The cross-case analysis presented in the paper reveals that evaluation capacity tends to be higher, both in terms of capacity to do and capacity to use, in organizations that have developed systematic mechanisms to institute an evaluation culture within their walls. Interestingly, however, we also found that capacity to use does not first require capacity to do, as evidenced in the non-profit organization under study. Copyright © 2015 Elsevier Ltd. All rights reserved.

Do Early Childhood Programs Have Lasting Effects on Children? Evaluation Science Brief

ERIC Educational Resources Information Center

National Forum on Early Childhood Program Evaluation, 2008

2008-01-01

"Evaluation Science Briefs" summarize the findings and implications of a recent study evaluating the effects of an early childhood program or environment. This Brief evaluates the study "Early Intervention in Low Birthweight Premature Infants: Results at 18 Years of Age for the Infant Health and Development Program (IHDP)" (M.C. McCormick, J.…

The Utility of Pitch Elevation in the Evaluation of Oropharyngeal Dysphagia: Preliminary Findings

ERIC Educational Resources Information Center

Malandraki, Georgia A.; Hind, Jacqueline A.; Gangnon, Ronald; Logemann, Jeri A.; Robbins, JoAnne

2011-01-01

Purpose: To evaluate the utility of a pitch elevation task in the assessment of oropharyngeal dysphagia. Method: This study was a pilot prospective cohort study including 40 consecutive patients (16 male and 24 female) who were referred by their physician for a swallowing evaluation. Patients were evaluated with a noninstrumental clinical…

Redesigning Teacher Evaluation: Lessons Learned from a Pilot Implementation in New Hampshire

ERIC Educational Resources Information Center

Riordan, Julie; Lacireno-Paquet, Natalie; Shakman, Karen; Bocala, Candice

2016-01-01

Many studies have called attention to the limitations of current teacher evaluation systems and the need for reform nationwide. This study addresses three research questions: (1) What are the features of the new teacher evaluation systems in New Hampshire's districts with SIG schools?; (2) To what extent did schools implement the evaluation system…

Using Student Evaluations at a Cambodian University to Improve Teaching Effectiveness

ERIC Educational Resources Information Center

Nash, John Lysne

2012-01-01

This is the first study done in Cambodia wherein students used the Student Evaluations of Educational Quality (SEEQ) evaluation tool to evaluate the teaching quality of their instructors. Respondents were instructors and students from the English Language Support Unit at the Royal University of Phnom Penh. This study generated data from mid- and…

Formative Evaluation of the No-Fee Teacher Education Program from the Students' Standpoint

ERIC Educational Resources Information Center

Han, Yumei; Hu, Meizhong; Li, Ling

2013-01-01

This exploratory case study applied a formative evaluation framework to evaluate the no-fee teacher education program at Southwest University. The study focused on the students' perspective and their perceptions of the program, both intrinsic and extrinsic. A self-evaluation checklist and a questionnaire were the instruments used to collect data.…

Evaluating the Evaluators: Comparative Study of High School Newspaper Critique Services.

ERIC Educational Resources Information Center

Davis, Nancy

High school publication staffs depend on national critique services as a major means of evaluation and recognition, but most have no measure of how one critique service compares to the others, because they can afford the entry fee for only one evaluation. Thus, a study was conducted to test the validity of three major national critique…

EVALUATION OF THE REAL-TIME AIR-QUALITY MODEL USING THE RAPS (REGIONAL AIR POLLUTION STUDY) DATA BASE. VOLUME 4. EVALUATION GUIDE

EPA Science Inventory

The theory and programming of statistical tests for evaluating the Real-Time Air-Quality Model (RAM) using the Regional Air Pollution Study (RAPS) data base are fully documented in four volumes. Moreover, the tests are generally applicable to other model evaluation problems. Volu...

[Valences of self-evaluation and approach-avoidance tendencies: research based on regulatory focus theory].

PubMed

Ozaki, Yuka; Karasawa, Kaori

2011-12-01

Four studies were conducted to investigate the relationship between valences of self-evaluation and approach-avoidance tendencies. Based on regulatory focus theory (Higgins, 1997, 1998), we predicted that positivity of self-evaluation is related to the tendency to approach gains, while negativity of self-evaluation is related to the tendency to avoid losses. In Study 1, a self-report measure of behavioral tendencies for approaching gains and avoiding losses was developed. In Studies 2 to 4, correlations between these approach/avoidance tendencies and various kinds of self-evaluations were examined. Overall, the authors' predictions were supported. The results suggest that the self-evaluation system and the self-regulation system work in close cooperation with each other in controlling human behavior.

Moving away from exhaustion: how core self-evaluations influence academic burnout.

PubMed

Lian, Penghu; Sun, Yunfeng; Ji, Zhigang; Li, Hanzhong; Peng, Jiaxi

2014-01-01

Academic burnout refers to students who have low interest, lack of motivation, and tiredness in studying. Studies concerning how to prevent academic burnout are rare. The present study aimed to investigate the impact of core self-evaluations on the academic burnout of university students, and mainly focused on the confirmation of the mediator role of life satisfaction. A total of 470 university students accomplished the core self-evaluation scale, Satisfaction with Life, and academic burnout scale. Both core self-evaluations and life satisfaction were significantly correlated with academic burnout. Structural equation modeling indicated that life satisfaction partially mediated the relationship between core self-evaluations and academic burnout. Core self-evaluations significantly influence academic burnout and are partially mediated by life satisfaction.

Review of economic evaluations of mask and respirator use for protection against respiratory infection transmission.

PubMed

Mukerji, Shohini; MacIntyre, C Raina; Newall, Anthony T

2015-10-13

There has been increasing debate surrounding mask and respirator interventions to control respiratory infection transmission in both healthcare and community settings. As decision makers are considering the recommendations they should evaluate how to provide the most efficient protection strategies with minimum costs. The aim of this review is to identify and evaluate the existing economic evaluation literature in this area and to offer advice on how future evaluations on this topic should be conducted. We searched the Scopus database for all literature on economic evaluation of mask or respirator use to control respiratory infection transmission. Reference lists from the identified studies were also manually searched. Seven studies met our inclusion criteria from the initial 806 studies identified by the search strategy and our manual search. Five studies considered interventions for seasonal and/or pandemic influenza, with one also considering SARS (Severe Acute Respiratory Syndrome). The other two studies focussed on tuberculosis transmission control interventions. The settings and methodologies of the studies varied greatly. No low-middle income settings were identified. Only one of the reviewed studies cited clinical evidence to inform their mask/respirator intervention effectiveness parameters. Mask and respirator interventions were generally reported by the study authors to be cost saving or cost-effective when compared to no intervention or other control measures, however the evaluations had important limitations. Given the large cost differential between masks and respirators, there is a need for more comprehensive economic evaluations to compare the relative costs and benefits of these interventions in situations and settings where alternative options are potentially applicable. There are at present insufficient well conducted cost-effectiveness studies to inform decision-makers on the value for money of alternative mask/respirator options.

Mastering moral misery: Emotional and coping responses to intragroup morality (vs. competence) evaluations.

PubMed

van der Lee, Romy; Ellemers, Naomi; Scheepers, Daan

2016-01-01

In social groups, individuals are often confronted with evaluations of their behaviour by other group members and are motivated to adapt their own behaviour accordingly. In two studies we examine emotional responses towards, and perceived coping abilities with, morality vs. competence evaluations individuals receive from other in-group members. In Study 1, we show that evaluations of one's immoral behaviour primarily induce guilt, whereas evaluations of incompetent behaviour raise anger. In Study 2, we elaborate on the psychological process associated with these emotional responses, and demonstrate that evaluations of immorality, compared to incompetence, diminish group members' perceived coping abilities, which in turn intensifies feelings of guilt. However, when anticipating an opportunity to restore one's self-image as a moral group member, perceived coping abilities are increased and the experience of guilt is alleviated. Together these studies demonstrate how group members can overcome their moral misery when restoring their self-image.

Impact of Educational Level on Study Attrition and Evaluation of Web-Based Computer-Tailored Interventions: Results From Seven Randomized Controlled Trials.

PubMed

Reinwand, Dominique A; Crutzen, Rik; Elfeddali, Iman; Schneider, Francine; Schulz, Daniela Nadine; Smit, Eline Suzanne; Stanczyk, Nicola Esther; Tange, Huibert; Voncken-Brewster, Viola; Walthouwer, Michel Jean Louis; Hoving, Ciska; de Vries, Hein

2015-10-07

Web-based computer-tailored interventions have shown to be effective in improving health behavior; however, high dropout attrition is a major issue in these interventions. The aim of this study is to assess whether people with a lower educational level drop out from studies more frequently compared to people with a higher educational level and to what extent this depends on evaluation of these interventions. Data from 7 randomized controlled trials of Web-based computer-tailored interventions were used to investigate dropout rates among participants with different educational levels. To be able to compare higher and lower educated participants, intervention evaluation was assessed by pooling data from these studies. Logistic regression analysis was used to assess whether intervention evaluation predicted dropout at follow-up measurements. In 3 studies, we found a higher study dropout attrition rate among participants with a lower educational level, whereas in 2 studies we found that middle educated participants had a higher dropout attrition rate compared to highly educated participants. In 4 studies, no such significant difference was found. Three of 7 studies showed that participants with a lower or middle educational level evaluated the interventions significantly better than highly educated participants ("Alcohol-Everything within the Limit": F2,376=5.97, P=.003; "My Healthy Behavior": F2,359=5.52, P=.004; "Master Your Breath": F2,317=3.17, P=.04). One study found lower intervention evaluation by lower educated participants compared to participants with a middle educational level ("Weight in Balance": F2,37=3.17, P=.05). Low evaluation of the interventions was not a significant predictor of dropout at a later follow-up measurement in any of the studies. Dropout attrition rates were higher among participants with a lower or middle educational level compared with highly educated participants. Although lower educated participants evaluated the interventions better in approximately half of the studies, evaluation did not predict dropout attrition. Further research is needed to find other explanations for high dropout rates among lower educated participants.

Shuttle Ground Operations Efficiencies/Technologies (SGOE/T) study. Volume 2: Ground Operations evaluation

NASA Technical Reports Server (NTRS)

Scholz, A. L.; Hart, M. T.; Lowry, D. J.

1987-01-01

The Ground Operations Evaluation describes the breath and depth of the various study elements selected as a result of an operational analysis conducted during the early part of the study. Analysis techniques used for the evaluation are described in detail. Elements selected for further evaluation are identified; the results of the analysis documented; and a follow-on course of action recommended. The background and rationale for developing recommendations for the current Shuttle or for future programs is presented.

Health services research evaluation principles. Broadening a general framework for evaluating health information technology.

PubMed

Sockolow, P S; Crawford, P R; Lehmann, H P

2012-01-01

Our forthcoming national experiment in increased health information technology (HIT) adoption funded by the American Recovery and Reinvestment Act of 2009 will require a comprehensive approach to evaluating HIT. The quality of evaluation studies of HIT to date reveals a need for broader evaluation frameworks that limits the generalizability of findings and the depth of lessons learned. Develop an informatics evaluation framework for health information technology (HIT) integrating components of health services research (HSR) evaluation and informatics evaluation to address identified shortcomings in available HIT evaluation frameworks. A systematic literature review updated and expanded the exhaustive review by Ammenwerth and deKeizer (AdK). From retained studies, criteria were elicited and organized into classes within a framework. The resulting Health Information Technology Research-based Evaluation Framework (HITREF) was used to guide clinician satisfaction survey construction, multi-dimensional analysis of data, and interpretation of findings in an evaluation of a vanguard community health care EHR. The updated review identified 128 electronic health record (EHR) evaluation studies and seven evaluation criteria not in AdK: EHR Selection/Development/Training; Patient Privacy Concerns; Unintended Consequences/ Benefits; Functionality; Patient Satisfaction with EHR; Barriers/Facilitators to Adoption; and Patient Satisfaction with Care. HITREF was used productively and was a complete evaluation framework which included all themes that emerged. We can recommend to future EHR evaluators that they consider adding a complete, research-based HIT evaluation framework, such as HITREF, to their evaluation tools suite to monitor HIT challenges as the federal government strives to increase HIT adoption.

Consider the source: persuasion of implicit evaluations is moderated by source credibility.

PubMed

Smith, Colin Tucker; De Houwer, Jan; Nosek, Brian A

2013-02-01

The long history of persuasion research shows how to change explicit, self-reported evaluations through direct appeals. At the same time, research on how to change implicit evaluations has focused almost entirely on techniques of retraining existing evaluations or manipulating contexts. In five studies, we examined whether direct appeals can change implicit evaluations in the same way as they do explicit evaluations. In five studies, both explicit and implicit evaluations showed greater evidence of persuasion following information presented by a highly credible source than a source low in credibility. Whereas cognitive load did not alter the effect of source credibility on explicit evaluations, source credibility had an effect on the persuasion of implicit evaluations only when participants were encouraged and able to consider information about the source. Our findings reveal the relevance of persuasion research for changing implicit evaluations and provide new ideas about the processes underlying both types of evaluation.

Evaluative judgments are based on evaluative information: Evidence against meaning change in evaluative context effects.

PubMed

Kaplan, M F

1975-07-01

Trait adjectives commonly employed in person perception studies have both evaluative and denotative meanings. Evaluative ratings of single traits shift with variations in the context of other traits ascribed to the stimulus person; the extent to which denotative changes underlie these evaluative context effects has been a theoretical controversy. In the first experiment, it was shown that context effects on quantitative ratings of denotation can be largely accounted for by evaluative halo effects. In the second experiment, increasing the denotative relatedness of context traits to the test trait didnot increase the effect of the context. Only the evaluative meaning of the context affected evaluation of the rated test trait. These studies suggest that the denotative relationship between a test adjective and its context has little influence on context effects in person perception, and that denotative meaning changes do not mediate context effects. Instead, evaluative judgments appear to be based on evaluative meaning.

A Systematic Review of the Level of Evidence in Economic Evaluations of Medical Devices: The Example of Vertebroplasty and Kyphoplasty

PubMed Central

van den Brink, Hélène; Pineau, Judith; Prognon, Patrice; Borget, Isabelle

2015-01-01

Context Economic evaluations are far less frequently reported for medical devices than for drugs. In addition, little is known about the quality of existing economic evaluations, particularly for innovative devices, such as those used in vertebroplasty and kyphoplasty. Objective To assess the level of evidence provided by the available economic evaluations for vertebroplasty and kyphoplasty. Data Sources A systematic review of articles in English or French listed in the MEDLINE, PASCAL, COCHRANE and National Health Service Economic Evaluation databases, with limits on publication date (up to the date of the review, March 2014). Study Selection We included only economic evaluations of vertebroplasty or kyphoplasty. Editorial and methodological articles were excluded. Data Extraction Data were extracted from articles by two authors working independently and using two analysis grids to measure the quality of economic evaluations. Data Synthesis Twenty-one studies met our inclusion criteria. All were published between 2008 and 2014. Eighteen (86%) were full economic evaluations. Cost-effectiveness analysis (CEA) was the most frequent type of economic evaluation, and was present in 11 (52%) studies. Only three CEAs complied fully with the British Medical Journal checklist. The quality of the data sources used in the 21 studies was high, but the CEAs conforming to methodological guidelines did not use high-quality data sources for all components of the analysis. Conclusions This systematic review shows that the level of evidence in economic evaluations of vertebroplasty and kyphoplasty is low, despite the recent publication of a large number of studies. This finding highlights the challenges to be faced to improve the quality of economic evaluations of medical devices. PMID:26661078

Are cardiovascular disease risk assessment and management programmes cost effective? A systematic review of the evidence.

PubMed

Lee, John Tayu; Lawson, Kenny D; Wan, Yizhou; Majeed, Azeem; Morris, Stephen; Soljak, Michael; Millett, Christopher

2017-06-01

The World Health Organization recommends that countries implement population-wide cardiovascular disease (CVD) risk assessment and management programmes. The aim of this study was to conduct a systematic review to evaluate whether this recommendation is supported by cost-effectiveness evidence. Published economic evaluations were identified via electronic medical and social science databases (including Medline, Web of Science, and the NHS Economic Evaluation Database) from inception to March 2016. Study quality was evaluated using a modified version of the Consolidated Health Economic Evaluation Reporting Standards. Fourteen economic evaluations were included: five studies based on randomised controlled trials, seven studies based on observational studies and two studies using hypothetical modelling synthesizing secondary data. Trial based studies measured CVD risk factor changes over 1 to 3years, with modelled projections of longer term events. Programmes were either not, or only, cost-effective under non-verified assumptions such as sustained risk factor changes. Most observational and hypothetical studies suggested programmes were likely to be cost-effective; however, study deigns are subject to bias and subsequent empirical evidence has contradicted key assumptions. No studies assessed impacts on inequalities. In conclusion, recommendations for population-wide risk assessment and management programmes lack a robust, real world, evidence basis. Given implementation is resource intensive there is a need for robust economic evaluation, ideally conducted alongside trials, to assess cost effectiveness. Further, the efficiency and equity impact of different delivery models should be investigated, and also the combination of targeted screening with whole population interventions recognising that there multiple approaches to prevention. Copyright © 2017. Published by Elsevier Inc.

Crescent Evaluation : appendix B : state case study evaluation report

DOT National Transportation Integrated Search

1994-02-01

The state case study evaluation approach uniquely captured an understanding of the potential of such a system by documenting the experiences, issues, and opportunities of selected key state government personnel from a cross-section of involved agenci...

NORTHERN OHIO AEROSOL STUDY: STAPHYLOCOCCUS AUREUS EVALUATION

EPA Science Inventory

A consortium of Universities, located in northwest Ohio have received funds to conduct a comprehensive evaluation of land applied biosolids in that state. This USDA funded study includes observing land application practices and evaluating biosolids, soils, runoff water and bioaer...

Anticipating evaluative social interactions involving persons with disabilities.

PubMed

Santuzzi, Alecia M

2011-08-01

Similar to other types of social interactions, the expected outcomes of interactions between persons with and without disabilities likely are influenced by global self-esteem such that individuals with high self-esteem should fare better than those with low self-esteem. The present set of laboratory studies examined whether simple and complex evaluative situations affect the role of self-esteem when anticipating interactions with individuals who have a physical disability. In Study 1, participants completed a measure of global self-esteem and then were randomly assigned to expect either a brief (5 minutes) or extended (45 minutes) interaction with a confederate in a wheelchair. Participants then completed measures of state affect and meta-evaluations (expected evaluations from the partner). Study 2 replicated the design of Study 1 and additionally randomly assigned participants to be outcome-dependent on the partner or not. Study 1 found that individuals reported affective experiences and expected meta-evaluations that were consistent with global self-esteem when expecting a brief interaction but not an extended interaction with the partner. Study 2 replicated and extended the results from Study 1, demonstrating that self-esteem resumed a priority when participants expected an extended interaction that also placed the participant in an outcome-dependent role. Taken together, the results seem to support a dilution of the role of self-esteem in simple evaluative situations but not in situations that are nonevaluative or situations with combined evaluative factors.

Descriptive analysis of context evaluation instrument for technical oral presentation skills evaluation: A case study in English technical communication course

NASA Astrophysics Data System (ADS)

Mohamed, Abdullah-Adnan; Asmawi, Adelina; Hamid, Mohd Rashid Ab; Mustafa, Zainol bin

2015-02-01

This paper reports a pilot study of Context Evaluation using a self-developed questionnaire distributed among engineering undergraduates at a university under study. The study aims to validate the self-developed questionnaires used in the Context evaluation, a component in the CIPP Model. The Context evaluation assesses background information for needs, assets, problems and opportunities relevant to beneficiaries of the study in a defined environment. Through the questionnaire, background information for the assessment of needs, assets and problems related to the engineering undergraduates' perceptions on the teaching and learning of technical oral presentation skills was collected and analysed. The questionnaire was developed using 5-points Likert scale to measure the constructs under study. They were distributed to 100 respondents with 79 returned. The respondents consisted of engineering undergraduates studied at various faculties at one technical university in Malaysia. The descriptive analysis of data for each item which makes up the construct for Context evaluation is found to be high. This implied that engineering undergraduates showed high interest in teaching and learning of technical oral presentation skills, thus their needs are met. Also, they agreed that assets and facilities are conducive to their learning. In conclusion, the context evaluation involving needs and assets factors are both considerably important; their needs are met and the assets and facilities do support their technical oral presentation skills learning experience.

Integrating service development with evaluation in telehealthcare: an ethnographic study.

PubMed

Finch, Tracy; May, Carl; Mair, Frances; Mort, Maggie; Gask, Linda

2003-11-22

To identify issues that facilitate the successful integration of evaluation and development of telehealthcare services. Ethnographic study using various qualitative research techniques to obtain data from several sources, including in-depth semistructured interviews, project steering group meetings, and public telehealthcare meetings. Seven telehealthcare evaluation projects (four randomised controlled trials and three pragmatic service evaluations) in the United Kingdom, studied over two years. Projects spanned a range of specialties-dermatology, psychiatry, respiratory medicine, cardiology, and oncology. Clinicians, managers, technical experts, and researchers involved in the projects. Key problems in successfully integrating evaluation and service development in telehealthcare are, firstly, defining existing clinical practices (and anticipating changes) in ways that permit measurement; secondly, managing additional workload and conflicting responsibilities brought about by combining clinical and research responsibilities (including managing risk); and, thirdly, understanding various perspectives on effectiveness and the limitations of evaluation results beyond the context of the research study. Combined implementation and evaluation of telehealthcare systems is complex, and is often underestimated. The distinction between quantitative outcomes and the workability of the system is important for producing evaluative knowledge that is of practical value. More pragmatic approaches to evaluation, that permit both quantitative and qualitative methods, are required to improve the quality of such research and its relevance for service provision in the NHS.

Effect of odour on multisensory environmental evaluations of road traffic

DOE Office of Scientific and Technical Information (OSTI.GOV)

Jiang, Like, E-mail: jianglike@yahoo.com; Masullo, Massimiliano, E-mail: Massimiliano.MASULLO@unina2.it; Maffei, Luigi, E-mail: luigi.maffei@unina2.it

This study investigated the effect of odour on multisensory environmental evaluations of road traffic. The study aimed to answer: (1) Does odour have any effect on evaluations on noise, landscape and the overall environment? (2) How different are participants' responses to odour stimuli and are these differences influential on the evaluations? Experimental scenarios varied in three Traffic levels, three Tree screening conditions and two Odour presence conditions were designed, and presented to participants in virtual reality. Perceived Loudness, Noise Annoyance, Landscape Quality and Overall Pleasantness of each scenario were evaluated and the results were analysed. It shows that Odour presencemore » did not have significant main effect on any of the evaluations, but has significant interactions with Traffic level on Noise Annoyance and with Tree screening on Landscape Quality, indicating the potential of odour to modulate noise and visual landscape perceptions in specific environmental content. Concerning participants' responses to odour stimuli, large differences were found in this study. However, the differences did not seem to be influential on environmental evaluations in this study. Larger samples of participants may benefit this study for more significant results of odour effect.« less

The Influence of Evaluators' Principles on Evaluation Resource Decisions

ERIC Educational Resources Information Center

Crohn, Kara Shea Davis

2009-01-01

This study examines ways in which evaluators' principles influence decisions about evaluation resources. Evaluators must seek-out and allocate (often scarce) resources (e.g., money, time, data, people, places) in a way that allows them to conduct the best possible evaluation given clients' and evaluation participants' constraints. Working within…

Health information systems evaluation frameworks: A systematic review.

PubMed

Eslami Andargoli, Amirhossein; Scheepers, Helana; Rajendran, Diana; Sohal, Amrik

2017-01-01

Evaluation of health information systems (HISs) is complicated because of the complex nature of the health care domain. Various studies have proposed different frameworks to reduce the complexity in the assessment of these systems. The aim of these frameworks is to provide a set of guidelines for the evaluation of the adequacy of health care information systems. This paper aims to analyse studies on the evaluation of HISs by applying a content, context and process (CCP) framework to address the 'who', 'what', 'how', 'when', and 'why' of the evaluation processes used. This will allow for a better understanding of the relative strengths and weaknesses of various HISs evaluation frameworks, and will pave the way for developing a more complete framework for HISs. A systematic literature review on HIS evaluation studies was undertaken to identify the currently available HIS evaluation frameworks. Five academic databases were selected to conduct this systematic literature review. Most of the studies only address some, but not all, of the five main questions, i.e. the who, what, how, when, why, and that there was a lack of consensus in the way these questions were addressed. The critical role of context was also largely neglected in these studies. Evaluation of HISs is complex. The health care domain is highly context sensitive and in order to have a complete assessment of HISs, consideration of contextual factors is necessary. Specifically, to have the right set of criteria to measure the 'what', the answer to the 'who' of the evaluation is necessary. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Measurement and Design Issues in the Study of Adolescent Sexual Behavior and the Evaluation of Adolescent Sexual Health Behavior Interventions

ERIC Educational Resources Information Center

Young, Michael; Palacios, Rebecca; Penhollow, Tina M.

2012-01-01

To improve the quality of research and commentary concerning adolescent sexuality and evaluation of both comprehensive sexuality education and abstinence education programs, this article aims to help readers (1) select appropriate measures to study adolescent sexual behavior, (2) develop appropriate study designs to evaluate adolescent sexual…

A Qualitative Evaluation of the Use of Multimedia Case Studies in an Introductory Engineering Course at Two Southeastern Universities

ERIC Educational Resources Information Center

Huett, Kim C.; Kawulich, Barbara

2015-01-01

Collaborating at two universities to improve teaching and learning in undergraduate engineering, an interdisciplinary team of researchers, instructors, and evaluators planned and implemented the use of multimedia case studies with students enrolled in an introductory engineering course. This qualitative action evaluation study focuses on results…

An Evaluation of the Relationship between Classroom Practices and Mathematics Motivation from Student and Teacher Perspectives

ERIC Educational Resources Information Center

Chen, Jinsong

2011-01-01

This study evaluated the relationships between classroom practices and mathematics motivation. The evaluation was given in a specific context, namely eighth grade in U.S. middle schools. Using quantitative methods, the study adopted data from the Trends in International Mathematics and Science Study (TIMSS) 2007 and compared classroom practices…

Evaluation of Turkish Education Programs

ERIC Educational Resources Information Center

Durmuscelebi, Mustafa

2010-01-01

The aim of this study is to evaluate the new Turkish education program that has been being implemented since 2005 gradually in light of teacher suggestions. The study has been done in scanning model. In this study which has been conducted with the purpose of evaluating the newly prepared Turkish education programs, the program has been tried to be…

Case Studies of Teacher Satisfaction with Three Plans for Evaluation and Supervision

ERIC Educational Resources Information Center

Arp, Jeb-Stuart Bennett

2013-01-01

The purpose of this study was to identify which plans for evaluation and supervision are most satisfactory to teachers. The study's participants come from three school systems in northwest Georgia. Each school district uses a different plan for evaluation and supervision; each claiming that the end goal is to improve teacher performance. Prior…

78 FR 68461 - Guidance for Industry: Studies To Evaluate the Utility of Anti-Salmonella Chemical Food Additives...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-11-14

...] Guidance for Industry: Studies To Evaluate the Utility of Anti- Salmonella Chemical Food Additives in Feeds... Industry: Studies to Evaluate the Utility of Anti-Salmonella Chemical Food Additives in Feeds,'' and is... of Anti-Salmonella Chemical Food Additives in Feeds (GFI 80) is to help sponsors design efficacy...

A Multi-Stage Longitudinal Comparative Design Stage II Evaluation of the Changing Lives Program: The Life Course Interview (RDA-LCI)

ERIC Educational Resources Information Center

Arango, Lisa Lewis; Kurtines, William M.; Montgomery, Marilyn J.; Ritchie, Rachel

2008-01-01

The study reported in this article, a Multi-Stage Longitudinal Comparative Design Stage II evaluation conducted as a planned preliminary efficacy evaluation (psychometric evaluation of measures, short-term controlled outcome studies, etc.) of the Changing Lives Program (CLP), provided evidence for the reliability and validity of the qualitative…

The Impact of Disciplinary Background and Teaching Experience on the Use of Evaluative Language in Teacher Feedback

ERIC Educational Resources Information Center

Hu, Guangwei; Choo, Lilin

2016-01-01

This study was designed to examine secondary teachers' use of evaluative language resources in their qualitative written feedback on student work and factors shaping the deployment of such resources. Drawing on appraisal theory as an analytic framework for the language of evaluation, the study analyzed 84 teachers' evaluative reports on their…

Use and Non-use of Evaluation Results: Case Study of Environmental Influences in the Private Sector.

ERIC Educational Resources Information Center

Russ-Eft, Darlene; Atwood, Regina; Egherman, Tori

2002-01-01

Studied the evaluation of a sale program within a business services organization. Interviewed 23 sales people, 10 implementation specialists, 9 consultants, and 40 clients of the business. Results show the use and nonuse of evaluation results and point out the factors contributing to use or nonuse of evaluation results in this private sector…

An Examination of Performance-Based Teacher Evaluation Systems in Five States. Summary. Issues & Answers. REL 2012-No. 129

ERIC Educational Resources Information Center

Shakman, Karen; Riordan, Julie; Sanchez, Maria Teresa; Cook, Kyle DeMeo; Fournier, Richard; Brett, Jessica

2012-01-01

This study reports on performance-based teacher evaluation systems in five states that have implemented such systems. It investigates two primary research questions: (1) What are the key characteristics of state-level performance-based teacher evaluation systems in the study states?; and (2) How do state teacher evaluation measures, the teaching…

Teaching Evaluation: Antecedents of Teachers' Perceived Usefulness of Follow-Up Sessions and Perceived Stress Related to the Evaluation Process

ERIC Educational Resources Information Center

Lejonberg, Eli; Elstad, Eyvind; Christophersen, Knut Andreas

2018-01-01

This study examines teaching evaluation based on student feedback which is intended to contribute to teachers' professional development. Although studies have highlighted that teaching evaluation is a potential source of professional development and that follow-up sessions are crucial to this benefit, little attention has been paid to the…

Teacher Attitudes as Related to the Implementation of the New Jersey Teacher Evaluation System

ERIC Educational Resources Information Center

Gallo, Donna

2016-01-01

The focus of this qualitative case study is on an attitudinal change of teachers towards teacher evaluation with the implementation of a new evaluation system. The goal of this qualitative case study was to gain an understanding of how the implementation of the new New Jersey teacher evaluation system relates to teachers' attitudes toward…

Software Quality Evaluation Models Applicable in Health Information and Communications Technologies. A Review of the Literature.

PubMed

Villamor Ordozgoiti, Alberto; Delgado Hito, Pilar; Guix Comellas, Eva María; Fernandez Sanchez, Carlos Manuel; Garcia Hernandez, Milagros; Lluch Canut, Teresa

2016-01-01

Information and Communications Technologies in healthcare has increased the need to consider quality criteria through standardised processes. The aim of this study was to analyse the software quality evaluation models applicable to healthcare from the perspective of ICT-purchasers. Through a systematic literature review with the keywords software, product, quality, evaluation and health, we selected and analysed 20 original research papers published from 2005-2016 in health science and technology databases. The results showed four main topics: non-ISO models, software quality evaluation models based on ISO/IEC standards, studies analysing software quality evaluation models, and studies analysing ISO standards for software quality evaluation. The models provide cost-efficiency criteria for specific software, and improve use outcomes. The ISO/IEC25000 standard is shown as the most suitable for evaluating the quality of ICTs for healthcare use from the perspective of institutional acquisition.

ENGINEERING STUDY FOR THE 200 AREA EFFLUENT TREATMENT FACILITY (ETF) SECONDARY WASTE TREATMENT OF PROJECTED FUTURE WASTE FEEDS

DOE Office of Scientific and Technical Information (OSTI.GOV)

LUECK, K.J.

2004-10-18

This report documents an engineering study conducted to evaluate alternatives for treating secondary waste in the secondary treatment train (STT) of the Hanford Site 200 Area Effluent Treatment Facility (ETF). The study evaluates ETF STT treatment alternatives and recommends preferred alternatives for meeting the projected future missions of the ETF. The preferred alternative(s) will process projected future ETF influents to produce a solid waste acceptable for final disposal on the Hanford Site. The main text of this report summarizes the ETF past and projected operations, lists the assumptions about projected operations that provide the basis for the engineering evaluation, andmore » summarizes the evaluation process. The evaluation process includes identification of available modifications to the current ETF process, screens those modifications for technical viability, evaluates the technically viable alternatives, and provides conclusions and recommendations based on that evaluation.« less

Diagnosing Alzheimer's disease: a systematic review of economic evaluations.

PubMed

Handels, Ron L H; Wolfs, Claire A G; Aalten, Pauline; Joore, Manuela A; Verhey, Frans R J; Severens, Johan L

2014-03-01

The objective of this study is to systematically review the literature on economic evaluations of interventions for the early diagnosis of Alzheimer's disease (AD) and related disorders and to describe their general and methodological characteristics. We focused on the diagnostic aspects of the decision models to assess the applicability of existing decision models for the evaluation of the recently revised diagnostic research criteria for AD. PubMed and the National Institute for Health Research Economic Evaluation database were searched for English-language publications related to economic evaluations on diagnostic technologies. Trial-based economic evaluations were assessed using the Consensus on Health Economic Criteria list. Modeling studies were assessed using the framework for quality assessment of decision-analytic models. The search retrieved 2109 items, from which eight decision-analytic modeling studies and one trial-based economic evaluation met all eligibility criteria. Diversity among the study objective and characteristics was considerable and, despite considerable methodological quality, several flaws were indicated. Recommendations were focused on diagnostic aspects and the applicability of existing models for the evaluation of recently revised diagnostic research criteria for AD. Copyright © 2014 The Alzheimer's Association. Published by Elsevier Inc. All rights reserved.

Evaluating quantity and quality of literature focusing on health economics and pharmacoeconomics in Gulf Cooperation Council countries.

PubMed

Eljilany, Islam; El-Dahiyat, Faris; Curley, Louise Elizabeth; Babar, Zaheer-Ud-Din

2018-05-30

The importance of pharmacoeconomics and health economics has been augmented. It has the potential to provide evidence to aid in optimal decision-making in the funding of cost-effective medicines and services in Gulf Cooperation Council countries (G.C.C). To evaluate the quality and quantity of health economic researches published until the end of 2017 in G.C.C. and to identify the factors that affect the quality of studies. Studies were included according to predefined inclusion and exclusion criteria. The quantity was recorded, and the quality was assessed using the Quality of Health Economic Studies (QHES) instrument. Forty-nine studies were included. The mean (SD) quality score of all studies was 57.83 (25.05), and a high number of reviewed studies (47%) were evaluated as either poor or extremely poor quality. The factors that affect the quality of studies with statistical significance were, the type and method of economic evaluation, the economic outcome was the objective of the research, author`s background, the perspective of the study, health intervention and source of funding. The use of economic evaluation studies in G.C.C was limited. Different factors that affect the quality of articles such as performing a full economic evaluation and choosing societal perspective were identified. Strategies to improve the quality of future studies were recommended.

Current status of quality evaluation of nursing care through director review and reflection from the Nursing Quality Control Centers

PubMed Central

Duan, Xia; Shi, Yan

2014-01-01

Background: The quality evaluation of nursing care is a key link in medical quality management. It is important and worth studying for the nursing supervisors to know the disadvantages during the process of quality evaluation of nursing care and then to improve the whole nursing quality. This study was to provide director insight on the current status of quality evaluation of nursing care from Nursing Quality Control Centers (NQCCs). Material and Methods: This qualitative study used a sample of 12 directors from NQCCs who were recruited from 12 provinces in China to evaluate the current status of quality evaluation of nursing care. Data were collected by in-depth interviews. Content analysis method was used to analyze the data. Results: Four themes emerged from the data: 1) lag of evaluation index; 2) limitations of evaluation content; 3) simplicity of evaluation method; 4) excessive emphasis on terminal quality. Conclusion: It is of great realistic significance to ameliorate nursing quality evaluation criteria, modify the evaluation content based on patient needs-oriented idea, adopt scientific evaluation method to evaluate nursing quality, and scientifically and reasonably draw horizontal comparisons of nursing quality between hospitals, as well as longitudinal comparisons of a hospital’s nursing quality. These methods mentioned above can all enhance a hospital’s core competitiveness and benefit more patients. PMID:25419427

Case study of evaluations that go beyond clinical outcomes to assess quality improvement diabetes programmes using the Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE)

PubMed Central

Harris, Stewart B.; Naqshbandi Hayward, Mariam; Tompkins, Jordan W.

2016-01-01

Abstract Rationale, aims and objectives Investments in efforts to reduce the burden of diabetes on patients and health care are critical; however, more evaluation is needed to provide evidence that informs and supports future policies and programmes. The newly developed Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE) incorporates the theoretical concepts needed to facilitate the capture of critical information to guide investments, policy and programmatic decision making. The aim of the study is to assess the applicability and value of DEFINE in comprehensive real‐world evaluation. Method Using a critical and positivist approach, this intrinsic and collective case study retrospectively examines two naturalistic evaluations to demonstrate how DEFINE could be used when conducting real‐world comprehensive evaluations in health care settings. Results The variability between the cases and the evaluation designs are described and aligned to the DEFINE goals, steps and sub‐steps. The majority of the theoretical steps of DEFINE were exemplified in both cases, although limited for knowledge translation efforts. Application of DEFINE to evaluate diverse programmes that target various chronic diseases is needed to further test the inclusivity and built‐in flexibility of DEFINE and its role in encouraging more comprehensive knowledge translation. Conclusions This case study shows how DEFINE could be used to structure or guide comprehensive evaluations of programmes and initiatives implemented in health care settings and support scale‐up of successful innovations. Future use of the framework will continue to strengthen its value in guiding programme evaluation and informing health policy to reduce the burden of diabetes and other chronic diseases. PMID:26804339

Evaluating markers for the early detection of cancer: overview of study designs and methods.

PubMed

Baker, Stuart G; Kramer, Barnett S; McIntosh, Martin; Patterson, Blossom H; Shyr, Yu; Skates, Steven

2006-01-01

The field of cancer biomarker development has been evolving rapidly. New developments both in the biologic and statistical realms are providing increasing opportunities for evaluation of markers for both early detection and diagnosis of cancer. To review the major conceptual and methodological issues in cancer biomarker evaluation, with an emphasis on recent developments in statistical methods together with practical recommendations. We organized this review by type of study: preliminary performance, retrospective performance, prospective performance and cancer screening evaluation. For each type of study, we discuss methodologic issues, provide examples and discuss strengths and limitations. Preliminary performance studies are useful for quickly winnowing down the number of candidate markers; however their results may not apply to the ultimate target population, asymptomatic subjects. If stored specimens from cohort studies with clinical cancer endpoints are available, retrospective studies provide a quick and valid way to evaluate performance of the markers or changes in the markers prior to the onset of clinical symptoms. Prospective studies have a restricted role because they require large sample sizes, and, if the endpoint is cancer on biopsy, there may be bias due to overdiagnosis. Cancer screening studies require very large sample sizes and long follow-up, but are necessary for evaluating the marker as a trigger of early intervention.

A framework for evaluating e-health: Systematic review of studies assessing the quality of health information and services for patients on the Internet

PubMed Central

Eysenbach, Gunther

2000-01-01

Context A recent concern and topic of many publications in the last three years has been the quality of health information and services for the public on the Internet. Objectives To identify and summarize studies published in the peer-reviewed literature evaluating the quality of information and services for consumers on the Internet, including information published on web sites, information on newsgroups and mailing lists and other venues such as email contacts with doctors, as well as studies evaluating the quality of ehealth services such as cyberdoctors and cyberpharmacies. Data Sources MEDLINE and PREMEDLINE (1966 - May 2000), Science Citation Index (1992-May 2000), Social Sciences Citation Index (1992- May 2000), Arts and Humanities Citation Index (1992-May 2000) and a personal bibliographic database. Study Selection We included empirical studies where investigators searched the Internet systematically for specific health information or clearly define a set of specific services to be included, evaluated the quality of information or services found, and reported quantitative data. Data Extraction Study characteristics, medical domain, search strategies used, quality criteria and methodology of quality assessment, results (number of sites rated as sufficient pertaining to a quality), quality and rigor of study methodology and reporting. Data Synthesis A total of 41 studies met the inclusion criteria, dealing either with content of websites, information on e-commerce sites, quality of online-care or community venues. A) Content: 29 evaluated information on websites, of those 5 evaluated information on websites from the field of pediatrics, 3 from oncology, 3 pharmacology information, 2 nutrition information, 4 general clinical information and 12 specific information from other clinical disciplines. Studies varied widely in methodology, quality and results. Among the 29 studies dealing with quality of health information on websites, one study evaluated the authority of source, 19 studies checked sites for presence or absences of technical criteria (such as disclosure of sponsorship, authorship, presence of references or last update), three evaluated readability, 20 evaluated the accuracy of information, and 12 content completeness. B) E-Commerce: Three studies dealt with drug information on e-commerce sites. To evaluate the quality authors extracted prices, checked for completeness of online-history taking and/or information provided on the site e.g. pertaining to contraindications or the presence/absence of disclaimers and/or liability waivers. Only one of these three studies actually used the service by ordering drugs, allowing to evaluate quality of online-advice, qualification of cyberdocs, delivery time, reasons for non-delivery. C) Care: Five studies dealt with the quality of online consultations, of them one dealt with advice given by ordinary physicians in response to a unsolicited fictionous patient request, two evaluated the responses of cyberdocs soliciting patient requests, and two evaluated advice given in response to a request for an online prescription directed to drug e-commerce sites. D) Community: Seven studies evaluated messages on mailing lists or usenet newsgroups. Two studies collected messages from a venue and evaluated them for accuracy, two studies posted a "test" message on a newsgroup and evaluated the accuracy of responses. In three studies authors used the cumulative impact factor of the published research of the mailing list contributors as an indicator for the qualification of the authors and thus as quality indicator of the mailing list. Of the 23 studies evaluating accuracy and/or completeness of information provided on websites, 10 used guidelines as gold standard (all of the to extract a-priori criteria), five used peer-reviewed literature (all to compare information a posteriori), two used textbooks, one used consensus among the raters and three used the personal experience or opinion of the author. All three studies which compared information against personal opinion came to a positive conclusion regarding the accuracy of information, while all of the more rigorous studies comparing information against guidelines concluded that much of the information found on the web were of low quality. In two cases it was not reported where the gold standard comes from. 9 studies used more than one rater to assess a website, 6 of them provided some sort of information on inter-observer variability. Conclusions Methodology, results and conclusions of investigators vary widely. There is a wide variability even regarding the evaluation of formal criteria such as authorship and references. Differences in study conclusions regarding the quality of Internet information are likely a result of difference in study rigor, evaluation criteria and topic chosen. All but five studies concluded that quality is a problem on the Internet. The rigor of these five studies coming to a more positive conclusion as expressed in a "assessment quality score" was significantly inferior to the remaining studies. Although there were two comparative studies of the quality of information on the Internet compared with information found outside of the Internet, there is little evidence that health information found on the Internet is worse than health information in traditional media. A conceptual and methodological framework is presented for describing, comparing, and analyzing the structure and quality of e-health, based on Donabedians quality measures of structure, process and outcome.

A Critical Review of Methods to Evaluate the Impact of FDA Regulatory Actions

PubMed Central

Briesacher, Becky A.; Soumerai, Stephen B.; Zhang, Fang; Toh, Sengwee; Andrade, Susan E.; Wagner, Joann L.; Shoaibi, Azadeh; Gurwitz, Jerry H.

2013-01-01

Purpose To conduct a synthesis of the literature on methods to evaluate the impacts of FDA regulatory actions, and identify best practices for future evaluations. Methods We searched MEDLINE for manuscripts published between January 1948 and August 2011 that included terms related to FDA, regulatory actions, and empirical evaluation; the review additionally included FDA-identified literature. We used a modified Delphi method to identify preferred methodologies. We included studies with explicit methods to address threats to validity, and identified designs and analytic methods with strong internal validity that have been applied to other policy evaluations. Results We included 18 studies out of 243 abstracts and papers screened. Overall, analytic rigor in prior evaluations of FDA regulatory actions varied considerably; less than a quarter of studies (22%) included control groups. Only 56% assessed changes in the use of substitute products/services, and 11% examined patient health outcomes. Among studies meeting minimal criteria of rigor, 50% found no impact or weak/modest impacts of FDA actions and 33% detected unintended consequences. Among those studies finding significant intended effects of FDA actions, all cited the importance of intensive communication efforts. There are preferred methods with strong internal validity that have yet to be applied to evaluations of FDA regulatory actions. Conclusions Rigorous evaluations of the impact of FDA regulatory actions have been limited and infrequent. Several methods with strong internal validity are available to improve trustworthiness of future evaluations of FDA policies. PMID:23847020

Urban pedestrian accident countermeasures experimental evaluation. Volume 1, Behavioral evaluation studies

DOT National Transportation Integrated Search

1975-02-01

A series of site and accident specific pedestrian safety countermeasures had been developed in a previous study, but the effectiveness of these countermeasures had not been empirically evaluated. This project focused on the determination of the effec...

78 FR 6329 - Disease, Disability, and Injury Prevention and Control Special Emphasis Panel (SEP): Initial Review

Federal Register 2010, 2011, 2012, 2013, 2014

2013-01-30

... announced below concerns Birth Defects Study to Evaluate Pregnancy exposureS (BD-STEPS), FOA DD13-003... evaluation of applications received in response to ``Birth Defects Study to Evaluate Pregnancy exposureS (BD...

Safety evaluation of STOP AHEAD pavement markings

DOT National Transportation Integrated Search

2007-12-01

The Federal Highway Administration (FHWA) organized a Pooled Fund Study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. One of the strategies chosen to be evaluated for this study was STOP AHEAD pav...

Safety evaluation of STOP AHEAD pavement markings

DOT National Transportation Integrated Search

2008-03-01

The Federal Highway Administration (FHWA) organized a Pooled Fund Study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. One of the strategies chosen to be evaluated for this study was STOP AHEAD pav...

Safety evaluation of wet-reflective pavement markings : tech brief.

DOT National Transportation Integrated Search

2015-12-01

The Federal Highway Administration (FHWA) organized : 38 States for the FHWA Evaluation of Low-Cost Safety : Improvements Pooled Fund Study as part of its strategic : highway safety plan support effort. The purpose of the : study is to evaluate the s...

Review of existing experimental approaches for the clinical evaluation of the benefits of plant food supplements on cardiovascular function.

PubMed

Meoni, Paolo; Restani, Patrizia; Mancama, Dalu T

2013-06-01

We conducted a survey of the National Centre for Biotechnology Information (NCBI) PubMed database to identify methods most commonly used for the evaluation of the effect of plant food supplements on the cardiovascular system and their relevance to the regulatory status of these products. Particularly, our search strategy was aimed at the selection of studies concerning the clinical evaluation of the beneficial effects of the most commonly studied plant food supplements acting on the cardiovascular system. Following the screening of 3839 papers for inclusion criteria, 48 published reports were retained for this review. Most studies included in this review used a double blind controlled design, and evaluated the effect of plant food supplements on individuals affected by a disease of the cardiovascular system. The majority of the studies were found to be of low methodological quality on the Jadad scale, mainly because of inadequate reporting of adverse events and of patient withdrawals. In comparison, measures used for the evaluation of benefits included mostly cardiovascular risk factors as recommended in international guidelines and in accordance with principles laid down for the evaluation of health claims in food. The risk factors most frequently evaluated belonged to the category of "lipid function and levels", "heart function" and "blood pressure". For the absolute majority of the studies, the study period did not exceed one month. This review highlights critical factors to be considered in the design of studies evaluating the health effects of plant food supplements on the cardiovascular system. Between others, the inclusion of healthy individuals, better reporting and description of the characteristics of the product used could improve the quality and relevance of these studies.

Pharmacy Information Systems in Teaching Hospitals: A Multi-dimensional Evaluation Study.

PubMed

Kazemi, Alireza; Rabiei, Reza; Moghaddasi, Hamid; Deimazar, Ghasem

2016-07-01

In hospitals, the pharmacy information system (PIS) is usually a sub-system of the hospital information system (HIS). The PIS supports the distribution and management of drugs, shows drug and medical device inventory, and facilitates preparing needed reports. In this study, pharmacy information systems implemented in general teaching hospitals affiliated to medical universities in Tehran (Iran) were evaluated using a multi-dimensional tool. This was an evaluation study conducted in 2015. To collect data, a checklist was developed by reviewing the relevant literature; this checklist included both general and specific criteria to evaluate pharmacy information systems. The checklist was then validated by medical informatics experts and pharmacists. The sample of the study included five PIS in general-teaching hospitals affiliated to three medical universities in Tehran (Iran). Data were collected using the checklist and through observing the systems. The findings were presented as tables. Five PIS were evaluated in the five general-teaching hospitals that had the highest bed numbers. The findings showed that the evaluated pharmacy information systems lacked some important general and specific criteria. Among the general evaluation criteria, it was found that only two of the PIS studied were capable of restricting repeated attempts made for unauthorized access to the systems. With respect to the specific evaluation criteria, no attention was paid to the patient safety aspect. The PIS studied were mainly designed to support financial tasks; little attention was paid to clinical and patient safety features.

A lifestyle intervention study targeting individuals with low socioeconomic status of different ethnic origins: important aspects for successful implementation.

PubMed

Teuscher, Dorit; Bukman, Andrea J; van Baak, Marleen A; Feskens, Edith J M; Renes, Reint Jan; Meershoek, Agnes

2017-07-25

Evaluation of the implementation process of trials is important, because the way a study is implemented modifies its outcomes. Furthermore, lessons learned during implementation can inform other researchers on factors that play a role when implementing interventions described in research. This study evaluates the implementation of the MetSLIM study, targeting individuals with low socioeconomic status of different ethnic origins. The MetSLIM study was set up to evaluate the effectiveness of a lifestyle programme on waist circumference and other cardio-metabolic risk factors. The objective of this evaluation was to identify components that were essential for the implementation of the MetSLIM study and to inform other researchers on methodological aspects when working with inadequately reached populations in health research. In this evaluation study the experiences of health professionals, study assistants, a community worker and regional research coordinators involved in the MetSLIM study were explored using semi-structured interviews. Questionnaires were used to evaluate participants' satisfaction with the lifestyle intervention. Our analyses show that a flexible recruitment protocol eventually leads to recruitment of sufficient participants; that trust in the recruiter is an important factor in the recruitment of individuals with low socioeconomic status of different ethnic origins; and that health professionals will unavoidably shape the form of intervention activities. Furthermore, our evaluation shows that daily practice and research mutually influence each other and that the results of an intervention are a product of this interaction. Health promotion research would benefit from a perspective that sees intervention activities not as fixed entities but rather as social interaction that can take on numerous forms. Analysing and reporting the implementation process of studies, like in this evaluation, will allow readers to get a detailed view on the appropriateness of the (intended) study design and intervention for the targeted population. Evaluation studies that shed light on the reasons for adaptations, rather than describing them as deviation from the original plan, would point out methodological aspects important for a study's replication. Furthermore, they would show how various factors can influence the implementation, and therewith initiate a learning cycle for the development of future intervention studies. Netherlands Trial Register NTR3721 (since November 27, 2012).

Moving Away from Exhaustion: How Core Self-Evaluations Influence Academic Burnout

PubMed Central

Lian, Penghu; Sun, Yunfeng; Ji, Zhigang; Li, Hanzhong; Peng, Jiaxi

2014-01-01

Background Academic burnout refers to students who have low interest, lack of motivation, and tiredness in studying. Studies concerning how to prevent academic burnout are rare. Objective The present study aimed to investigate the impact of core self-evaluations on the academic burnout of university students, and mainly focused on the confirmation of the mediator role of life satisfaction. Methods A total of 470 university students accomplished the core self-evaluation scale, Satisfaction with Life, and academic burnout scale. Results Both core self-evaluations and life satisfaction were significantly correlated with academic burnout. Structural equation modeling indicated that life satisfaction partially mediated the relationship between core self-evaluations and academic burnout. Conclusions Core self-evaluations significantly influence academic burnout and are partially mediated by life satisfaction. PMID:24489857

Study on the evaluation index of active power reserve

NASA Astrophysics Data System (ADS)

Guo, Xiaorui; Liu, Jiantao; Wang, Ke; Min, Lu

2018-01-01

Based on the role of active reserve at different time scales, divides the evaluation dimension of active reserve. Analysis the calculation principle of traditional reliability index such as probability of system safety, lack of power shortage and electricity shortage expectancy, and studies the applicability of these indicators to evaluate the reserve capacity on different dimensions. Resolves the evaluation index of active reserve capacity from the dimensions of time dimension, spatial dimension, system state, risk degree and economy, then construct evaluation index of active reserve capacity.

Conceptual framework for development of comprehensive e-health evaluation tool.

PubMed

Khoja, Shariq; Durrani, Hammad; Scott, Richard E; Sajwani, Afroz; Piryani, Usha

2013-01-01

The main objective of this study was to develop an e-health evaluation tool based on a conceptual framework including relevant theories for evaluating use of technology in health programs. This article presents the development of an evaluation framework for e-health programs. The study was divided into three stages: Stage 1 involved a detailed literature search of different theories and concepts on evaluation of e-health, Stage 2 plotted e-health theories to identify relevant themes, and Stage 3 developed a matrix of evaluation themes and stages of e-health programs. The framework identifies and defines different stages of e-health programs and then applies evaluation theories to each of these stages for development of the evaluation tool. This framework builds on existing theories of health and technology evaluation and presents a conceptual framework for developing an e-health evaluation tool to examine and measure different factors that play a definite role in the success of e-health programs. The framework on the horizontal axis divides e-health into different stages of program implementation, while the vertical axis identifies different themes and areas of consideration for e-health evaluation. The framework helps understand various aspects of e-health programs and their impact that require evaluation at different stages of the life cycle. The study led to the development of a new and comprehensive e-health evaluation tool, named the Khoja-Durrani-Scott Framework for e-Health Evaluation.

When Mode Does Not Matter: Evaluation in Class versus Out of Class

ERIC Educational Resources Information Center

Kordts-Freudinger, Robert; Geithner, Eva

2013-01-01

This article investigates if online evaluation leads to different results than paper-and-pencil evaluation. Given that most previous studies confound the evaluation mode (online versus paper) with the evaluation situation (in class versus after class), we expected that evaluation results would be influenced only by the evaluation situation,…

Evaluator and Program Manager Perceptions of Evaluation Capacity and Evaluation Practice

ERIC Educational Resources Information Center

Fierro, Leslie A.; Christie, Christina A.

2017-01-01

The evaluation community has demonstrated an increased emphasis and interest in evaluation capacity building in recent years. A need currently exists to better understand how to measure evaluation capacity and its potential outcomes. In this study, we distributed an online questionnaire to managers and evaluation points of contact working in…

Documenting Evaluation Use: Guided Evaluation Decisionmaking. Evaluation Productivity Project.

ERIC Educational Resources Information Center

Burry, James

This paper documents the evaluation use process among districts using the Guide for Evaluation Decision Makers, published by the Center for the Study of Evaluation (CSE) during the 1984-85 school year. Included are the following: (1) a discussion of research that led to conclusions concerning the administrator's role in evaluation use; (2) a…

Toward Better Research on--and Thinking about--Evaluation Influence, Especially in Multisite Evaluations

ERIC Educational Resources Information Center

Mark, Melvin M.

2011-01-01

Evaluation is typically carried out with the intention of making a difference in the understandings and actions of stakeholders and decision makers. The author provides a general review of the concepts of evaluation "use," evaluation "influence," and "influence pathways," with connections to multisite evaluations. The study of evaluation influence…

Informing the Discussion on Evaluator Training: A Look at Evaluators' Course Taking and Professional Practice

ERIC Educational Resources Information Center

Christie, Christina A.; Quiñones, Patricia; Fierro, Leslie

2014-01-01

This classification study examines evaluators' coursework training as a way of understanding evaluation practice. Data regarding courses that span methods and evaluation topics were collected from evaluation practitioners. Using latent class analysis, we establish four distinct classes of evaluator course-taking patterns: quantitative,…

Improving Beta Test Evaluation Response Rates: A Meta-Evaluation

ERIC Educational Resources Information Center

Russ-Eft, Darlene; Preskill, Hallie

2005-01-01

This study presents a meta-evaluation of a beta-test of a customer service training program. The initial evaluation showed a low response rate. Therefore, the meta-evaluation focused on issues related to the conduct of the initial evaluation and reasons for nonresponse. The meta-evaluation identified solutions to the nonresponse problem as related…

Does sunshine prime loyal … or summer? Effects of associative relatedness on the evaluative priming effect in the valent/neutral categorisation task.

PubMed

Werner, Benedikt; von Ramin, Elisabeth; Spruyt, Adriaan; Rothermund, Klaus

2018-02-01

After 30 years of research, the mechanisms underlying the evaluative priming effect are still a topic of debate. In this study, we tested whether the evaluative priming effect can result from (uncontrolled) associative relatedness rather than evaluative congruency. Stimuli that share the same evaluative connotation are more likely to show some degree of non-evaluative associative relatedness than stimuli that have a different evaluative connotation. Therefore, unless associative relatedness is explicitly controlled for, evaluative priming effects reported in earlier research may be driven by associative relatedness instead of evaluative relatedness. To address this possibility, we performed an evaluative priming study in which evaluative congruency and associative relatedness were manipulated independently from each other. The valent/neutral categorisation task was used to ensure evaluative stimulus processing in the absence of response priming effects. Results showed an effect of associative relatedness but no (overall) effect of evaluative congruency. Our findings highlight the importance of controlling for associative relatedness when testing for evaluative priming effects.

Teachers' Applications of Principals' Evaluative Feedback

ERIC Educational Resources Information Center

Mueller, Lori M.

2016-01-01

The purpose of this study was to identify considerations that influenced teachers' implementation of principals' evaluative feedback to improve instructional practices. The research question for this study was: What considerations contribute to teachers' applications of principals' evaluative feedback? Participants in this qualitative study…

Safety evaluation of flashing beacons at STOP-controlled intersections

DOT National Transportation Integrated Search

2008-03-01

The Federal Highway Administration (FHWA) organized a pooled fund study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. One of the strategies chosen to be evaluated for this study was flashing beaco...

Evaluation of Computer Simulations for Teaching Apparel Merchandising Concepts.

ERIC Educational Resources Information Center

Jolly, Laura D.; Sisler, Grovalynn

1988-01-01

The study developed and evaluated computer simulations for teaching apparel merchandising concepts. Evaluation results indicated that teaching method (computer simulation versus case study) does not significantly affect cognitive learning. Student attitudes varied, however, according to topic (profitable merchandising analysis versus retailing…

Safety evaluation of increasing retroreflectivity of STOP signs

DOT National Transportation Integrated Search

2008-03-01

The Federal Highway Administration (FHWA) organized a Pooled Fund Study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. One of the strategies chosen to be evaluated for this study was STOP signs wit...

A case study evaluation of the use of video technology in concrete pavement evaluation.

DOT National Transportation Integrated Search

2000-01-01

This report presents the results of an evaluation of video technology as a possible solution to the problem of safely collecting objective condition data for prioritizing concrete pavement rehabilitation needs in Virginia. The study involved the eval...

77 FR 506 - Agency Information Collection Request. 30-Day Public Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2012-01-05

.... Proposed Project: Teen Pregnancy Prevention Replication Evaluation Study: Baseline Data Collection--OMB No... for Children and Families (ACF) on adolescent pregnancy prevention evaluation activities. OAH will jointly oversee with ASPE the Teen Pregnancy Prevention Replication Evaluation Study (TPP Replication...

Safety evaluation of offset improvements for left-turn lanes

DOT National Transportation Integrated Search

2009-06-01

The Federal Highway Administration (FHWA) organized a pooled fund study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. One of the strategies chosen to be evaluated for this study was offset improve...

Evaluating the abuse potential of opioids and abuse-deterrent -opioid formulations: A review of clinical study methodology.

PubMed

Setnik, Beatrice; Schoedel, Kerri A; Levy-Cooperman, Naama; Shram, Megan; Pixton, Glenn C; Roland, Carl L

With the development of opioid abuse-deterrent formulations (ADFs), there is a need to conduct well-designed human abuse potential studies to evaluate the effectiveness of their deterrent properties. Although these types of studies have been conducted for many years, largely to evaluate inherent abuse potential of a molecule and inform drug scheduling, methodological approaches have varied across studies. The focus of this review is to describe current "best practices" and methodological adaptations required to assess abuse-deterrent opioid formulations for regulatory submissions. A literature search was conducted in PubMed® to review methodological approaches (study conduct and analysis) used in opioid human abuse potential studies. Search terms included a combination of "opioid," "opiate," "abuse potential," "abuse liability," "liking," AND "pharmacodynamic," and only studies that evaluated single doses of opioids in healthy, nondependent individuals with or without prior opioid experience were included. Seventy-one human abuse potential studies meeting the prespecified criteria were identified, of which 21 studies evaluated a purported opioid ADF. Based on these studies, key methodological considerations were reviewed and summarized according to participant demographics, study prequalification, comparator and dose selection, route of administration and drug manipulation, study blinding, outcome measures and training, safety, and statistical analyses. The authors recommend careful consideration of key elements (eg, a standardized definition of a "nondependent recreational user"), as applicable, and offer key principles and "best practices" when conducting human abuse potential studies for opioid ADFs. Careful selection of appropriate study conditions is dependent on the type of ADF technology being evaluated.

Teacher Leadership and Teacher Efficacy: A Correlational Study Comparing Teacher Perceptions of Leadership and Efficacy and Teacher Evaluation Scores from the North Carolina Educator Evaluation System

ERIC Educational Resources Information Center

Guenzler, April M.

2016-01-01

This study sought to identify correlations between constructs of teacher leadership, teacher efficacy, and teacher evaluation. Teacher perceptual data of support of teacher leadership, perceptual data on personal teacher efficacy, and teacher self-reported scores from the North Carolina Educator Evaluation System were gathered. The relationships…

Are Some Countries More Prone to Pressure Evaluators than Others? Comparing Findings from the United States, United Kingdom, Germany, and Switzerland

ERIC Educational Resources Information Center

Pleger, Lyn; Sager, Fritz; Morris, Michael; Meyer, Wolfgang; Stockmann, Reinhard

2017-01-01

Pressure on evaluators has been investigated recently by surveys in the USA, the UK, Germany, and Switzerland. This study compares the results of those studies regarding pressure on evaluators in different countries. The findings suggest that independence of evaluations does not exist for many respondents. Moreover, the person who commissioned the…

Making the Invisible Visible: A Responsive Evaluation Study of ESL and Spanish Language Services for Immigrants in a Small Rural County in Indiana

ERIC Educational Resources Information Center

Pawan, Faridah; Thomalla, Therese Groff

2005-01-01

This article describes a responsive evaluation study of ESL services and Spanish language services for immigrants in a rural county in Indiana. An ESL specialist led the evaluation of language services in the county from the perspectives of language providers and recipients. The responsive evaluation--a form of action research that uses…

A Study on Developing Evaluation Criteria for Electronic Resources in Evaluation Indicators of Libraries

ERIC Educational Resources Information Center

Noh, Younghee

2010-01-01

This study aimed to improve the current state of electronic resource evaluation in libraries. While the use of Web DB, e-book, e-journal, and other e-resources such as CD-ROM, DVD, and micro materials is increasing in libraries, their use is not comprehensively factored into the general evaluation of libraries and may diminish the reliability of…

Training the East German Labour Force. Microeconometric Evaluations of Continuous Vocational Training after Unification. Studies in Contemporary Economics.

ERIC Educational Resources Information Center

Lechner, Michael

This book presents empirical evaluations of the effects of different types of training programs in East Germany. Chapter 1 presents an overview of labor, the study objectives and results, and discussion of causality and the identification problem in evaluation studies. Chapter 2 examines point estimates of the effects of two types of continuous…

The Usability Evaluation of Fakih Method Based on Technology for Students with Hearing Difficulties: The User's Retrospective

ERIC Educational Resources Information Center

Sabdan, Muhammad Sayuti Bin; Alias, Norlidah; Jomhari, Nazean; Jamaludin, Khairul Azhar; DeWitt, Dorothy

2014-01-01

The study is aimed at evaluating the FAKIH method based on technology in teaching al-Quran, based on the user's retrospective. The participants of this study were five students selected based on hearing difficulties. The study employed the user evaluation framework. Teacher's journals were used to determine the frequency and percentage of…

The Evaluation of Health Education Program (HEP) of 9th Graders

ERIC Educational Resources Information Center

Erdogan, Polat; Gürol, Mehmet

2016-01-01

The main purpose of evaluation is to improve the quality of program. So, the purpose of this study was to evaluate the Health Education Program of 9th graders (HEP) with Educational Criticism Model of Eisner. The study was conducted with the qualitative research method. A holistic single case design was employed in this study. The typical case…

Study of Teacher Evaluation and Dismissal: Recommendations for 1998. Report to the Colorado General Assembly. Research Publication No. 429.

ERIC Educational Resources Information Center

Colorado State General Assembly, Denver. Legislative Council.

The Interim Committee on Teacher Evaluation and Dismissal was established by (Colorado) Senate Joint Resolution 97-14 to study the state's teacher evaluation and dismissal laws and explore alternatives. The committee was directed to study the relationship between education reform and employment protections for teachers; the effectiveness of adding…

Participation in Performance-Evaluation Studies by U.S. Geological Survey National Water Quality Laboratory

USGS Publications Warehouse

Glodt, Stephen R.; Pirkey, Kimberly D.

1998-01-01

Performance-evaluation studies provide customers of the U.S. Geological Survey National Water Quality Laboratory (NWQL) with data needed to evaluate performance and to compare of select laboratories for analytical work. The NWQL participates in national and international performance-evaluation (PE) studies that consist of samples of water, sediment, and aquatic biological materials for the analysis of inorganic constituents, organic compounds, and radionuclides. This Fact Sheet provides a summary of PE study results from January 1993 through April 1997. It should be of particular interest to USGS customers and potential customers of the NWQL, water-quality specialists, cooperators, and agencies of the Federal Government.

Photogrammetry as a tool for the postural evaluation of the spine: A systematic review.

PubMed

Furlanetto, Tássia Silveira; Sedrez, Juliana Adami; Candotti, Cláudia Tarragô; Loss, Jefferson Fagundes

2016-02-18

To evaluate the use of photogrammetry and identify the mathematical procedures applied when evaluating spinal posture. A systematic search using keywords was conducted in the PubMed, EMBASE, Scopus, Science and Medicine(®) databases. The following inclusion criteria adopted were: (1) the use of photogrammetry as a method to evaluate spinal posture; (2) evaluations of spinal curvature in the sagittal and/or frontal plane; (3) studies published within the last three decades; and (4) written entirely in English. The exclusion criteria were: (1) studies which objective involved the verification of some aspect of validation of instruments; (2) studies published as abstracts and those published in scientific events; and (3) studies using evaluation of the anteriorization of the head to determine the angular positioning of the cervical spine. The articles in this review were included and evaluated for their methodological quality, based on the Downs and Black scale, by two independent reviewers. Initially, 1758 articles were found, 76 of which were included upon reading the full texts and 29 were included in accordance with the predetermined criteria. In addition, after analyzing the references in those articles, a further six articles were selected, so that 35 articles were included in this review. This systematic review revealed that the photogrammetry has been using in observational studies. Furthermore, it was also found that, although the data collection methodologies are similar across the studies, in relation to aspects of data analysis, the methodologies are very different, especially regarding the mathematical routines employed to support different postural evaluation software. With photogrammetry, the aim of the assessment, whether it is for clinical, research or collective health purposes, must be considered when choosing which protocol to use to evaluate spinal posture.

Photogrammetry as a tool for the postural evaluation of the spine: A systematic review

PubMed Central

Furlanetto, Tássia Silveira; Sedrez, Juliana Adami; Candotti, Cláudia Tarragô; Loss, Jefferson Fagundes

2016-01-01

AIM: To evaluate the use of photogrammetry and identify the mathematical procedures applied when evaluating spinal posture. METHODS: A systematic search using keywords was conducted in the PubMed, EMBASE, Scopus, Science and Medicine® databases. The following inclusion criteria adopted were: (1) the use of photogrammetry as a method to evaluate spinal posture; (2) evaluations of spinal curvature in the sagittal and/or frontal plane; (3) studies published within the last three decades; and (4) written entirely in English. The exclusion criteria were: (1) studies which objective involved the verification of some aspect of validation of instruments; (2) studies published as abstracts and those published in scientific events; and (3) studies using evaluation of the anteriorization of the head to determine the angular positioning of the cervical spine. The articles in this review were included and evaluated for their methodological quality, based on the Downs and Black scale, by two independent reviewers. RESULTS: Initially, 1758 articles were found, 76 of which were included upon reading the full texts and 29 were included in accordance with the predetermined criteria. In addition, after analyzing the references in those articles, a further six articles were selected, so that 35 articles were included in this review. This systematic review revealed that the photogrammetry has been using in observational studies. Furthermore, it was also found that, although the data collection methodologies are similar across the studies, in relation to aspects of data analysis, the methodologies are very different, especially regarding the mathematical routines employed to support different postural evaluation software. CONCLUSION: With photogrammetry, the aim of the assessment, whether it is for clinical, research or collective health purposes, must be considered when choosing which protocol to use to evaluate spinal posture. PMID:26925386

Foster youth evaluate the performance of group home services in California.

PubMed

Green, Rex S; Ellis, Peter T

2008-05-01

In 2003 foster youth employed by a foster youth advocacy organization suggested that an evaluation of group home services to foster youth be conducted in Alameda County, California. This report presents the development and conduct of this evaluation study; how funding was obtained; and how foster youth were hired, trained, and employed to produce a timely and informative evaluation of the performance of 32 group homes where some of the foster youth formerly resided. The results of the study are described in another paper. This report contributes to evaluation practice in the newly emerging field of youth-led evaluations. The achievements of this project in utilizing group home clients to evaluate services with which they were familiar may stimulate other evaluators to develop similar projects, thereby enriching the development of our youth and promoting more informative evaluation findings.

Hand function evaluation: a factor analysis study.

PubMed

Jarus, T; Poremba, R

1993-05-01

The purpose of this study was to investigate hand function evaluations. Factor analysis with varimax rotation was used to assess the fundamental characteristics of the items included in the Jebsen Hand Function Test and the Smith Hand Function Evaluation. The study sample consisted of 144 subjects without disabilities and 22 subjects with Colles fracture. Results suggest a four factor solution: Factor I--pinch movement; Factor II--grasp; Factor III--target accuracy; and Factor IV--activities of daily living. These categories differentiated the subjects without Colles fracture from the subjects with Colles fracture. A hand function evaluation consisting of these four factors would be useful. Such an evaluation that can be used for current clinical purposes is provided.

Economic Studies in Motor Neurone Disease: A Systematic Methodological Review.

PubMed

Moore, Alan; Young, Carolyn A; Hughes, Dyfrig A

2017-04-01

Motor neurone disease (MND) is a devastating condition which greatly diminishes patients' quality of life and limits life expectancy. Health technology appraisals of future interventions in MND need robust data on costs and utilities. Existing economic evaluations have been noted to be limited and fraught with challenges. The aim of this study was to identify and critique methodological aspects of all published economic evaluations, cost studies, and utility studies in MND. We systematically reviewed all relevant published studies in English from 1946 until January 2016, searching the databases of Medline, EMBASE, Econlit, NHS Economic Evaluation Database (NHS EED) and the Health Economics Evaluation Database (HEED). Key data were extracted and synthesised narratively. A total of 1830 articles were identified, of which 15 economic evaluations, 23 cost and 3 utility studies were included. Most economic studies focused on riluzole (n = 9). Six studies modelled the progressive decline in motor function using a Markov design but did not include mutually exclusive health states. Cost estimates for a number of evaluations were based on expert opinion and were hampered by high variability and location-specific characteristics. Few cost studies reported disease-stage-specific costs (n = 3) or fully captured indirect costs. Utilities in three studies of MND patients used the EuroQol EQ-5D questionnaire or standard gamble, but included potentially unrepresentative cohorts and did not consider any health impacts on caregivers. Economic evaluations in MND suffer from significant methodological issues such as a lack of data, uncertainty with the disease course and use of inappropriate modelling framework. Limitations may be addressed through the collection of detailed and representative data from large cohorts of patients.

A Systematic Review of the Level of Evidence in Economic Evaluations of Medical Devices: The Example of Vertebroplasty and Kyphoplasty.

PubMed

Martelli, Nicolas; Devaux, Capucine; van den Brink, Hélène; Pineau, Judith; Prognon, Patrice; Borget, Isabelle

2015-01-01

Economic evaluations are far less frequently reported for medical devices than for drugs. In addition, little is known about the quality of existing economic evaluations, particularly for innovative devices, such as those used in vertebroplasty and kyphoplasty. To assess the level of evidence provided by the available economic evaluations for vertebroplasty and kyphoplasty. A systematic review of articles in English or French listed in the MEDLINE, PASCAL, COCHRANE and National Health Service Economic Evaluation databases, with limits on publication date (up to the date of the review, March 2014). We included only economic evaluations of vertebroplasty or kyphoplasty. Editorial and methodological articles were excluded. Data were extracted from articles by two authors working independently and using two analysis grids to measure the quality of economic evaluations. Twenty-one studies met our inclusion criteria. All were published between 2008 and 2014. Eighteen (86%) were full economic evaluations. Cost-effectiveness analysis (CEA) was the most frequent type of economic evaluation, and was present in 11 (52%) studies. Only three CEAs complied fully with the British Medical Journal checklist. The quality of the data sources used in the 21 studies was high, but the CEAs conforming to methodological guidelines did not use high-quality data sources for all components of the analysis. This systematic review shows that the level of evidence in economic evaluations of vertebroplasty and kyphoplasty is low, despite the recent publication of a large number of studies. This finding highlights the challenges to be faced to improve the quality of economic evaluations of medical devices.

Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement

PubMed Central

Des Jarlais, Don C.; Lyles, Cynthia; Crepaz, Nicole

2004-01-01

Developing an evidence base for making public health decisions will require using data from evaluation studies with randomized and nonrandomized designs. Assessing individual studies and using studies in quantitative research syntheses require transparent reporting of the study, with sufficient detail and clarity to readily see differences and similarities among studies in the same area. The Consolidated Standards of Reporting Trials (CONSORT) statement provides guidelines for transparent reporting of randomized clinical trials. We present the initial version of the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) statement. These guidelines emphasize the reporting of theories used and descriptions of intervention and comparison conditions, research design, and methods of adjusting for possible biases in evaluation studies that use nonrandomized designs. PMID:14998794

Evaluating Training.

ERIC Educational Resources Information Center

Brethower, Karen S.; Rummler, Geary A.

1979-01-01

Presents general systems models (ballistic system, guided system, and adaptive system) and an evaluation matrix to help in examining training evaluation alternatives and in deciding what evaluation is appropriate. Includes some guidelines for conducting evaluation studies using four designs (control group, reversal, multiple baseline, and…

A case report of evaluating a large-scale health systems improvement project in an uncontrolled setting: a quality improvement initiative in KwaZulu-Natal, South Africa.

PubMed

Mate, Kedar S; Ngidi, Wilbroda Hlolisile; Reddy, Jennifer; Mphatswe, Wendy; Rollins, Nigel; Barker, Pierre

2013-11-01

New approaches are needed to evaluate quality improvement (QI) within large-scale public health efforts. This case report details challenges to large-scale QI evaluation, and proposes solutions relying on adaptive study design. We used two sequential evaluative methods to study a QI effort to improve delivery of HIV preventive care in public health facilities in three districts in KwaZulu-Natal, South Africa, over a 3-year period. We initially used a cluster randomised controlled trial (RCT) design. During the RCT study period, tensions arose between intervention implementation and evaluation design due to loss of integrity of the randomisation unit over time, pressure to implement changes across the randomisation unit boundaries, and use of administrative rather than functional structures for the randomisation. In response to this loss of design integrity, we switched to a more flexible intervention design and a mixed-methods quasiexperimental evaluation relying on both a qualitative analysis and an interrupted time series quantitative analysis. Cluster RCT designs may not be optimal for evaluating complex interventions to improve implementation in uncontrolled 'real world' settings. More flexible, context-sensitive evaluation designs offer a better balance of the need to adjust the intervention during the evaluation to meet implementation challenges while providing the data required to evaluate effectiveness. Our case study involved HIV care in a resource-limited setting, but these issues likely apply to complex improvement interventions in other settings.

An urban energy performance evaluation system and its computer implementation.

PubMed

Wang, Lei; Yuan, Guan; Long, Ruyin; Chen, Hong

2017-12-15

To improve the urban environment and effectively reflect and promote urban energy performance, an urban energy performance evaluation system was constructed, thereby strengthening urban environmental management capabilities. From the perspectives of internalization and externalization, a framework of evaluation indicators and key factors that determine urban energy performance and explore the reasons for differences in performance was proposed according to established theory and previous studies. Using the improved stochastic frontier analysis method, an urban energy performance evaluation and factor analysis model was built that brings performance evaluation and factor analysis into the same stage for study. According to data obtained for the Chinese provincial capitals from 2004 to 2013, the coefficients of the evaluation indicators and key factors were calculated by the urban energy performance evaluation and factor analysis model. These coefficients were then used to compile the program file. The urban energy performance evaluation system developed in this study was designed in three parts: a database, a distributed component server, and a human-machine interface. Its functions were designed as login, addition, edit, input, calculation, analysis, comparison, inquiry, and export. On the basis of these contents, an urban energy performance evaluation system was developed using Microsoft Visual Studio .NET 2015. The system can effectively reflect the status of and any changes in urban energy performance. Beijing was considered as an example to conduct an empirical study, which further verified the applicability and convenience of this evaluation system. Copyright © 2017 Elsevier Ltd. All rights reserved.

Specific guidelines for assessing and improving the methodological quality of economic evaluations of newborn screening

PubMed Central

2012-01-01

Background Economic evaluation of newborn screening poses specific methodological challenges. Amongst others, these challenges refer to the use of quality adjusted life years (QALYs) in newborns, and which costs and outcomes need to be considered in a full evaluation of newborn screening programmes. Because of the increasing scale and scope of such programmes, a better understanding of the methods of high-quality economic evaluations may be crucial for both producers/authors and consumers/reviewers of newborn screening-related economic evaluations. The aim of this study was therefore to develop specific guidelines designed to assess and improve the methodological quality of economic evaluations in newborn screening. Methods To develop the guidelines, existing guidelines for assessing the quality of economic evaluations were identified through a literature search, and were reviewed and consolidated using a deductive iterative approach. In a subsequent test phase, these guidelines were applied to various economic evaluations which acted as case studies. Results The guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening are organized into 11 categories: “bibliographic details”, “study question and design”, “modelling”, “health outcomes”, “costs”, “discounting”, “presentation of results”, “sensitivity analyses”, “discussion”, “conclusions”, and “commentary”. Conclusions The application of the guidelines highlights important issues regarding newborn screening-related economic evaluations, and underscores the need for such issues to be afforded greater consideration in future economic evaluations. The variety in methodological quality detected by this study reveals the need for specific guidelines on the appropriate methods for conducting sound economic evaluations in newborn screening. PMID:22947299

Preference Reversals Between Joint and Separate Evaluations With Multiple Alternatives and Context Effects.

PubMed

Cheng, Yin-Hui; Yu, Annie P; Huang, Molly Chien-Jung; Dai, Chia-Jung

2017-01-01

Most research on preference reversal (PR) focuses on the evaluability hypothesis with one or two alternatives. However, people normally encounter more than two options in daily life. In this research, a third option was added to the PR effect choice sets in the traditional joint-separate evaluations mode to create a context effect. Three studies were conducted. Studies 1 and 2 showed that adding a third option to the choice sets changed the PR effect; either the attributes were both important or one was important and the other was not. Study 3 showed that the PR effect reappeared when a third option was added to the choice sets that had no PR effect with just two attributes that were difficult to evaluate independently in traditional evaluations modes. The three studies confirmed that preferences changed in multi-alternative evaluation modes, contradicting Hsee's (1996) work and showing that the context effect is stronger than that of the attribute's importance in the PR effect.

Vending machine assessment methodology. A systematic review.

PubMed

Matthews, Melissa A; Horacek, Tanya M

2015-07-01

The nutritional quality of food and beverage products sold in vending machines has been implicated as a contributing factor to the development of an obesogenic food environment. How comprehensive, reliable, and valid are the current assessment tools for vending machines to support or refute these claims? A systematic review was conducted to summarize, compare, and evaluate the current methodologies and available tools for vending machine assessment. A total of 24 relevant research studies published between 1981 and 2013 met inclusion criteria for this review. The methodological variables reviewed in this study include assessment tool type, study location, machine accessibility, product availability, healthfulness criteria, portion size, price, product promotion, and quality of scientific practice. There were wide variations in the depth of the assessment methodologies and product healthfulness criteria utilized among the reviewed studies. Of the reviewed studies, 39% evaluated machine accessibility, 91% evaluated product availability, 96% established healthfulness criteria, 70% evaluated portion size, 48% evaluated price, 52% evaluated product promotion, and 22% evaluated the quality of scientific practice. Of all reviewed articles, 87% reached conclusions that provided insight into the healthfulness of vended products and/or vending environment. Product healthfulness criteria and complexity for snack and beverage products was also found to be variable between the reviewed studies. These findings make it difficult to compare results between studies. A universal, valid, and reliable vending machine assessment tool that is comprehensive yet user-friendly is recommended. Copyright © 2015 Elsevier Ltd. All rights reserved.

Integrating service development with evaluation in telehealthcare: an ethnographic study

PubMed Central

Finch, Tracy; May, Carl; Mair, Frances; Mort, Maggie; Gask, Linda

2003-01-01

Objectives To identify issues that facilitate the successful integration of evaluation and development of telehealthcare services. Design Ethnographic study using various qualitative research techniques to obtain data from several sources, including in-depth semistructured interviews, project steering group meetings, and public telehealthcare meetings. Setting Seven telehealthcare evaluation projects (four randomised controlled trials and three pragmatic service evaluations) in the United Kingdom, studied over two years. Projects spanned a range of specialties—dermatology, psychiatry, respiratory medicine, cardiology, and oncology. Participants Clinicians, managers, technical experts, and researchers involved in the projects. Results and discussion Key problems in successfully integrating evaluation and service development in telehealthcare are, firstly, defining existing clinical practices (and anticipating changes) in ways that permit measurement; secondly, managing additional workload and conflicting responsibilities brought about by combining clinical and research responsibilities (including managing risk); and, thirdly, understanding various perspectives on effectiveness and the limitations of evaluation results beyond the context of the research study. Conclusions Combined implementation and evaluation of telehealthcare systems is complex, and is often underestimated. The distinction between quantitative outcomes and the workability of the system is important for producing evaluative knowledge that is of practical value. More pragmatic approaches to evaluation, that permit both quantitative and qualitative methods, are required to improve the quality of such research and its relevance for service provision in the NHS. PMID:14630758

Is the evaluation of risk of bias in periodontology and implant dentistry comprehensive? A systematic review.

PubMed

Faggion, Clovis Mariano; Listl, Stefan; Alarcón, Marco Antonio

2015-05-01

The objective of this study was to assess how authors of systematic reviews (SRs) with meta-analyses published in periodontology and implant dentistry evaluate risk of bias (ROB) in primary studies included in these reviews. A literature search for SRs with meta-analyses was performed in PubMed and Cochrane library databases up to July 20th 2014. The reference lists of included articles were screened for further reviews. The standards of evaluating ROB in primary studies were evaluated by using a 14-item checklist based on the Cochrane approach for evaluating ROB. Standards in ROB evaluations in Cochrane and paper-based SRs were compared using the Fisher's exact test. All searches, data extraction and evaluations were performed independently and in duplicate. Seventy SRs were included (45 paper-based and 25 Cochrane SRs, respectively). The median percentage of items addressed was 58% (interquartile range 4-100%). Cochrane SRs more frequently included ROB assessments than paper-based reviews in terms of examiner blinding (p = 0.0026), selective outcome reporting (p = 0.0207) and other bias (p = 0.0241). The ROB evaluation in primary studies currently included in SRs with meta-analyses in periodontology and implant dentistry is not sufficiently comprehensive. Cochrane SRs have more comprehensive ROB evaluation than paper-based reviews. © 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Evaluator Training: Content and Topic Valuation in University Evaluation Courses

ERIC Educational Resources Information Center

Davies, Randall; MacKay, Kathryn

2014-01-01

Quality training opportunities for evaluators will always be important to the evaluation profession. While studies have documented the number of university programs providing evaluation training, additional information is needed concerning what content is being taught in current evaluation courses. This article summarizes the findings of a survey…

An Empirical Examination of Validity in Evaluation

ERIC Educational Resources Information Center

Peck, Laura R.; Kim, Yushim; Lucio, Joanna

2012-01-01

This study addresses validity issues in evaluation that stem from Ernest R. House's book, "Evaluating With Validity". The authors examine "American Journal of Evaluation" articles from 1980 to 2010 that report the results of policy and program evaluations. The authors classify these evaluations according to House's "major approaches" typology…

Cross-Continental Reflections on Evaluation Practice: Methods, Use, and Valuing

ERIC Educational Resources Information Center

Kallemeyn, Leanne M.; Hall, Jori; Friche, Nanna; McReynolds, Clifton

2015-01-01

The evaluation theory tree typology reflects the following three components of evaluation practice: (a) methods, (b) use, and (c) valuing. The purpose of this study was to explore how evaluation practice is conceived as reflected in articles published in the "American Journal of Evaluation" ("AJE") and "Evaluation," a…

Reflections and Future Prospects for Evaluation in Human Resource Development

ERIC Educational Resources Information Center

Han, Heeyoung; Boulay, David

2013-01-01

Human resource development (HRD) evaluation has often been criticized for its limited function in organizational decision making. This article reviews evaluation studies to uncover the current status of HRD evaluation literature. The authors further discuss general evaluation theories in terms of value, use, and evaluator role to extend the…

An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit).

PubMed

Yusof, Maryati Mohd; Kuljis, Jasna; Papazafeiropoulou, Anastasia; Stergioulas, Lampros K

2008-06-01

The realization of Health Information Systems (HIS) requires rigorous evaluation that addresses technology, human and organization issues. Our review indicates that current evaluation methods evaluate different aspects of HIS and they can be improved upon. A new evaluation framework, human, organization and technology-fit (HOT-fit) was developed after having conducted a critical appraisal of the findings of existing HIS evaluation studies. HOT-fit builds on previous models of IS evaluation--in particular, the IS Success Model and the IT-Organization Fit Model. This paper introduces the new framework for HIS evaluation that incorporates comprehensive dimensions and measures of HIS and provides a technological, human and organizational fit. Literature review on HIS and IS evaluation studies and pilot testing of developed framework. The framework was used to evaluate a Fundus Imaging System (FIS) of a primary care organization in the UK. The case study was conducted through observation, interview and document analysis. The main findings show that having the right user attitude and skills base together with good leadership, IT-friendly environment and good communication can have positive influence on the system adoption. Comprehensive, specific evaluation factors, dimensions and measures in the new framework (HOT-fit) are applicable in HIS evaluation. The use of such a framework is argued to be useful not only for comprehensive evaluation of the particular FIS system under investigation, but potentially also for any Health Information System in general.

Electrophysiological responses to evaluative priming: the LPP is sensitive to incongruity.

PubMed

Herring, David R; Taylor, Jennifer H; White, Katherine R; Crites, Stephen L

2011-08-01

Previous studies examining event-related potentials and evaluative priming have been mixed; some find evidence that evaluative priming influences the N400, whereas others find evidence that it affects the late positive potential (LPP). Three experiments were conducted using either affective pictures (Experiments 1 and 2) or words (Experiment 3) in a sequential evaluative priming paradigm. In line with previous behavioral findings, participants responded slower to targets that were evaluatively incongruent with the preceding prime (e.g., negative preceded by positive) compared to evaluatively congruent targets (e.g., negative preceded by negative). In all three studies, the LPP was larger to evaluatively incongruent targets compared to evaluatively congruent ones, and there was no evidence that evaluative incongruity influenced the N400 component. Thus, the present results provide additional support for the notion that evaluative priming influences the LPP and not the N400. We discuss possible reasons for the inconsistent findings in prior research and the theoretical implications of the findings for both evaluative and semantic priming. 2011 APA, all rights reserved

Quantity and Quality of Economic Evaluations in U.S. Nursing Research, 1997-2015: A Systematic Review.

PubMed

Cook, Wendy A; Morrison, Megan L; Eaton, Linda H; Theodore, Brian R; Doorenbos, Ardith Z

The United States has a complex healthcare system that is undergoing substantial reformations. There is a need for high-quality, economic evaluations of nursing practice. An updated review of completed economic evaluations relevant to the field of nursing within the U.S. healthcare system is timely and needed. The purpose of this study was to evaluate and describe the quantity and quality of economic evaluations in nursing-relevant research performed in the United States between 1997 and 2015. Four databases were searched. Titles, abstracts, and full-text content were reviewed to identify studies that analyzed both costs and outcomes, relevant to nursing, performed in the United States, and used the quality-adjusted life year to measure effectiveness. For included studies, data were extracted from full-text articles using criteria from U.S. Public Health Service's Panel on Cost-Effectiveness in Health and Medicine. Twenty-eight studies met the inclusion criteria. Most (n = 25, 89%) were published in the last decade of the analysis, from 2006 to 2015. Assessment of quality, based on selected items from the panel guidelines, found that the evaluations did not consistently use the recommended societal perspective, use multiple resource utilization categories, use constant dollars, discount future costs and outcomes, use a lifetime horizon, or include an indication of uncertainty in results. The only resource utilization category consistently included across studies was healthcare resources. Only 28 nursing-related studies meeting the inclusion criteria were identified as meeting robust health economic evaluation methodological criteria, and most did not include all important guideline items. Despite increases in absolute numbers of published studies over the past decade, economic evaluation has been underutilized in U.S. nursing-relevant research in the past two decades.

Using program evaluation to support knowledge translation in an interprofessional primary care team: a case study.

PubMed

Donnelly, Catherine; Shulha, Lyn; Klinger, Don; Letts, Lori

2016-10-06

Evaluation is a fundamental component in building quality primary care and is ideally situated to support individual, team and organizational learning by offering an accessible form of participatory inquiry. The evaluation literature has begun to recognize the unique features of KT evaluations and has described attributes to consider when evaluating KT activities. While both disciplines have focused on the evaluation of KT activities neither has explored the role of evaluation in KT. The purpose of the paper is to examine how participation in program evaluation can support KT in a primary care setting. A mixed methods case study design was used, where evaluation was conceptualized as a change process and intervention. A Memory Clinic at an interprofessional primary care clinic was the setting in which the study was conducted. An evaluation framework, Pathways of Influence provided the theoretical foundation to understand how program evaluation can facilitate the translation of knowledge at the level of the individual, inter-personal (Memory Clinic team) and the organization. Data collection included questionnaires, interviews, evaluation log and document analysis. Questionnaires and interviews were administered both before and after the evaluation: Pattern matching was used to analyze the data based on predetermined propositions. Individuals gained program knowledge that resulted in changes to both individual and program practices. One of the key themes was the importance clinicians placed on local, program based knowledge. The evaluation had less influence on the broader health organization. Program evaluation facilitated individual, team and organizational learning. The use of evaluation to support KT is ideally suited to a primary care setting by offering relevant and applicable knowledge to primary care team members while being sensitive to local context.

Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases : a review and suggested reporting framework.

PubMed

Haji Ali Afzali, Hossein; Gray, Jodi; Karnon, Jonathan

2013-04-01

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess 'best practice' in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.

Evaluating Educational Programs.

ERIC Educational Resources Information Center

Ball, Samuel

The activities of Educational Testing Service (ETS) in evaluating educational programs are described. Program evaluations are categorized as needs assessment, formative evaluation, or summative evaluation. Three classic efforts which illustrate the range of ETS' participation are the Pennsylvania Goals Study (1965), the Coleman Report--Equality of…

The Role of Young Hoosier Book Award Nominees in the Classroom: Using Approaches to Multicultural Education as a Lens for Evaluating Literature

ERIC Educational Resources Information Center

Marks, Shaylyn Barrie

2013-01-01

This study demonstrates the need for the integration of multiculturalism in the K-12 curriculum as well as provides a heuristic based on the work conducted by Sleeter and Grant (2009) to evaluate literature for level of multiculturalism. In the study, the researcher uses an evaluate heuristic to critically analyze and evaluation ten of the books…

Improving Evaluation in the Nonprofit Sector: The Promise of Evaluation Capacity Building for Nonprofit Social Service Organizations in an Age of Accountability

ERIC Educational Resources Information Center

Pejsa, Laura J.

2011-01-01

The purpose of this study was to explore the real-world practice of evaluation capacity building (ECB) in the small-medium sized nonprofit context. The study aimed to map the activities, processes, and outcomes of ECB in these nonprofits in light of the competing external and internal evaluation contexts facing nonprofits in an age of…

College Governance: A Comparison of Faculty Evaluation in Public and Private Colleges with Implications for the Improvement of the Evaluation Process at Johnson and Wales College.

ERIC Educational Resources Information Center

Bukowski, Joseph E.

This study focuses on selected factors in the evaluation of faculty members in: (1) colleges accredited by the Association of Independent Colleges and Schools; (2) public junior and senior colleges; and (3) Rhode Island colleges. Results of the study indicate that faculty evaluation schemes must follow the basic goals and philosophy of the…

Evaluation of equipment and methods to map lost circulation zones in geothermal wells

DOE Office of Scientific and Technical Information (OSTI.GOV)

McDonald, W.J.; Leon, P.A.; Pittard, G.

A study and evaluation of methods to locate, characterize, and quantify lost circulation zones are described. Twenty-five methods of mapping and quantifying lost circulation zones were evaluated, including electrical, acoustical, mechanical, radioactive, and optical systems. Each tool studied is described. The structured, numerical evaluation plan, used as the basis for comparing the 25 tools, and the resulting ranking among the tools is presented.

Economic evaluation and end-stage renal disease: from basics to bedside.

PubMed

Manns, B J; Taub, K J; Donaldson, C

2000-07-01

Economic evaluation is the comparative analysis of alternative health care interventions in terms of their relative costs (resource use) and effectiveness (health effects). High-quality studies of economic evaluation have been increasingly published in medical journals and read by clinicians, although publication of these studies in nephrology journals has been a more recent phenomenon. This article shows how the basic principles of economics can be applied to health care through the use of economic evaluation. Different types of economic evaluation are discussed, and pitfalls common to such studies are identified. A simple framework is introduced that can be used to interpret the results of economic evaluations. Using this framework, selected therapies for patients with end-stage renal disease (ESRD) are categorized to highlight therapies that are very efficient, encourage their use, and draw attention to therapies in current use that are less effective and more expensive (ie, less efficient) than alternative therapy. Using examples pertinent to care of the patient with ESRD, we show how economic evaluation can be used to link medical outcomes, quality of life, and costs in a common index for multiple therapies with disparate outcome measures. This article highlights the need for clinical studies and economic evaluations of therapies in ESRD for which the effects of the therapy on health outcomes and/or costs are unknown.

Monitoring and evaluation of strategic change programme implementation-Lessons from a case analysis.

PubMed

Neumann, Jan; Robson, Andrew; Sloan, Diane

2018-02-01

This study considered the monitoring and evaluation of a large-scale and domestic and global strategic change programme implementation. It considers the necessary prerequisites to overcome challenges and barriers that prevent systematic and effective monitoring and evaluation to take place alongside its operationalisation. The work involves a case study based on a major industrial company from the energy sector. The change programme makes particular reference to changes in business models, business processes, organisation structures as well as Enterprise Resource Planning infrastructure. The case study focussed on the summative evaluation of the programme post-implementation. This assessment involved 25 semi-structured interviews with employees across a range of managerial strata capturing more than 65 roles within the change programme at both local and global levels. Data relating to their perception of evaluation effectiveness and shortcomings were analysed by means of template analysis. The study identifies responsibilities for executing an evaluation alongside various methods and tools that are appropriate, thereby focussing on the "Who" (roles, responsibility for particular activities) and "How" (methods and tools) rather than "What" to monitor and evaluate. The findings are presented generically so they offer new insights and transferability for practitioners involved in managing strategic change and its associated evaluation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program

PubMed Central

Jaafaripooyan, Ebrahim

2014-01-01

Background: Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. Methods: A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Results: Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs’ over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Conclusion: Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs. PMID:25279381

New pathways in the evaluation of programmes for men who perpetrate violence against their female partners.

PubMed

Wojnicka, Katarzyna; Scambor, Christian; Kraus, Heinrich

2016-08-01

Today, evaluation research in the field of intervention programmes for men who perpetrate violence against their female partners still makes a fragmentary impression. Across Europe various evaluation studies have been performed. However, the methodologies applied are too heterogeneous to allow the combination of the results in a meta-analytical way. In this paper we propose a future pathway for organising outcome evaluation studies of domestic violence perpetrator programmes in community settings, so that today's problems in this field can be overcome. In a pragmatic framework that acknowledges the limited pre-conditions for evaluation studies in the area of domestic violence perpetrator programmes as it is today, feasible approaches for outcome evaluation are outlined, with recent developments in the field taken as starting points. The framework for organising future evaluation studies of work with perpetrators of domestic violence is presented together with a strategy to promote this framework. International networks of practitioners and researchers play a central role in this strategy through upskilling the area of practical work, preparing the ground for evaluation research and improving cooperation between practitioners and researchers. This paper is based on the results of the European funded project IMPACT (under the Daphne-III-funding programme of the European Commission). Copyright © 2016 Elsevier Ltd. All rights reserved.

Algorithms for detecting and predicting influenza outbreaks: metanarrative review of prospective evaluations

PubMed Central

Spreco, A; Timpka, T

2016-01-01

Objectives Reliable monitoring of influenza seasons and pandemic outbreaks is essential for response planning, but compilations of reports on detection and prediction algorithm performance in influenza control practice are largely missing. The aim of this study is to perform a metanarrative review of prospective evaluations of influenza outbreak detection and prediction algorithms restricted settings where authentic surveillance data have been used. Design The study was performed as a metanarrative review. An electronic literature search was performed, papers selected and qualitative and semiquantitative content analyses were conducted. For data extraction and interpretations, researcher triangulation was used for quality assurance. Results Eight prospective evaluations were found that used authentic surveillance data: three studies evaluating detection and five studies evaluating prediction. The methodological perspectives and experiences from the evaluations were found to have been reported in narrative formats representing biodefence informatics and health policy research, respectively. The biodefence informatics narrative having an emphasis on verification of technically and mathematically sound algorithms constituted a large part of the reporting. Four evaluations were reported as health policy research narratives, thus formulated in a manner that allows the results to qualify as policy evidence. Conclusions Awareness of the narrative format in which results are reported is essential when interpreting algorithm evaluations from an infectious disease control practice perspective. PMID:27154479

A Study on Project Priority Evaluation Method on Road Slope Disaster Prevention Management

NASA Astrophysics Data System (ADS)

Sekiguchi, Nobuyasu; Ohtsu, Hiroyasu; Izu, Ryuutarou

To improve the safety and security of driving while coping with today's stagnant economy and frequent natural disasters, road slopes should be appropriately managed. To achieve the goals, road managers should establish project priority evaluation methods for each stage of road slope management by clarifying social losses that would result by drops in service levels. It is important that road managers evaluate a project priority properly to manage the road slope effectively. From this viewpoint, this study proposed "project priority evaluation methods" in road slope disaster prevention, which use available slope information at each stage of road slope management under limited funds. In addition, this study investigated the effect of managing it from the high slope of the priority by evaluating a risk of slope failure. In terms of the amount of available information, staged information provision is needed ranging from macroscopic studies, which involves evaluation of the entire route at each stage of decision making, to semi- and microscopic investigations for evaluating slopes, and microscopic investigations for evaluating individual slopes. With limited funds, additional detailed surveys are difficult to perform. It is effective to use the slope risk assessment system, which was constructed to complement detailed data, to extract sites to perform precise investigations.

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program.

PubMed

Jaafaripooyan, Ebrahim

2014-09-01

Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs' over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs.

The Estimation and Inclusion of Presenteeism Costs in Applied Economic Evaluation: A Systematic Review.

PubMed

Kigozi, Jesse; Jowett, Sue; Lewis, Martyn; Barton, Pelham; Coast, Joanna

2017-03-01

Given the significant costs of reduced productivity (presenteeism) in comparison to absenteeism, and overall societal costs, presenteeism has a potentially important role to play in economic evaluations. However, these costs are often excluded. The objective of this study is to review applied cost of illness studies and economic evaluations to identify valuation methods used for, and impact of including presenteeism costs in practice. A structured systematic review was carried out to explore (i) the extent to which presenteeism has been applied in cost of illness studies and economic evaluations and (ii) the overall impact of including presenteeism on overall costs and outcomes. Potential articles were identified by searching Medline, PsycINFO and NHS EED databases. A standard template was developed and used to extract information from economic evaluations and cost of illness studies incorporating presenteeism costs. A total of 28 studies were included in the systematic review which also demonstrated that presenteeism costs are rarely included in full economic evaluations. Estimation and monetisation methods differed between the instruments. The impact of disease on presenteeism whilst in paid work is high. The potential impact of presenteeism costs needs to be highlighted and greater consideration should be given to including these in economic evaluations and cost of illness studies. The importance of including presenteeism costs when conducting economic evaluation from a societal perspective should be emphasised in national economic guidelines and more methodological work is required to improve the practical application of presenteeism instruments to generate productivity cost estimates. Copyright © 2017 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Feasibility study of solar energy utilization in modular integrated utility systems

NASA Technical Reports Server (NTRS)

1975-01-01

The feasibility and benefits were evaluated of solar thermal energy systems on Integrated Utility Systems. The effort included the identification of potential system concepts, evaluation of hardware status, and performance of weighted system evaluations to select promising system concepts deserving of further study.

Traffic control device evaluation program: simulator evaluation of sponsored changeable message signs and in-situ evaluation of rumble strip alternatives.

DOT National Transportation Integrated Search

2016-07-01

This report describes two research activities. One research study tested the effects of adding commercial : logos acknowledging sponsorship to Changeable Message Signs. A driving simulator study was conducted : using test signs with travel times and ...

Safety evaluation of lane and shoulder width combinations on rural, two-lane, undivided roads

DOT National Transportation Integrated Search

2009-06-01

The Federal Highway Administration (FHWA) organized a pooled fund study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. The goal of this study is to evaluate the safety effectiveness of various lane...

Pilot evaluation of the use of contract towing and first responder strategies in the Virginia Department of Transportation's Staunton District.

DOT National Transportation Integrated Search

2017-06-01

This study evaluated the contract towing and first responder pilot projects in the Virginia Department of Transportations : Staunton District. The evaluation included before and after studies of incident duration and comparisons of clearance time....

Liberal Grading Improves Evaluations but Not Performance.

ERIC Educational Resources Information Center

Vasta, Ross; Sarmiento, Robert F.

1979-01-01

The effects of examination grades on undergraduates' study behaviors, attendance, and evaluations of instruction were examined. Liberal grading was found to result in higher evaluations of course and instructor but had no demonstrable effect on studying or attendance. The implications of these findings and alternative interpretations are…

Development and Psychometric Evaluation of the Interpersonal Sexual Objectification Scale

ERIC Educational Resources Information Center

Kozee, Holly B.; Tylka, Tracy L.; Augustus-Horvath, Casey L.; Denchik, Angela

2007-01-01

This study reports on the development and psychometric evaluation of the Interpersonal Sexual Objectification Scale (ISOS). Data from 576 college women were collected in three studies. Exploratory factor analysis uncovered two factors: Body Evaluation and Unwanted Explicit Sexual Advances; confirmatory factor analysis supported this factor…

42 CFR 456.241 - Purpose and general description.

Code of Federal Regulations, 2011 CFR

2011-10-01

... health care. (b) Medical care evaluation studies— (1) Emphasize identification and analysis of patterns... Ur Plan: Medical Care Evaluation Studies § 456.241 Purpose and general description. (a) The purpose of medical care evaluation studies is to promote the most effective and efficient use of available...

42 CFR 456.241 - Purpose and general description.

Code of Federal Regulations, 2010 CFR

2010-10-01

... health care. (b) Medical care evaluation studies— (1) Emphasize identification and analysis of patterns... Ur Plan: Medical Care Evaluation Studies § 456.241 Purpose and general description. (a) The purpose of medical care evaluation studies is to promote the most effective and efficient use of available...

Identfying the Needs of Pre-Service Classroom Teachers about Science Teaching Methodology Course in Terms of Parlett's Illuminative Program Evaluation Model

ERIC Educational Resources Information Center

Çaliskan, Ilke

2014-01-01

The aim of this study was to identify the needs of third grade classroom teaching students about science teaching course in terms of Parlett's Illuminative program evaluation model. Phenomographic research design was used in this study. Illuminative program evaluation model was chosen for this study in terms of its eclectic and process-based…

Identfying the Needs of Pre-Service Classroom Teachers about Science Teaching Methodology Courses in Terms of Parlett's Illuminative Program Evaluation Model

ERIC Educational Resources Information Center

Çaliskan, Ilke

2014-01-01

The aim of this study was to identify the needs of third grade classroom teaching students about science teaching course in terms of Parlett's Illuminative program evaluation model. Phenomographic research design was used in this study. Illuminative program evaluation model was chosen for this study in terms of its eclectic and process-based…

Using Faculty Evaluation to Improve Teaching Quality: A Longitudinal Case Study of Higher Education in Southeast Asia

ERIC Educational Resources Information Center

Hallinger, Philip

2010-01-01

This paper presents a case study of implementation of a new system of faculty teaching evaluation at a graduate school of business in Thailand. The research employed a non-experimental, longitudinal case study design in the analysis of student course evaluation data gathered over a period of 21 terms during a seven-year period. The report…

Needs assessment for next generation computer-aided mammography reference image databases and evaluation studies.

PubMed

Horsch, Alexander; Hapfelmeier, Alexander; Elter, Matthias

2011-11-01

Breast cancer is globally a major threat for women's health. Screening and adequate follow-up can significantly reduce the mortality from breast cancer. Human second reading of screening mammograms can increase breast cancer detection rates, whereas this has not been proven for current computer-aided detection systems as "second reader". Critical factors include the detection accuracy of the systems and the screening experience and training of the radiologist with the system. When assessing the performance of systems and system components, the choice of evaluation methods is particularly critical. Core assets herein are reference image databases and statistical methods. We have analyzed characteristics and usage of the currently largest publicly available mammography database, the Digital Database for Screening Mammography (DDSM) from the University of South Florida, in literature indexed in Medline, IEEE Xplore, SpringerLink, and SPIE, with respect to type of computer-aided diagnosis (CAD) (detection, CADe, or diagnostics, CADx), selection of database subsets, choice of evaluation method, and quality of descriptions. 59 publications presenting 106 evaluation studies met our selection criteria. In 54 studies (50.9%), the selection of test items (cases, images, regions of interest) extracted from the DDSM was not reproducible. Only 2 CADx studies, not any CADe studies, used the entire DDSM. The number of test items varies from 100 to 6000. Different statistical evaluation methods are chosen. Most common are train/test (34.9% of the studies), leave-one-out (23.6%), and N-fold cross-validation (18.9%). Database-related terminology tends to be imprecise or ambiguous, especially regarding the term "case". Overall, both the use of the DDSM as data source for evaluation of mammography CAD systems, and the application of statistical evaluation methods were found highly diverse. Results reported from different studies are therefore hardly comparable. Drawbacks of the DDSM (e.g. varying quality of lesion annotations) may contribute to the reasons. But larger bias seems to be caused by authors' own decisions upon study design. RECOMMENDATIONS/CONCLUSION: For future evaluation studies, we derive a set of 13 recommendations concerning the construction and usage of a test database, as well as the application of statistical evaluation methods.

METHODOLOGICAL QUALITY OF ECONOMIC EVALUATIONS ALONGSIDE TRIALS OF KNEE PHYSIOTHERAPY.

PubMed

García-Pérez, Lidia; Linertová, Renata; Arvelo-Martín, Alejandro; Guerra-Marrero, Carolina; Martínez-Alberto, Carlos Enrique; Cuéllar-Pompa, Leticia; Escobar, Antonio; Serrano-Aguilar, Pedro

2017-01-01

The methodological quality of an economic evaluation performed alongside a clinical trial can be underestimated if the paper does not report key methodological features. This study discusses methodological assessment issues on the example of a systematic review on cost-effectiveness of physiotherapy for knee osteoarthritis. Six economic evaluation studies included in the systematic review and related clinical trials were assessed using the 10-question check-list by Drummond and the Physiotherapy Evidence Database (PEDro) scale. All economic evaluations were performed alongside a clinical trial but the studied interventions were too heterogeneous to be synthesized. Methodological quality of the economic evaluations reported in the papers was not free of drawbacks, and in some cases, it improved when information from the related clinical trial was taken into account. Economic evaluation papers dedicate little space to methodological features of related clinical trials; therefore, the methodological quality can be underestimated if evaluated separately from the trials. Future economic evaluations should follow more strictly the recommendations about methodology and the authors should pay special attention to the quality of reporting.

On the evaluation of segmentation editing tools

PubMed Central

Heckel, Frank; Moltz, Jan H.; Meine, Hans; Geisler, Benjamin; Kießling, Andreas; D’Anastasi, Melvin; dos Santos, Daniel Pinto; Theruvath, Ashok Joseph; Hahn, Horst K.

2014-01-01

Abstract. Efficient segmentation editing tools are important components in the segmentation process, as no automatic methods exist that always generate sufficient results. Evaluating segmentation editing algorithms is challenging, because their quality depends on the user’s subjective impression. So far, no established methods for an objective, comprehensive evaluation of such tools exist and, particularly, intermediate segmentation results are not taken into account. We discuss the evaluation of editing algorithms in the context of tumor segmentation in computed tomography. We propose a rating scheme to qualitatively measure the accuracy and efficiency of editing tools in user studies. In order to objectively summarize the overall quality, we propose two scores based on the subjective rating and the quantified segmentation quality over time. Finally, a simulation-based evaluation approach is discussed, which allows a more reproducible evaluation without the need for human input. This automated evaluation complements user studies, allowing a more convincing evaluation, particularly during development, where frequent user studies are not possible. The proposed methods have been used to evaluate two dedicated editing algorithms on 131 representative tumor segmentations. We show how the comparison of editing algorithms benefits from the proposed methods. Our results also show the correlation of the suggested quality score with the qualitative ratings. PMID:26158063

A study on eco-environmental vulnerability of mining cities: a case study of Panzhihua city of Sichuan province in China

NASA Astrophysics Data System (ADS)

Shao, Huaiyong; Xian, Wei; Yang, Wunian

2009-07-01

The large-scale and super-strength development of mineral resources in mining cities in long term has made great contributions to China's economic construction and development, but it has caused serious damage to the ecological environment even ecological imbalance at the same time because the neglect of the environmental impact even to the expense of the environment to some extent. In this study, according to the characteristics of mining cities, the scientific and practical eco-environmental vulnerability evaluation index system of mining cities had been established. Taking Panzhihua city of Sichuan province as an example, using remote sensing and GIS technology, applying various types of remote sensing image (TM, SPOT5, IKONOS) and Statistical data, the ecological environment evaluation data of mining cities was extracted effectively. For the non-linear relationship between the evaluation indexes and the degree of eco-environmental vulnerability in mining cities, this study innovative took the evaluation of eco-environmental vulnerability of the study area by using artificial neural network whose training used SCE-UA algorithm that well overcome the slow learning and difficult convergence of traditional neural network algorithm. The results of ecoenvironmental vulnerability evaluation of the study area were objective, reasonable and the credibility was high. The results showed that the area distribution of five eco-environmental vulnerability grade types was basically normal, and the overall ecological environment situation of Panzhihua city was in the middle level, the degree of eco-environmental vulnerability in the south was higher than the north, and mining activities were dominant factors to cause ecoenvironmental damage and eco-environmental Vulnerability. In this study, a comprehensive theory and technology system of regional eco-environmental vulnerability evaluation which included the establishment of eco-environmental vulnerability evaluation index system, processing of evaluation data and establishing of evaluation model. New ideas and methods had provided for eco-environmental vulnerability of mining cities.

A systematic review of studies evaluating Australian indigenous community development projects: the extent of community participation, their methodological quality and their outcomes.

PubMed

Snijder, Mieke; Shakeshaft, Anthony; Wagemakers, Annemarie; Stephens, Anne; Calabria, Bianca

2015-11-21

Community development is a health promotion approach identified as having great potential to improve Indigenous health, because of its potential for extensive community participation. There has been no systematic examination of the extent of community participation in community development projects and little analysis of their effectiveness. This systematic review aims to identify the extent of community participation in community development projects implemented in Australian Indigenous communities, critically appraise the qualitative and quantitative methods used in their evaluation, and summarise their outcomes. Ten electronic peer-reviewed databases and two electronic grey literature databases were searched for relevant studies published between 1990 and 2015. The level of community participation and the methodological quality of the qualitative and quantitative components of the studies were assessed against standardised criteria. Thirty one evaluation studies of community development projects were identified. Community participation varied between different phases of project development, generally high during project implementation, but low during the evaluation phase. For the majority of studies, methodological quality was low and the methods were poorly described. Although positive qualitative or quantitative outcomes were reported in all studies, only two studies reported statistically significant outcomes. Partnerships between researchers, community members and service providers have great potential to improve methodological quality and community participation when research skills and community knowledge are integrated to design, implement and evaluate community development projects. The methodological quality of studies evaluating Australian Indigenous community development projects is currently too weak to confidently determine the cost-effectiveness of community development projects in improving the health and wellbeing of Indigenous Australians. Higher quality studies evaluating community development projects would strengthen the evidence base.

A Comparison of Participant and Practitioner Beliefs About Evaluation

PubMed Central

Whitehall, Anna K.; Hill, Laura G.; Koehler, Christian R.

2014-01-01

The move to build capacity for internal evaluation is a common organizational theme in social service delivery, and in many settings, the evaluator is also the practitioner who delivers the service. The goal of the present study was to extend our limited knowledge of practitioner evaluation. Specifically, the authors examined practitioner concerns about administering pretest and posttest evaluations within the context of a multisite 7-week family strengthening program and compared those concerns with self-reported attitudes of the parents who completed evaluations. The authors found that program participants (n = 105) were significantly less likely to find the evaluation process intrusive, and more likely to hold positive beliefs about the evaluation process, than practitioners (n = 140) expected. Results of the study may address a potential barrier to effective practitioner evaluation—the belief that having to administer evaluations interferes with establishing a good relationship with program participants. PMID:25328379

Using evaluation methods to guide the development of a tobacco-use prevention curriculum for youth: a case study.

PubMed

Bridge, P D; Gallagher, R E; Berry-Bobovski, L C

2000-01-01

Fundamental to the development of educational programs and curricula is the evaluation of processes and outcomes. Unfortunately, many otherwise well-designed programs do not incorporate stringent evaluation methods and are limited in measuring program development and effectiveness. Using an advertising lesson in a school-based tobacco-use prevention curriculum as a case study, the authors examine the role of evaluation in the development, implementation, and enhancement of the curricular lesson. A four-phase formative and summative evaluation design was developed to divide the program-evaluation continuum into a structured process that would aid in the management of the evaluation, as well as assess curricular components. Formative and summative evaluation can provide important guidance in the development, implementation, and enhancement of educational curricula. Evaluation strategies identified unexpected barriers and allowed the project team to make necessary "time-relevant" curricular adjustments during each stage of the process.

Evaluation of traction stirrup distraction technique to increase the joint space of the shoulder joint in the dog: A cadaveric study.

PubMed

Devesa, V; Rovesti, G L; Urrutia, P G; Sanroman, F; Rodriguez-Quiros, J

2015-06-01

The objective of this study was to evaluate technical feasibility and efficacy of a joint distraction technique by traction stirrup to facilitate shoulder arthroscopy and assess potential soft tissue damage. Twenty shoulders were evaluated radiographically before distraction. Distraction was applied with loads from 40 N up to 200 N, in 40 N increments, and the joint space was recorded at each step by radiographic images. The effects of joint flexion and intra-articular air injection at maximum load were evaluated. Radiographic evaluation was performed after distraction to evaluate ensuing joint laxity. Joint distraction by traction stirrup technique produces a significant increase in the joint space; an increase in joint laxity could not be inferred by standard and stress radiographs. However, further clinical studies are required to evaluate potential neurovascular complications. A wider joint space may be useful to facilitate arthroscopy, reducing the likelihood for iatrogenic damage to intra-articular structures. Copyright © 2015 Elsevier Ltd. All rights reserved.

The portfolio as an evaluation tool: an analysis of its use in an undergraduate nursing program.

PubMed

Friedrich, Denise Barbosa de Castro; Gonçalves, Angela Maria Corrêa; de Sá, Tatiana Santos; Sanglard, Leticia Ribeiro; Duque, Débora Ribeiro; de Oliveira, Gabriela Mota Antunes

2010-01-01

This qualitative study was carried out between April and August 2007. It analyzed the use of portfolios in the academic community. A total of nine full-time professors and 119 students enrolled in their third semester were interviewed through a semi-structured interview. Content analysis was used to analyze data. Learning evaluations are seen as a verification of knowledge and efficacy of pedagogical method, and also as an incentive to study. Evaluations are procedural, that is, evaluation is continuous, or one-time, e.g. semester end tests. The portfolio is defined as a gradual and continuous evaluation tool. The faculty members and students need to accept the use of portfolios and evaluate the possibilities of this resource. This study is a first attempt to appraise the evaluation process of an undergraduate program, and the use of portfolios and other strategies needs to be consolidated in order to improve the educational process in undergraduate nursing programs.

38 CFR 1.17 - Evaluation of studies relating to health effects of radiation exposure.

Code of Federal Regulations, 2012 CFR

2012-07-01

... health effects of radiation exposure. (a) From time to time, the Secretary shall publish evaluations of... paragraph a valid study is one which: (i) Has adequately described the study design and methods of data... studies affecting epidemiological assessments including case series, correlational studies and studies...

38 CFR 1.17 - Evaluation of studies relating to health effects of radiation exposure.

Code of Federal Regulations, 2013 CFR

2013-07-01

... health effects of radiation exposure. (a) From time to time, the Secretary shall publish evaluations of... paragraph a valid study is one which: (i) Has adequately described the study design and methods of data... studies affecting epidemiological assessments including case series, correlational studies and studies...

38 CFR 1.17 - Evaluation of studies relating to health effects of radiation exposure.

Code of Federal Regulations, 2014 CFR

2014-07-01

... health effects of radiation exposure. (a) From time to time, the Secretary shall publish evaluations of... paragraph a valid study is one which: (i) Has adequately described the study design and methods of data... studies affecting epidemiological assessments including case series, correlational studies and studies...

38 CFR 1.17 - Evaluation of studies relating to health effects of radiation exposure.

Code of Federal Regulations, 2010 CFR

2010-07-01

... health effects of radiation exposure. (a) From time to time, the Secretary shall publish evaluations of... paragraph a valid study is one which: (i) Has adequately described the study design and methods of data... studies affecting epidemiological assessments including case series, correlational studies and studies...

38 CFR 1.17 - Evaluation of studies relating to health effects of radiation exposure.

Code of Federal Regulations, 2011 CFR

2011-07-01

... health effects of radiation exposure. (a) From time to time, the Secretary shall publish evaluations of... paragraph a valid study is one which: (i) Has adequately described the study design and methods of data... studies affecting epidemiological assessments including case series, correlational studies and studies...

[Evaluation by case managers dementia : An explorative practice based study on types and content].

PubMed

Ketelaar, Nicole A B M; Jukema, Jan S; van Bemmel, Marlies; Adriaansen, Marian J M; Smits, Carolien H M

2017-06-01

This practice based explorative study aims to provide insight into the ways in which case managers shape and fill up the evaluation phase of their support of the informal care network of persons with dementia. A combination of quantitative and qualitative research methods were used. A group of 57 case managers of persons with dementia in three different organisational networks took part in this study. Results from the quantitative and qualitative data are organized into four themes: (1) attitude towards evaluation, (2) forms of evaluation, (3) implementation of evaluation and (4) content of evaluation. There are different ways in shaping evaluation and the content of it. The importance of interim and final evaluation is recognized, but is difficult to realize in a methodical way. Barriers experienced by the case managers include various factors associated both with clients as professionals. Case managers evaluate continuously and in an informal way to assess whether the extent of their assistance is meeting the needs of the client and informal network. Case managers do not use systematic evaluation to measure the quality of care they offer to persons with dementia and their caregivers. The findings demand a discussion on the level of clients, as well as on the professional and societal level about the way case managers should evaluate their support.

Economic evaluations in shoulder pathologies: a systematic review of the literature.

PubMed

Kuye, Ifedayo O; Jain, Nitin B; Warner, Lauren; Herndon, James H; Warner, Jon J P

2012-03-01

Economic evaluations provide decision makers with a tool for reducing health care costs because they assess both the costs and consequences of health care interventions. This study reviewed the quality of published economic evaluations for shoulder pathologies. A MEDLINE search was conducted to identify articles published from 1980 to 2010 that contained "cost" or "economic" combined with terms for several shoulder disorders and treatments. We selected studies that fit the definition of 1 of the 4 routinely performed economic evaluations: cost-minimization, cost-effectiveness, cost-utility, and cost-benefit analyses. Study quality was determined by measuring adherence to 6 established health economic principles, as described in the literature. The search retrieved 942 studies. Of these, 32 were determined to be economic evaluations, and 53% of the economic evaluations were published from 2005 to 2010. Only 8 of the 32 studies (25%) adhered to all 6 health economic principles. Publication in a nonsurgical journal (P < .05) or in more recent years (P < .01) was significantly associated with higher quality. Future health care resource allocation will likely be based on the economic feasibility of treatments. Although the number and quality of economic evaluations of shoulder disorders have risen in recent years, the current state of the literature is poor. Given that availability of such data may factor in private and public reimbursement decisions, there is a clear demand for more rigorous economic evaluations. Copyright © 2012 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Mosby, Inc. All rights reserved.

Vocational Education Outcomes: An Evaluative Bibliography of Empirical Studies. Bibliography Series No. 49.

ERIC Educational Resources Information Center

Bolland, Kathleen A.

As part of a project designed to identify appropriate and feasible methods of evaluating vocational education with respect to its outcomes, this bibliography contains descriptive and evaluative annotations of a selection of empirical studies of vocational education outcomes. Following a technical introduction discussing the methodological issues…

The Problem of Evaluative Categorization of Human Intelligence in Linguistic World Images

ERIC Educational Resources Information Center

Abisheva, Klara M.; Dossanova, Altynay Zh.; Ismakova, Bibissara S.; Aupova, Gulbagira K.; Ayapbergenov, Bulat K.; Tlegenova, Kulyan A.

2016-01-01

The aim of the research is to determine the peculiarities of the evaluative categorization of human intelligence in linguistic world images. The study describes the interdisciplinary approach to studying evaluative categorization, which assumes the use of complex methodology including the anthropocentric, the interdisciplinary, and the cognitive…

The Development of Message Evaluation Skills in Young Children.

ERIC Educational Resources Information Center

Beal, Carole R.; Belgrad, Susan L.

Two studies investigated the origins of message evaluation skills in preschool children. In the first study, 14 preschoolers and 14 second graders were asked to evaluate the informativeness of pictorial messages in a referential communication task in two sessions. Results showed that the younger children overestimated the informativeness of…

Conflicting Expertise and Uncertainty: Quality Assurance in High-Level Radioactive Waste Management.

ERIC Educational Resources Information Center

Fitzgerald, Michael R.; McCabe, Amy Snyder

1991-01-01

Dynamics of a large, expensive, and controversial surface and underground evaluation of a radioactive waste management program at the Yucca Mountain power plant are reviewed. The use of private contractors in the quality assurance study complicates the evaluation. This case study illustrates high stakes evaluation problems. (SLD)

Enhancing Learning Outcomes through Evaluation of Serious Gaming: A Mixed Methods Study

ERIC Educational Resources Information Center

Douglas, Kerrie Anna

2012-01-01

This study compared the change in counseling student's self-efficacy and skill related to suicide assessment and intervention through the use of a novel intervention-oriented evaluation method, evaluation focused discussion groups, in an experimental embedded mixed methods design. An innovation counselor pedagogical tool, Suicide Risk Assessment…

Evaluation of Training Programs for Rural Development

ERIC Educational Resources Information Center

Indira, A.

2008-01-01

An Evaluation of the "Impact Assessment of the Training Programs" of a National Level Training Institution in India was conducted using the Kirkpatrick Method (KP Method). The studied Institution takes up research, provides training, offers consultancy and initiates action in the rural sector of India. The evaluation study used a…

Connecting Theory to Practice: Evaluating a Brain-Based Writing Curriculum

ERIC Educational Resources Information Center

Griffee, Dale T.

2007-01-01

This 10 week longitudinal evaluation study evaluated a brain-based learning curriculum proposed by Smilkstein (2003) by comparing student performance in a traditional basic writing curriculum with NHLP-oriented basic writing curriculum. The study included two classes each of experimental and traditional methods. Results of the data, gathered by…

Escaping the Dusty Shelf: Legislative Evaluation Offices' Efforts to Promote Utilization

ERIC Educational Resources Information Center

Vanlandingham, Gary R.

2011-01-01

Although scholars have proposed many steps to increase evaluation use, there has been little comparative empirical study of whether researchers follow these recommendations and whether specific steps are associated with greater utilization. This study of state legislative evaluators finds that those that regularly meet with stakeholders and…

Bench-Scale Evaluation Of Chemically Bonded Phosphate Ceramic Technology To Stabilize Mercury Waste Mixtures

EPA Science Inventory

This bench-scale study was conducted to evaluate the stabilization of mercury (Hg) and mercuric chloride-containing surrogate test materials by the chemically bonded phosphate ceramics technology. This study was performed as part of a U.S. EPA program to evaluate treatment and d...

Teacher Evaluation in Colorado: How Policy Frustrates Practice

ERIC Educational Resources Information Center

Ramirez, Al; Clouse, Wendi; Davis, Kristyn White

2014-01-01

This article is a report of a study that used data from multiple sources to explore the hypothesis that systemic barriers inherent in Colorado's teacher evaluation policies often contribute to ineffective teacher evaluations across the state. Data were collected from extant studies, focus groups, and surveys of teachers, site administrators/head…

Linking Job-Embedded Professional Development and Mandated Teacher Evaluation: Teacher as Learner

ERIC Educational Resources Information Center

Derrington, Mary Lynne; Kirk, Julia

2017-01-01

This study explores the link between individualized, job-embedded professional development and teacher evaluation. Moreover, the study explores and describes job-embedded strategies that principals used to facilitate teacher development while working within a state-mandated evaluation system. The theoretical frame utilized four elements of…

42 CFR 456.141 - Purpose and general description.

Code of Federal Regulations, 2010 CFR

2010-10-01

... care evaluation studies— (1) Emphasize identification and analysis of patterns of patient care; and (2...: Medical Care Evaluation Studies § 456.141 Purpose and general description. (a) The purpose of medical care evaluation studies is to promote the most effective and efficient use of available health facilities and...

42 CFR 456.141 - Purpose and general description.

Code of Federal Regulations, 2011 CFR

2011-10-01

... care evaluation studies— (1) Emphasize identification and analysis of patterns of patient care; and (2...: Medical Care Evaluation Studies § 456.141 Purpose and general description. (a) The purpose of medical care evaluation studies is to promote the most effective and efficient use of available health facilities and...

A Mixed-Methods Longitudinal Evaluation of a One-Day Mental Health Wellness Intervention

ERIC Educational Resources Information Center

Doyle, Louise; de Vries, Jan; Higgins, Agnes; Keogh, Brian; McBennett, Padraig; O'Shea, Marié T.

2017-01-01

Objectives: This study evaluated the impact of a one-day mental health Wellness Workshop on participants' mental health and attitudes towards mental health. Design: Convergent, longitudinal mixed-methods approach. Setting: The study evaluated Wellness Workshops which took place throughout the Republic of Ireland. Method: Questionnaires measuring…

Safety evaluation of installing center two-way left-turn lanes on two-lane roads

DOT National Transportation Integrated Search

2008-03-01

The Federal Highway Administration organized a Pooled Fund Study of 26 States to evaluate low-cost safety strategies as part of its strategic highway safety effort. One of the strategies chosen to be evaluated for this study was the installation of c...

Best practices for evaluating the capability of nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for damage characterization

NASA Astrophysics Data System (ADS)

Aldrin, John C.; Annis, Charles; Sabbagh, Harold A.; Lindgren, Eric A.

2016-02-01

A comprehensive approach to NDE and SHM characterization error (CE) evaluation is presented that follows the framework of the `ahat-versus-a' regression analysis for POD assessment. Characterization capability evaluation is typically more complex with respect to current POD evaluations and thus requires engineering and statistical expertise in the model-building process to ensure all key effects and interactions are addressed. Justifying the statistical model choice with underlying assumptions is key. Several sizing case studies are presented with detailed evaluations of the most appropriate statistical model for each data set. The use of a model-assisted approach is introduced to help assess the reliability of NDE and SHM characterization capability under a wide range of part, environmental and damage conditions. Best practices of using models are presented for both an eddy current NDE sizing and vibration-based SHM case studies. The results of these studies highlight the general protocol feasibility, emphasize the importance of evaluating key application characteristics prior to the study, and demonstrate an approach to quantify the role of varying SHM sensor durability and environmental conditions on characterization performance.

Are complementary therapies and integrative care cost-effective? A systematic review of economic evaluations

PubMed Central

Herman, Patricia M; Poindexter, Beth L; Witt, Claudia M; Eisenberg, David M

2012-01-01

Objective A comprehensive systematic review of economic evaluations of complementary and integrative medicine (CIM) to establish the value of these therapies to health reform efforts. Data sources PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE were searched from inception through 2010. In addition, bibliographies of found articles and reviews were searched, and key researchers were contacted. Eligibility criteria for selecting studies Studies of CIM were identified using criteria based on those of the Cochrane complementary and alternative medicine group. All studies of CIM reporting economic outcomes were included. Study appraisal methods All recent (and likely most cost-relevant) full economic evaluations published 2001–2010 were subjected to several measures of quality. Detailed results of higher-quality studies are reported. Results A total of 338 economic evaluations of CIM were identified, of which 204, covering a wide variety of CIM for different populations, were published 2001–2010. A total of 114 of these were full economic evaluations. And 90% of these articles covered studies of single CIM therapies and only one compared usual care to usual care plus access to multiple licensed CIM practitioners. Of the recent full evaluations, 31 (27%) met five study-quality criteria, and 22 of these also met the minimum criterion for study transferability (‘generalisability’). Of the 56 comparisons made in the higher-quality studies, 16 (29%) show a health improvement with cost savings for the CIM therapy versus usual care. Study quality of the cost-utility analyses (CUAs) of CIM was generally comparable to that seen in CUAs across all medicine according to several measures, and the quality of the cost-saving studies was slightly, but not significantly, lower than those showing cost increases (85% vs 88%, p=0.460). Conclusions This comprehensive review identified many CIM economic evaluations missed by previous reviews and emerging evidence of cost-effectiveness and possible cost savings in at least a few clinical populations. Recommendations are made for future studies. PMID:22945962

Clinical Utility of Serologic Testing for Celiac Disease in Asymptomatic Patients

PubMed Central

2011-01-01

Executive Summary Objective The objective of this evidence-based analysis was to evaluate the clinical utility of serologic testing for celiac disease in asymptomatic individuals presenting with one of the non-gastrointestinal conditions evaluated in this report. The clinical utility was based on the effects of a gluten-free diet (GFD) on outcomes specific to each of these conditions. The prevalence of celiac disease in asymptomatic individuals and one of these non-gastrointestinal conditions was also evaluated. Clinical Need and Target Population Celiac Disease Celiac disease is an autoimmune disease characterized by a chronic inflammatory state of the proximal small bowel mucosa accompanied by structural and functional changes. Technology Under Evaluation Serologic Tests for Celiac Disease There are a number of serologic tests for celiac disease available. Serologic tests are automated with the exception of the anti-endomysial antibody test, which is more time-consuming and operator-dependent than the other tests. Research Questions What is the prevalence of asymptomatic celiac disease in patients presenting with one of the non-gastrointestinal conditions evaluated? What is the effect of the gluten-free diet on condition-specific outcomes in patients with asymptomatic celiac disease presenting with one of the non-gastrointestinal conditions evaluated? What is the clinical utility of serologic testing for celiac disease in asymptomatic patients presenting with one of the non-gastrointestinal conditions evaluated? The clinical utility was defined as the impact of the GFD on disease specific outcomes. What is the risk of all-cause mortality and lymphoma in individuals with asymptomatic celiac disease? What is the budget impact of serologic testing for celiac disease in asymptomatic subjects presenting with one of the non-gastrointestinal conditions evaluated? Research Methods Study Population The study population consisted of individuals with newly diagnosed celiac disease without any symptoms consistent with the disease presenting with one of the non-gastrointestinal conditions evaluated. When evaluating the risk of lymphoma and all-cause mortality, the study population consisted of asymptomatic individuals with a positive celiac disease serologic test and/or small bowel biopsy. Literature Search Search Strategy Literature searches were performed for each disease/condition evaluated between December 2010 and March 2011 using OVID MEDLINE, the Cochrane Library, and the International Agency for Health Technology Assessment (INAHTA). No restrictions for start date of search were used. Abstracts were reviewed by a single reviewer and, for those studies meeting the eligibility criteria, full-text articles were obtained. Reference lists were also examined for any additional relevant studies not identified through the search. Articles with an unknown eligibility were reviewed with a second clinical epidemiologist and then a group of epidemiologists until consensus was established. Inclusion Criteria Studies, systematic reviews, and meta-analyses that assessed the effects of a GFD in patients with newly diagnosed asymptomatic celiac disease presenting with one of the non-gastrointestinal conditions evaluated. If symptoms were not reported in the study but subjects were identified through screening for celiac disease the study was included. Studies, systematic reviews, and meta-analyses that assessed the prevalence of newly diagnosed asymptomatic celiac disease in patients with one of the non-gastrointestinal conditions evaluated. If symptoms were not reported in the study but subjects were identified through screening for celiac disease the study was included. Studies, systematic reviews, and meta-analyses that evaluated the risk of all-cause mortality or lymphoma in individuals with asymptomatic celiac disease. Sample size ≥ 10. Publications in English. Exclusion Criteria Studies that retrospectively assessed the prevalence of asymptomatic celiac disease. Studies that reported the prevalence of one of the non-gastrointestinal conditions evaluated in subjects already diagnosed with celiac disease. Studies in individuals with one of the non-gastrointestinal conditions evaluated if the condition could be explained by other causes. Studies in subjects with celiac disease and symptoms consistent with the disease. If the study included individuals with and without symptoms consistent with celiac disease and their results were analysed separately, the results in individuals without symptoms were included in the analysis. Studies in which individuals did not report any symptoms consistent with celiac disease at study start but that either retrospectively reported the presence of such symptoms after following a GFD, or that previously presented with symptoms consistent with celiac disease. Study results published in letters to the editor or comments about other studies. Studies with a sample size ≥ 10, however, in which less than 10 patients were included in the analysis. Outcomes of Interest The effects of a GFD on disease-specific outcomes for each condition evaluated in patients with asymptomatic celiac disease was assessed. The prevalence of asymptomatic celiac disease in patients presenting with one of the conditions evaluated was also assessed. Results of Evidence-Based Analysis Three eligible observational studies evaluated the effects of GFD on growth parameters in subjects with asymptomatic celiac disease and idiopathic short stature. Four eligible observational studies evaluated the effects of GFD on metabolic control in subjects with asymptomatic celiac disease and type 1 diabetes. Five eligible observational studies evaluated the risk of all-cause mortality and five eligible observational studies evaluated the risk of lymphoma in subjects with asymptomatic celiac disease. No eligible studies on the effects of the GFD for the other conditions evaluated were identified. Twenty-three eligible studies measured the prevalence of asymptomatic celiac disease in subjects presenting with one of the conditions evaluated. Prevalence of Celiac Disease in Asymptomatic Patients The prevalence of celiac disease in asymptomatic patients presenting with one of the conditions evaluated was analysed. Most studies also included a control group that generally consisted of individuals randomly selected from the general population. Although there was a trend to a higher prevalence of asymptomatic celiac disease in individuals with the conditions evaluated compared to the controls, it only reached statistical significance in type 1 diabetes. No eligible prevalence studies were identified in patients with amenorrhea, delayed puberty, alopecia, and depression. The Effects of a Gluten-Free Diet on Disease-Specific Outcomes in Patients with Asymptomatic Celiac Disease The effects of GFD on metabolic control in patients with asymptomatic celiac disease and Type 1 Diabetes The effects of a GFD on metabolic control (HbA1c, number of hypoglycemic episodes, and changes in insulin dosage) in subjects with asymptomatic celiac disease and type 1 diabetes were evaluated. One prospective case-control study reported an increase in HbA1c levels in cases with type 1 diabetes and asymptomatic celiac disease after the introduction of a GFD, however, the clinical significance of this change is unclear. Only one eligible retrospective case-control study evaluated the effects of a GFD on hypoglycemia episodes and since there were inadequate details in the study about both the ascertainment and severity of hypoglycemia episodes in both cases and controls, it is not possible to draw conclusions regarding the effects of a GFD on hypoglycemia episodes based on this study. One prospective case-control study did not show a statistically significant change in insulin dosage between cases with type 1 diabetes and asymptomatic celiac disease and controls with type 1 diabetes either before or after the introduction of a GFD. No eligible studies that evaluated the effects of a GFD on the long-term outcomes of type 1 diabetes such as cardiovascular or renal events in patients with asymptomatic celiac disease were identified. The effects of a Gluten-Free Diet in Patients with Idiopathic Short Stature and Asymptomatic Celiac Disease A total of 3 eligible studies were identified. All studies consisted of case series that compared growth parameters in subjects with asymptomatic celiac disease and idiopathic short stature before and after the celiac disease was diagnosed and the GFD was instituted. Most subjects included in the studies demonstrated an improvement in growth parameters. Compliance with the GFD was not reported in the studies. The results of the studies suggest an increase in growth velocity in pediatric patients with asymptomatic celiac disease and idiopathic short stature once a GFD is introduced. Risk of lymphoma in patients with asymptomatic celiac disease One retrospective cohort study evaluated the risk of lymphoma in patients with asymptomatic celiac disease. The authors concluded that the number of events identified was low during the long follow-up period and that the risk of overall malignancies was not increased among patients with asymptomatic celiac disease. Risk of Asymptomatic Celiac Disease in Patients with Lymphoma Four case-control studies, one of which retrospective, evaluated the risk of asymptomatic celiac disease in patients newly diagnosed with lymphoma. One retrospective cohort study did not show an increase in the risk of lymphoma among subjects with asymptomatic celiac disease. Three prospective case-control studies did not find a statistically significant risk of asymptomatic celiac disease in patients with newly diagnosed lymphoma. Risk of All-Cause Mortality in Patients with Asymptomatic Celiac Disease A total of 5 studies that evaluated the risk of all-cause mortality in asymptomatic patients with celiac disease were identified. There were 5 cohort studies, 2 prospective and 3 retrospective. The two prospective studies did not show an increased risk of all-cause mortality in subjects with asymptomatic celiac disease. Grading of Evidence The quality of the evidence for each serologic tests evaluated based on the GRADE Working Group criteria. Overall, the quality of the evidence ranged from low to very low depending on the outcome evaluated. The Clinical Utility of Serologic Testing for Celiac Disease in Asymptomatic Subjects Eligible studies that evaluated the effects of a GFD on disease-specific outcomes were only identified for two of the conditions evaluated, type 1 diabetes and idiopathic short stature. The clinical utility of serologic testing for celiac disease in patients with type 1 diabetes without symptoms consistent with celiac disease was not demonstrated since the studies identified did not provide evidence of the impact of the GFD on either metabolic control or long-term outcomes in these patients. The clinical utility of serologic testing for celiac disease in patients with idiopathic short stature without symptoms consistent with celiac disease was demonstrated since the studies identified showed an acceleration in growth once the diagnosis of celiac disease was made and a GFD was introduced. The Budget Impact of Serologic Testing for Celiac Disease in Asymptomatic Patients The budget impact of serologic testing for celiac disease in asymptomatic patients was calculated for the conditions for which clinical utility for serologic testing was demonstrated. The budget impact in patients with idiopathic short stature without symptoms consistent with celiac disease was estimated as C$552,000 as calculated by multiplying the number of individuals in Ontario with idiopathic short stature that may be eligible for the test by the cost of the serologic test for celiac disease. Conclusions Based on a review of the literature, there is an increased risk of asymptomatic celiac disease in patients with type 1 diabetes. Based on low quality evidence, in patients with idiopathic short stature and asymptomatic celiac disease there is an acceleration in growth once a gluten-free diet is introduced. With the exception of idiopathic short stature, there was no published evidence of clinical utility of celiac disease testing in asymptomatic patients with respect to a gluten-free diet intervention in the other conditions evaluated. Based on low to very low quality evidence, asymptomatic celiac disease does not confer an increased risk of lymphoma or mortality. Similarly, in patients with lymphoma there is no increased risk of asymptomatic celiac disease. PMID:23074415

Interventions to Improve the Quality of Outpatient Specialty Referral Requests: A Systematic Review.

PubMed

Hendrickson, Chase D; Lacourciere, Stacy L; Zanetti, Cole A; Donaldson, Patrick C; Larson, Robin J

2016-09-01

Requests for outpatient specialty consultations occur frequently but often are of poor quality because of incompleteness. The authors searched bibliographic databases, trial registries, and references during October 2014 for studies evaluating interventions to improve the quality of outpatient specialty referral requests compared to usual practice. Two reviewers independently extracted data and assessed quality. Findings were qualitatively summarized for completeness of information relayed in a referral request within naturally emerging intervention categories. Of 3495 articles screened, 11 were eligible. All 3 studies evaluating software-based interventions found statistically significant improvements. Among 4 studies evaluating template/pro forma interventions, completeness was uniformly improved but with variable or unreported statistical significance. Of 4 studies evaluating educational interventions, 2 favored the intervention and 2 found no difference. One study evaluating referral management was negative. Current evidence for improving referral request quality is strongest for software-based interventions and templates, although methodological quality varied and findings may be setting specific. © The Author(s) 2015.

IDIOS: An innovative index for evaluating dental imaging-based osteoporosis screening indices.

PubMed

Barngkgei, Imad; Halboub, Esam; Almashraqi, Abeer Abdulkareem; Khattab, Razan; Al Haffar, Iyad

2016-09-01

The goal of this study was to develop a new index as an objective reference for evaluating current and newly developed indices used for osteoporosis screening based on dental images. Its name; IDIOS, stands for Index of Dental-imaging Indices of Osteoporosis Screening. A comprehensive PubMed search was conducted to retrieve studies on dental imaging-based indices for osteoporosis screening. The results of the eligible studies, along with other relevant criteria, were used to develop IDIOS, which has scores ranging from 0 (0%) to 15 (100%). The indices presented in the studies we included were then evaluated using IDIOS. The 104 studies that were included utilized 24, 4, and 9 indices derived from panoramic, periapical, and computed tomographic/cone-beam computed tomographic techniques, respectively. The IDIOS scores for these indices ranged from 0 (0%) to 11.75 (78.32%). IDIOS is a valuable reference index that facilitates the evaluation of other dental imaging-based osteoporosis screening indices. Furthermore, IDIOS can be utilized to evaluate the accuracy of newly developed indices.

Evaluations of Antigay Hate Crimes and Hate Crime Legislation: Independent and Differentially Predicted.

PubMed

Wilkinson, Wayne W; Peters, Christopher S

2017-08-11

Minimal studies have investigated individuals' evaluations of antigay hate crimes and hate crime legislation simultaneously, with most research focusing on one or the other. In a sample of 246 heterosexual undergraduates, the present study found that evaluations of antigay hate crimes and hate crime legislation were unrelated. Higher social dominance orientation (SDO) and crime control orientation scores were associated with more positive evaluations of antigay hate crimes. Positive evaluations of hate crime legislation were associated with more positive attitudes toward gay men and lesbians. We also found that the relationship between SDO and evaluations were mediated by crime control beliefs (for hate crimes evaluations) and antigay attitudes (for hate crime legislation evaluations). The present findings have possible implications for the manner in which organizations advocate for the extension of hate crime legislation to include sexual orientation.

Evaluation of professional supervision in Aotearoa/New Zealand: An interprofessional study.

PubMed

Davys, Allyson Mary; O'Connell, Michael; May, Janet; Burns, Beverley

2017-06-01

The evaluation of professional supervision has been a focus for discussion in the supervision literature over past decades. A review of the literature in this area, however, suggests that evaluation has been differently defined, variously addressed, and a range of outcomes reported. The present study reports the findings of the first stage of a three-stage study of evaluation in professional supervision in Aotearoa/New Zealand. Experienced practitioners from the four professions of counselling, mental health nursing, psychology, and social work were interviewed to explore how evaluation in professional supervision is understood and actioned in practice. Twenty four semistructured interviews were conducted with supervisees, supervisors, and managers from each of the identified professions. The findings from these interviews indicate that a majority of participants applied some form of evaluation to their supervision arrangement. These evaluations, however, did not reflect an overarching organizational or professional culture of formal evaluation, but rather, an individualized ad-hoc process initiated by one or both of the participants (supervisor and supervisee). These evaluations focussed predominantly on the process, rather than the outcomes, of supervision. While many respondents expressed interest in a formal process for evaluating supervision, a number of concerns were also raised. These concerns included a lack of evaluation skills and resource, the potential for formal evaluation to have a negative impact on the supervision relationship, the importance of maintaining the boundaries of confidentiality, and a wariness regarding the possible use of any information gathered. © 2016 Australian College of Mental Health Nurses Inc.

FBI fingerprint identification automation study. AIDS 3 evaluation report. Volume 1: Compendium

NASA Technical Reports Server (NTRS)

Mulhall, B. D. L.

1980-01-01

The primary features of the overall study are encompassed and an evaluation of an automation system is presented. Objectives of the study are described, methods of evaluation are summarized and conclusions about the system's feasibility are presented. Also included is a brief history of fingerprint automation activities within the FBI, the organization of the FBI, a bibliography of documents and records, a data dictionary and a reference set of all of the transparencies presented throughout the study.

Electronic prescribing: criteria for evaluating handheld prescribing systems and an evaluation of a new, handheld, wireless wide area network (WWAN) prescribing system.

PubMed

Goldblum, O M

2001-02-01

The objectives of this study were: 1) to establish criteria for evaluating handheld computerized prescribing systems; and 2) to evaluate out-of-box performance and features of a new, Palm Operating System (OS)-based, handheld, wireless wide area network (WWAN) prescribing system. The system consisted of a Palm Vx handheld organizer, a Novatel Minstrel V wireless modem, OmniSky wireless internet access and ePhysician ePad 1.1, the Palm OS electronic prescribing software program. A dermatologist familiar with healthcare information technology conducted an evaluation of the performance and features of a new, handheld, WWAN electronic prescribing system in an office practice during a three-month period in 2000. System performance, defined as transmission success rate, was determined from data collected during the three-month trial. Evaluation criteria consisted of an analysis of features found in electronic prescribing systems. All prescriptions written for all patients seen during a three-month period (August - November, 2000) were eligible for inclusion. Prescriptions written for patients who intended to fill them at pharmacies without known facsimile receiving capabilities were excluded from the study. The performance of the system was evaluated using data collected during the study. Criteria for evaluating features of electronic prescribing systems were developed and used to analyze the system employed in this study. During this three-month trial, 200 electronic prescriptions were generated for 132 patients included in the study. Of these prescriptions, 92.5 percent were successfully transmitted to pharmacies. Transmission failures resulted from incorrect facsimile numbers and non-functioning facsimile machines. Criteria established for evaluation of electronic prescribing systems included System (Hardware & Software), Costs, System Features, Printing & Transmission, Formulary & Insurance, Customization, Drug Safety and Security. This study is the first effort to establish comprehensive criteria for evaluating handheld prescribing systems and to evaluate the performance and features of a handheld, electronic prescribing system. The results demonstrated that the evaluated system: 1) was simple to install; 2) successfully interfaced with a commonly used practice management system; 3) was user-friendly and easy to operate; 4) offered a robust variety of standard features; and, 5) resulted in a high rate of success for transmitting electronic prescriptions. The criteria established for the evaluation of features of an electronic prescribing system can be used to critically evaluate the performance and features of other handheld and personal computer-based electronic prescribing systems.

A systematic review of studies evaluating diffusion and dissemination of selected cancer control interventions.

PubMed

Ellis, Peter; Robinson, Paula; Ciliska, Donna; Armour, Tanya; Brouwers, Melissa; O'Brien, Mary Ann; Sussman, Jonathan; Raina, Parminder

2005-09-01

With this review, the authors sought to determine what strategies have been evaluated (including the outcomes assessed) to disseminate cancer control interventions that promote the uptake of behavior change. Five topic areas along the cancer care continuum (smoking cessation, healthy diet, mammography, cervical cancer screening, and control of cancer pain) were selected to be representative. A systematic review was conducted of primary studies evaluating dissemination of a cancer control intervention. Thirty-one studies were identified that evaluated dissemination strategies in the 5 topic areas. No strong evidence currently exists to recommend any one dissemination strategy as effective in promoting the uptake of cancer control interventions. The authors conclude that there is a strong need for more research into dissemination of cancer control interventions. Future research should consider methodological issues such as the most appropriate study design and outcomes to be evaluated. (c) 2005 APA, all rights reserved

The effects of spatially displaced visual feedback on remote manipulator performance

NASA Technical Reports Server (NTRS)

Smith, Randy L.; Stuart, Mark A.

1993-01-01

The results of this evaluation have important implications for the arrangement of remote manipulation worksites and the design of workstations for telerobot operations. This study clearly illustrates the deleterious effects that can accompany the performance of remote manipulator tasks when viewing conditions are less than optimal. Future evaluations should emphasize telerobot camera locations and the use of image/graphical enhancement techniques in an attempt to lessen the adverse effects of displaced visual feedback. An important finding in this evaluation is the extent to which results from previously performed direct manipulation studies can be generalized to remote manipulation studies. Even though the results obtained were very similar to those of the direct manipulation evaluations, there were differences as well. This evaluation has demonstrated that generalizations to remote manipulation applications based upon the results of direct manipulation studies are quite useful, but they should be made cautiously.

Maritime Platform Sleep and Performance Study: Evaluating the SAFTE Model for Maritime Workplace Application

DTIC Science & Technology

2012-06-01

SLEEP AND PERFORMANCE STUDY: EVALUATING THE SAFTE MODEL FOR MARITIME WORKPLACE APPLICATION by Stephanie A. T. Brown June 2012 Thesis...REPORT DATE June 2012 3. REPORT TYPE AND DATES COVERED Master’s Thesis 4. TITLE AND SUBTITLE Maritime Platform Sleep and Performance Study...Evaluating the SAFTE Model for Maritime Workplace Application 5. FUNDING NUMBERS 6. AUTHOR(S) Stephanie A. T. Brown 7. PERFORMING ORGANIZATION

Using Pre-test/Post-test Data To Evaluate the Effectiveness of Computer Aided Instruction (A Study of CAI and Its Use with Developmental Reading Students).

ERIC Educational Resources Information Center

Lansford, Carl E.

As computer aided instruction (CAI) and distance learning become more popular, a model for easily evaluating these teaching methods must be developed, one which will enable replication of the study each year. This paper discusses the results of a study using existing dependent and independent variables to evaluate CAI for developmental reading…

Patients' evaluations of European general practice--revisited after 11 years.

PubMed

Petek, Davorina; Künzi, Beat; Kersnik, Janko; Szecsenyi, Joachim; Wensing, Michel

2011-12-01

In the last decade many things have changed in healthcare systems, primary care practices and populations. To describe evaluations of general practice care by patients with a chronic illness in 2009 and compare these with a previous study done in 1998. A descriptive analysis of patients' evaluations, using data from the European practice assessment Cardio study on cardiovascular patients in eight European countries in 2009. We compared these evaluations with a subgroup of patients with self-defined chronic illness from the study in 1998, using a linear regression model. Patients' evaluation of general practice using the EUROPEP questionnaire. The EUROPEP is a 23-item validated measure of patient evaluations of general practice care. In 2009, 7472 patients from 251 practices participated in the study with an overall response rate of 49.6%. The percentage of patients with positive evaluations (good/excellent) was 80% or higher for all items, except for the waiting time. More positive evaluations were found in older patients, patients with a longer attachment to the practice, patients with a higher self-evaluation of their health, patients with fewer mental health problems and less pain/discomfort. The comparison between 1998 and 2009 showed no overall trends for all countries combined. Whereas English patients became fairly more positive about general practice in 2009, German patients became slightly less positive, although still more positive than English patients. Overall, the patients' evaluations of general practice were very positive in family practice care in the years 1998 and 2009. The trends over the years need to be carefully interpreted over time.

Evaluating trauma nursing education: An integrative literature review.

PubMed

Ding, Min; Metcalfe, Helene; Gallagher, Olivia; Hamdorf, Jeffrey M

2016-09-01

A review of the current literature evaluating trauma nursing education. A variety of trauma nursing courses exist, to educate nurses working in trauma settings, and to maintain their continuing professional development. Despite an increase in the number of courses delivered, there appears to be a lack of evidence to demonstrate the effectiveness of trauma nursing education and in particular the justification for this resource allocation. Integrative literature review. A search of international literature on trauma nursing education evaluation published in English from 1985 to 2015 was conducted through electronic databases CINAHL Plus, Google Scholar, PubMed, Austhealth, Science Citation Index Expanded (Web of Science), Sciverse Science Direct (Elsevier) & One file (Gale). Only peer reviewed journal articles identifying trauma course and trauma nursing course evaluation have been included in the selection criteria. An integrative review of both quantitative and qualitative literature guided by Whittemore and Knafl's theoretical framework using Bowling's and Pearson's validated appraisal checklists, has been conducted for three months. Only 17 studies met the inclusion criteria, including 14 on trauma course evaluation and 3 on trauma nursing course evaluation. Study findings are presented as two main themes: the historical evolution of trauma nursing education and evaluation of trauma nursing education outcomes. Trauma nursing remains in its infancy and education in this specialty is mainly led by continuing professional development courses. The shortage of evaluation studies on trauma nursing courses reflects the similar status in continuing professional development course evaluation. A trauma nursing course evaluation study will address the gap in this under researched area. Copyright © 2016 Elsevier Ltd. All rights reserved.

Meta-evaluation of published studies on evaluation of health disaster preparedness exercises through a systematic review.

PubMed

Sheikhbardsiri, Hojjat; Yarmohammadian, Mohammad H; Khankeh, Hamid Reza; Nekoei-Moghadam, Mahmoud; Raeisi, Ahmad Reza

2018-01-01

Exercise evaluation is one of the most important steps and sometimes neglected in designing and taking exercises, in this stage of exercise, it systematically identifying, gathering, and interpreting related information to indicate how an exercise has fulfilled its objectives. The present study aimed to assess the most important evaluation techniques applied in evaluating health exercises for emergencies and disasters. This was meta-evaluation study through a systematic review. In this research, we searched papers based on specific and relevant keywords in research databases including ISI web of science, PubMed, Scopus, Science Direct, Ovid, ProQuest, Wiley, Google Scholar, and Persian database such as ISC and SID. The search keywords and strategies are followed; "simulation," "practice," "drill," "exercise," "instrument," "tool," "questionnaire," " measurement," "checklist," "scale," "test," "inventory," "battery," "evaluation," "assessment," "appraisal," "emergency," "disaster," "cricise," "hazard," "catastrophe,: "hospital", "prehospital," "health centers," "treatment centers," were used in combination with Boolean operators OR and AND. The research findings indicate that there are different techniques and methods for data collection to evaluate performance exercises of health centers and affiliated organizations in disasters and emergencies including debriefing inventories, self-report, questionnaire, interview, observation, shooting video, and photographing, electronic equipment which can be individually or collectively used depending on exercise objectives or purposes. Taking exercise in the health sector is one of the important steps in preparation and implementation of disaster risk management programs. This study can be thus utilized to improve preparedness of different sectors of health system according to the latest available evaluation techniques and methods for better implementation of disaster exercise evaluation stages.

The use of information and communications technologies in the delivery of interprofessional education: A review of evaluation outcome levels.

PubMed

Curran, Vernon; Reid, Adam; Reis, Pamela; Doucet, Shelley; Price, Sheri; Alcock, Lindsay; Fitzgerald, Shari

2015-01-01

Interprofessional education (IPE) in health and human services educational and clinical settings has proliferated internationally. The use of information and communication technologies (ICTs) in the facilitation of interprofessional learning is also growing, yet reviews of the effectiveness of ICTs in the delivery of pre- and/or post-licensure IPE have been limited. The current study's purpose was to review the evaluation outcomes of IPE initiatives delivered using ICTs. Relevant electronic databases and journals from 1996 to 2013 were searched. Studies which evaluated the effectiveness of an IPE intervention using ICTs were included and analyzed using the Barr et al. modified Kirkpatrick educational outcomes typology. Fifty-five studies were identified and a majority reported evaluation findings at the level 1 (reaction/satisfaction). Analysis revealed that learners react favorably to the use of ICTs in the delivery of IPE, and ICT-mediated IPE can lead to positive attitudinal and knowledge change. A majority of the studies reported positive evaluation outcomes at the learner satisfaction level, with the use of web-based learning modalities. The limited number of studies at other levels of the outcomes typology and deficiencies in study designs indicate the need for more rigorous evaluation of outcomes in ICT-mediated IPE.

Quality Assessment of Published Articles in Iranian Journals Related to Economic Evaluation in Health Care Programs Based on Drummond's Checklist: A Narrative Review.

PubMed

Rezapour, Aziz; Jafari, Abdosaleh; Mirmasoudi, Kosha; Talebianpour, Hamid

2017-09-01

Health economic evaluation research plays an important role in selecting cost-effective interventions. The purpose of this study was to assess the quality of published articles in Iranian journals related to economic evaluation in health care programs based on Drummond's checklist in terms of numbers, features, and quality. In the present review study, published articles (Persian and English) in Iranian journals related to economic evaluation in health care programs were searched using electronic databases. In addition, the methodological quality of articles' structure was analyzed by Drummond's standard checklist. Based on the inclusion criteria, the search of databases resulted in 27 articles that fully covered economic evaluation in health care programs. A review of articles in accordance with Drummond's criteria showed that the majority of studies had flaws. The most common methodological weakness in the articles was in terms of cost calculation and valuation. Considering such methodological faults in these studies, it is anticipated that these studies would not provide an appropriate feedback to policy makers to allocate health care resources correctly and select suitable cost-effective interventions. Therefore, researchers are required to comply with the standard guidelines in order to better execute and report on economic evaluation studies.

Quality Assessment of Published Articles in Iranian Journals Related to Economic Evaluation in Health Care Programs Based on Drummond’s Checklist: A Narrative Review

PubMed Central

Rezapour, Aziz; Jafari, Abdosaleh; Mirmasoudi, Kosha; Talebianpour, Hamid

2017-01-01

Health economic evaluation research plays an important role in selecting cost-effective interventions. The purpose of this study was to assess the quality of published articles in Iranian journals related to economic evaluation in health care programs based on Drummond’s checklist in terms of numbers, features, and quality. In the present review study, published articles (Persian and English) in Iranian journals related to economic evaluation in health care programs were searched using electronic databases. In addition, the methodological quality of articles’ structure was analyzed by Drummond’s standard checklist. Based on the inclusion criteria, the search of databases resulted in 27 articles that fully covered economic evaluation in health care programs. A review of articles in accordance with Drummond’s criteria showed that the majority of studies had flaws. The most common methodological weakness in the articles was in terms of cost calculation and valuation. Considering such methodological faults in these studies, it is anticipated that these studies would not provide an appropriate feedback to policy makers to allocate health care resources correctly and select suitable cost-effective interventions. Therefore, researchers are required to comply with the standard guidelines in order to better execute and report on economic evaluation studies. PMID:29234174

Usability evaluation techniques in mobile commerce applications: A systematic review

NASA Astrophysics Data System (ADS)

Hussain, Azham; Mkpojiogu, Emmanuel O. C.

2016-08-01

Obviously, there are a number of literatures concerning the usability of mobile commerce (m-commerce) applications and related areas, but they do not adequately provide knowledge about usability techniques used in most of the empirical usability evaluation for m-commerce application. Therefore, this paper is aimed at producing the usability techniques frequently used in the aspect of usability evaluation for m-commerce applications. To achieve the stated objective, systematic literature review was employed. Sixty seven papers were downloaded in usability evaluation for m-commerce and related areas; twenty one most relevant studies were selected for review in order to extract the appropriate information. The results from the review shows that heuristic evaluation, formal test and think aloud methods are the most commonly used methods in m-commerce application in comparison to cognitive walkthrough and the informal test methods. Moreover, most of the studies applied control experiment (33.3% of the total studies); other studies that applied case study for usability evaluation are 14.28%. The results from this paper provide additional knowledge to the usability practitioners and research community for the current state and use of usability techniques in m-commerce application.

Testing and refining the Science in Risk Assessment and Policy (SciRAP) web-based platform for evaluating the reliability and relevance of in vivo toxicity studies.

PubMed

Beronius, Anna; Molander, Linda; Zilliacus, Johanna; Rudén, Christina; Hanberg, Annika

2018-05-28

The Science in Risk Assessment and Policy (SciRAP) web-based platform was developed to promote and facilitate structure and transparency in the evaluation of ecotoxicity and toxicity studies for hazard and risk assessment of chemicals. The platform includes sets of criteria and a colour-coding tool for evaluating the reliability and relevance of individual studies. The SciRAP method for evaluating in vivo toxicity studies was first published in 2014 and the aim of the work presented here was to evaluate and develop that method further. Toxicologists and risk assessors from different sectors and geographical areas were invited to test the SciRAP criteria and tool on a specific set of in vivo toxicity studies and to provide feedback concerning the scientific soundness and user-friendliness of the SciRAP approach. The results of this expert assessment were used to refine and improve both the evaluation criteria and the colour-coding tool. It is expected that the SciRAP web-based platform will continue to be developed and enhanced to keep up to date with the needs of end-users. Copyright © 2018 John Wiley & Sons, Ltd.

Examining the impact of sexism on evaluations of social scientific evidence in discrimination litigation.

PubMed

Kim, Anita; Tidwell, Natasha

2014-12-01

The present 2 studies involved undergraduate participants and investigated whether various types of sexism and other correlated predictors, such as political conservatism and scientific discounting, can predict people's evaluations of social science research on sex stereotypes, sexism, and sex discrimination. In Study 1, participants high in hostile sexism, scientific discounting, and/or political conservatism were more critical of scientific studies that provided evidence for sexism than identical studies showing null results. Study 2 showed that participants high in modern sexism, hostile sexism, and political conservatism evaluated social scientific studies more negatively; in addition, assessments of social scientific evidence quality mediated the effect of modern sexism on admissibility ratings (b = -0.15, z = -4.16, p = .00). Overall, these results suggest that sexist beliefs can bias one's judgments of social scientific evidence. Future research should explore whether the same psychological processes operate for judges and jurors as they evaluate the admissibility of evidence and examine ways to attenuate the effect of sexism on evaluations. PsycINFO Database Record (c) 2014 APA, all rights reserved.

2010 Perinatal GBS Prevention Guideline and Resource Utilization

PubMed Central

Mukhopadhyay, Sagori; Dukhovny, Dmitry; Mao, Wenyang; Eichenwald, Eric C.

2014-01-01

OBJECTIVES: To quantify differences in early-onset sepsis (EOS) evaluations, evaluation-associated resource utilization, and EOS cases detected, when comparing time periods before and after the implementation of an EOS algorithm based on the Centers for Disease Control and Prevention (CDC) 2010 guidelines for prevention of perinatal Group B Streptococcus (GBS) disease. METHODS: Retrospective cohort study of infants born at ≥36 weeks’ gestation from 2009 to 2012 in a single tertiary care center. One 12-month period during which EOS evaluations were based on the CDC 2002 guideline was compared with a second 12-month period during which EOS evaluations were based on the CDC 2010 guideline. A cost minimization analysis was performed to determine the EOS evaluation-associated costs and resources during each time period. RESULTS: During the study periods, among well-appearing infants ≥36 weeks’ gestation, EOS evaluations for inadequate GBS prophylaxis decreased from 32/1000 to <1/1000 live births; EOS evaluation-associated costs decreased by $6994 per 1000 live births; and EOS evaluation-associated work hours decreased by 29 per 1000 live births. We found no increase in EOS evaluations for other indications, total NICU admissions, frequency of infants evaluated for symptoms before hospital discharge, or incidence of EOS during the 2 study periods. CONCLUSIONS: Implementation of an EOS algorithm based on CDC 2010 GBS guidelines resulted in a 25% decrease in EOS evaluations performed among well-appearing infants ≥36 weeks’ gestation, attributable to decreased evaluation of infants born in the setting of inadequate indicated GBS prophylaxis. This resulted in significant changes in EOS evaluation-associated resource expenditures. PMID:24446442

Use of a structured functional evaluation process for independent medical evaluations of claimants presenting with disabling mental illness: rationale and design for a multi-center reliability study.

PubMed

Bachmann, Monica; de Boer, Wout; Schandelmaier, Stefan; Leibold, Andrea; Marelli, Renato; Jeger, Joerg; Hoffmann-Richter, Ulrike; Mager, Ralph; Schaad, Heinz; Zumbrunn, Thomas; Vogel, Nicole; Bänziger, Oskar; Busse, Jason W; Fischer, Katrin; Kunz, Regina

2016-07-29

Work capacity evaluations by independent medical experts are widely used to inform insurers whether injured or ill workers are capable of engaging in competitive employment. In many countries, evaluation processes lack a clearly structured approach, standardized instruments, and an explicit focus on claimants' functional abilities. Evaluation of subjective complaints, such as mental illness, present additional challenges in the determination of work capacity. We have therefore developed a process for functional evaluation of claimants with mental disorders which complements usual psychiatric evaluation. Here we report the design of a study to measure the reliability of our approach in determining work capacity among patients with mental illness applying for disability benefits. We will conduct a multi-center reliability study, in which 20 psychiatrists trained in our functional evaluation process will assess 30 claimants presenting with mental illness for eligibility to receive disability benefits [Reliability of Functional Evaluation in Psychiatry, RELY-study]. The functional evaluation process entails a five-step structured interview and a reporting instrument (Instrument of Functional Assessment in Psychiatry [IFAP]) to document the severity of work-related functional limitations. We will videotape all evaluations which will be viewed by three psychiatrists who will independently rate claimants' functional limitations. Our primary outcome measure is the evaluation of claimant's work capacity as a percentage (0 to 100 %), and our secondary outcomes are the 12 mental functions and 13 functional capacities assessed by the IFAP-instrument. Inter-rater reliability of four psychiatric experts will be explored using multilevel models to estimate the intraclass correlation coefficient (ICC). Additional analyses include subgroups according to mental disorder, the typicality of claimants, and claimant perceived fairness of the assessment process. We hypothesize that a structured functional approach will show moderate reliability (ICC ≥ 0.6) of psychiatric evaluation of work capacity. Enrollment of actual claimants with mental disorders referred for evaluation by disability/accident insurers will increase the external validity of our findings. Finding moderate levels of reliability, we will continue with a randomized trial to test the reliability of a structured functional approach versus evaluation-as-usual.

The quality of the evidence base for clinical pathway effectiveness: room for improvement in the design of evaluation trials.

PubMed

Rotter, Thomas; Kinsman, Leigh; James, Erica; Machotta, Andreas; Steyerberg, Ewout W

2012-06-18

The purpose of this article is to report on the quality of the existing evidence base regarding the effectiveness of clinical pathway (CPW) research in the hospital setting. The analysis is based on a recently published Cochrane review of the effectiveness of CPWs. An integral component of the review process was a rigorous appraisal of the methodological quality of published CPW evaluations. This allowed the identification of strengths and limitations of the evidence base for CPW effectiveness. We followed the validated Cochrane Effective Practice and Organisation of Care Group (EPOC) criteria for randomized and non-randomized clinical pathway evaluations. In addition, we tested the hypotheses that simple pre-post studies tend to overestimate CPW effects reported. Out of the 260 primary studies meeting CPW content criteria, only 27 studies met the EPOC study design criteria, with the majority of CPW studies (more than 70 %) excluded from the review on the basis that they were simple pre-post evaluations, mostly comparing two or more annual patient cohorts. Methodologically poor study designs are often used to evaluate CPWs and this compromises the quality of the existing evidence base. Cochrane EPOC methodological criteria, including the selection of rigorous study designs along with detailed descriptions of CPW development and implementation processes, are recommended for quantitative evaluations to improve the evidence base for the use of CPWs in hospitals.

Graphical Tests for Power Comparison of Competing Designs.

PubMed

Hofmann, H; Follett, L; Majumder, M; Cook, D

2012-12-01

Lineups have been established as tools for visual testing similar to standard statistical inference tests, allowing us to evaluate the validity of graphical findings in an objective manner. In simulation studies lineups have been shown as being efficient: the power of visual tests is comparable to classical tests while being much less stringent in terms of distributional assumptions made. This makes lineups versatile, yet powerful, tools in situations where conditions for regular statistical tests are not or cannot be met. In this paper we introduce lineups as a tool for evaluating the power of competing graphical designs. We highlight some of the theoretical properties and then show results from two studies evaluating competing designs: both studies are designed to go to the limits of our perceptual abilities to highlight differences between designs. We use both accuracy and speed of evaluation as measures of a successful design. The first study compares the choice of coordinate system: polar versus cartesian coordinates. The results show strong support in favor of cartesian coordinates in finding fast and accurate answers to spotting patterns. The second study is aimed at finding shift differences between distributions. Both studies are motivated by data problems that we have recently encountered, and explore using simulated data to evaluate the plot designs under controlled conditions. Amazon Mechanical Turk (MTurk) is used to conduct the studies. The lineups provide an effective mechanism for objectively evaluating plot designs.

The effect of a geriatric evaluation on treatment decisions for older cancer patients--a systematic review.

PubMed

Hamaker, Marije E; Schiphorst, Anandi H; ten Bokkel Huinink, Daan; Schaar, Cees; van Munster, Barbara C

2014-03-01

The aim of this systematic review is to summarise all available data on the effect of a geriatric evaluation on the multidisciplinary treatment of older cancer patients, focussing on oncologic treatment decisions and the implementation of non-oncologic interventions. A systematic search in MEDLINE and EMBASE for studies on the effect of a geriatric evaluation on oncologic and non-oncologic treatment for older cancer patients. Literature search identified 1654 reports (624 from Medline and 1030 from Embase), of which 10 studies were included in the review. Three studies used a geriatric consultation while seven used a geriatric assessment performed by a cancer specialist, healthcare worker or (research) nurse. Six studies addressed a change in oncologic treatment, the initial treatment plan was modified in a median of 39% of patients after geriatric evaluation, of which two thirds resulted in less intensive treatment. Seven studies focused on the implementation of non-oncologic interventions based on the results of the geriatric evaluation; all but one reported that interventions were suggested for over 70% of patients, even in studies that did not focus specifically on frail older patients. In the other study, implementation of non-oncologic interventions was left to the cancer specialist's discretion. A geriatric evaluation has significant impact on oncologic and non-oncologic treatment decisions in older cancer patients and deserves consideration in the oncologic work-up for these patients.

Paediatric health economic evaluations: a world view.

PubMed

Ungar, Wendy J

2007-01-01

As economic evaluation methods evolve, their applicability to special populations, such as children, has received increased scrutiny. The objective was to review paediatric health economic evaluations published over the last quarter century, comment on trends, discuss gaps between developed and developing nations, and point to future directions for research. Data compiled for the Paediatric Economic Database Evaluation (PEDE) project to 2003 were used to describe temporal and geographic trends and evaluate the frequency of intervention categories and conditions studied. The volume of paediatric health economic evaluations rose rapidly since 1980. Studies of infective/parasitic diseases, congenital anomalies and complications of pregnancy accounted for the majority. Prevention rather than treatment was emphasized. Most evaluations performed since 1998 (78%) were cost-effectiveness analyses. Cost-utility analyses were rare. The US produced half of all publications, with the U.K. contributing 12%. Economic evaluations from developing countries were uncommon, despite an urgent need for evidence-based decision-making in these regions. The interventions studied reflected local health priorities; HIV and malaria prevention were more commonly studied in developing nations, whereas treatments for asthma and birth malformations were more often evaluated in developed nations. Despite global initiatives to combat disease, developing nations rely on foreign research to inform implementation of local health programs. There is a need for better methods for data transfer and extrapolation. Future research must focus on paediatric models of costs and consequences and the development of tools to measure long-term effects.

Social media as an open-learning resource in medical education: current perspectives.

PubMed

Sutherland, S; Jalali, A

2017-01-01

Numerous studies evaluate the use of social media as an open-learning resource in education, but there is a little published knowledge of empirical evidence that such open-learning resources produce educative outcomes, particularly with regard to student performance. This study undertook a systematic review of the published literature in medical education to determine the state of the evidence as to empirical studies that conduct an evaluation or research regarding social media and open-learning resources. The authors searched MEDLINE, ERIC, Embase, PubMed, Scopus, and Google Scholar from 2012 to 2017. This search included using keywords related to social media, medical education, research, and evaluation, while restricting the search to peer reviewed, English language articles only. To meet inclusion criteria, manuscripts had to employ evaluative methods and undertake empirical research. Empirical work designed to evaluate the impact of social media as an open-learning resource in medical education is limited as only 13 studies met inclusion criteria. The majority of these studies used undergraduate medical education as the backdrop to investigate open-learning resources, such as Facebook, Twitter, and YouTube. YouTube appears to have little educational value due to the unsupervised nature of content added on a daily basis. Overall, extant reviews have demonstrated that we know a considerable amount about social media use, although to date, its impacts remain unclear. There is a paucity of outcome-based, empirical studies assessing the impact of social media in medical education. The few empirical studies identified tend to focus on evaluating the affective outcomes of social media and medical education as opposed to understanding any linkages between social media and performance outcomes. Given the potential for social media use in medical education, more empirical evaluative studies are required to determine educational value.

Evaluation of psychiatric symptoms in cocaine users in the Brazilian public health system: need for data and structure.

PubMed

Kessler, F; Woody, G; De Boni, R; Von Diemen, L; Benzano, D; Faller, S; Pechansky, F

2008-12-01

Few studies of comorbidity among cocaine users have been undertaken in Brazil, despite the fact that cocaine is one of the most commonly used illegal drugs in the country. The aim of this paper is to review existing data on psychiatric evaluations of cocaine users, and present data from two studies that have addressed this issue as it pertains to the Brazilian public health system. Review and results from two studies (cross-sectional and matched control). The Brazilian literature on PubMed, Lilacs, Psychinfo and DATASUS was searched using the key words: 'psychiatric symptoms', 'diagnosis', 'evaluation', 'assessment', 'cocaine disorders' and others related to this issue. Intake data from two studies of male and female cocaine users were also analysed with regard to psychiatric symptoms as measured by the Symptom Check List - 90 Revised (SCL-90). The literature review found no specific studies regarding psychiatric evaluation of cocaine users in Brazil. Analyses from the two studies presented showed high levels of psychiatric symptoms in this population. In the first study, psychiatric symptoms were measured at treatment entry and their prevalence was high, ranging from 27.4% to 53.4%. In the second study, SCL-90R scores at programme admission were higher in cocaine users than normal controls, with effect sizes ranging from moderate to high. To the authors' knowledge, this is the first article to discuss psychiatric evaluations of comorbidity among cocaine users in Brazil. The results indicate a need to: pay more attention to the evaluation of psychiatric symptoms in cocaine users; emphasize the importance of standardized data collection in this area; and evaluate the course of these symptoms, their impact on outcome, and how they are best addressed in treatment.

Pitfalls in the Evaluation of Teachers by Principals.

ERIC Educational Resources Information Center

Natriello, Gary; Dornbusch, Sanford M.

1980-01-01

Presents the findings of several studies of evaluation processes and identifies a model that helps to make explicit the components of the evaluation process. Suggests rules of thumb for conducting successful evaluations. (Author/JM)

Assessing Statistical Change Indices in Selected Social Work Intervention Research Studies

ERIC Educational Resources Information Center

Ham, Amanda D.; Huggins-Hoyt, Kimberly Y.; Pettus, Joelle

2016-01-01

Objectives: This study examined how evaluation and intervention research (IR) studies assessed statistical change to ascertain effectiveness. Methods: Studies from six core social work journals (2009-2013) were reviewed (N = 1,380). Fifty-two evaluation (n= 27) and intervention (n = 25) studies met the inclusion criteria. These studies were…

Economic Evaluations of Pathology Tests, 2010-2015: A Scoping Review.

PubMed

Watts, Rory D; Li, Ian W; Geelhoed, Elizabeth A; Sanfilippo, Frank M; St John, Andrew

2017-09-01

Concerns about pathology testing such as the value provided by new tests and the potential for inappropriate utilization have led to a greater need to assess costs and benefits. Economic evaluations are a formal method of analyzing costs and benefits, yet for pathology tests, questions remain about the scope and quality of the economic evidence. To describe the extent and quality of published evidence provided by economic evaluations of pathology tests from 2010 to 2015. Economic evaluations relating to pathology tests from 2010 to 2015 were reviewed. Eight databases were searched for published studies, and details recorded for the country, clinical focus, type of testing, and consideration of sensitivity, specificity, and false test results. The reporting quality of studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards checklist and cost-effectiveness ratios were analyzed for publication bias. We found 356 economic evaluations of pathology tests, most of which regarded developed countries. The most common economic evaluations were cost-utility analyses and the most common clinical focus was infectious diseases. More than half of the studies considered sensitivity and specificity, but few studies considered the impact of false test results. The average Consolidated Health Economic Evaluation Reporting Standards checklist score was 17 out of 24. Cost-utility ratios were commonly less than $10,000/quality-adjusted life-year or more than $200,000/quality-adjusted life-year. The number of economic evaluations of pathology tests has increased in recent years, but the rate of increase has plateaued. Furthermore, the quality of studies in the past 5 years was highly variable, and there is some question of publication bias in reporting cost-effectiveness ratios. Copyright © 2017 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Evaluating Educational Programmes: The Need and the Response.

ERIC Educational Resources Information Center

Stake, Robert E.

This survey of recent developments in educational program evaluation is intended for persons who commission, implement, direct, or carry out evaluation studies. The attitudes of government officials, educators, and researchers toward assessment and their own evaluation needs are discussed. Various approaches to evaluation are briefly described;…

Forestry research evaluation: current progress, future directions.

Treesearch

Christopher D. Risbrudt; Pamela J. Jakes

1985-01-01

Research evaluation is a relatively recent endeavor in forestry economics. This workshop represents most of the current and recently completed studies available in this subfield of forestry and evaluation. Also included are discussions of scientists and policymakers concerning the uses of forestry research evaluations, evaluation problems encountered, solutions...

Evaluation Use: Results from a Survey of U.S. American Evaluation Association Members

ERIC Educational Resources Information Center

Fleischer, Dreolin N.; Christie, Christina A.

2009-01-01

This paper presents the results of a cross-sectional survey on evaluation use completed by 1,140 U.S. American Evaluation Association members. This study had three foci: evaluators' current attitudes, perceptions, and experiences related to evaluation use theory and practice, how these data are similar to those reported in a previous study…

An Evaluation of Output Quality of Machine Translation (Padideh Software vs. Google Translate)

ERIC Educational Resources Information Center

Azer, Haniyeh Sadeghi; Aghayi, Mohammad Bagher

2015-01-01

This study aims to evaluate the translation quality of two machine translation systems in translating six different text-types, from English to Persian. The evaluation was based on criteria proposed by Van Slype (1979). The proposed model for evaluation is a black-box type, comparative and adequacy-oriented evaluation. To conduct the evaluation, a…

Model-based economic evaluations in smoking cessation and their transferability to new contexts: a systematic review.

PubMed

Berg, Marrit L; Cheung, Kei Long; Hiligsmann, Mickaël; Evers, Silvia; de Kinderen, Reina J A; Kulchaitanaroaj, Puttarin; Pokhrel, Subhash

2017-06-01

To identify different types of models used in economic evaluations of smoking cessation, analyse the quality of the included models examining their attributes and ascertain their transferability to a new context. A systematic review of the literature on the economic evaluation of smoking cessation interventions published between 1996 and April 2015, identified via Medline, EMBASE, National Health Service (NHS) Economic Evaluation Database (NHS EED), Health Technology Assessment (HTA). The checklist-based quality of the included studies and transferability scores was based on the European Network of Health Economic Evaluation Databases (EURONHEED) criteria. Studies that were not in smoking cessation, not original research, not a model-based economic evaluation, that did not consider adult population and not from a high-income country were excluded. Among the 64 economic evaluations included in the review, the state-transition Markov model was the most frequently used method (n = 30/64), with quality adjusted life years (QALY) being the most frequently used outcome measure in a life-time horizon. A small number of the included studies (13 of 64) were eligible for EURONHEED transferability checklist. The overall transferability scores ranged from 0.50 to 0.97, with an average score of 0.75. The average score per section was 0.69 (range = 0.35-0.92). The relative transferability of the studies could not be established due to a limitation present in the EURONHEED method. All existing economic evaluations in smoking cessation lack in one or more key study attributes necessary to be fully transferable to a new context. © 2017 The Authors. Addiction published by John Wiley & Sons Ltd on behalf of Society for the Study of Addiction.

Regulatory considerations on new adjuvants and delivery systems.

PubMed

Sesardic, D

2006-04-12

New and improved vaccines and delivery systems are increasingly being developed for prevention, treatment and diagnosis of human diseases. Prior to their use in humans, all new biological products must undergo pre-clinical evaluation. These pre-clinical studies are important not only to establish the biological properties of the material and to evaluate its possible risk to the public, but also to plan protocols for subsequent clinical trials from which safety and efficacy can be evaluated. For vaccines, evaluation in pre-clinical studies is particularly important as information gained may also contribute to identifying the optimum composition and formulation process and provide an opportunity to develop suitable indicator tests for quality control. Data from pre-clinical and laboratory evaluation studies, which continue during clinical studies, is used to support an application for marketing authorisation. Addition of a new adjuvant and exploration of new delivery systems for vaccines presents challenges to both manufacturers and regulatory authorities. Because no adjuvant is licensed as a medicinal product in its own right, but only as a component of a particular vaccine, pre-clinical and appropriate toxicology studies need to be designed on a case-by-case basis to evaluate the safety profile of the adjuvant and adjuvant/vaccine combination. Current regulatory requirements for the pharmaceutical and pre-clinical safety assessment of vaccines are insufficient and initiatives are in place to develop more specific guidelines for evaluation of adjuvants in vaccines.

The evaluator as technical assistant: A model for systemic reform support

NASA Astrophysics Data System (ADS)

Century, Jeanne Rose

This study explored evaluation of systemic reform. Specifically, it focused on the evaluation of a systemic effort to improve K-8 science, mathematics and technology education. The evaluation was of particular interest because it used both technical assistance and evaluation strategies. Through studying the combination of these roles, this investigation set out to increase understanding of potentially new evaluator roles, distinguish important characteristics of the evaluator/project participant relationship, and identify how these roles and characteristics contribute to effective evaluation of systemic science education reform. This qualitative study used interview, document analysis, and participant observation as methods of data collection. Interviews were conducted with project leaders, project participants, and evaluators and focused on the evaluation strategies and process, the use of the evaluation, and technical assistance. Documents analyzed included transcripts of evaluation team meetings and reports, memoranda and other print materials generated by the project leaders and the evaluators. Data analysis consisted of analytic and interpretive procedures consistent with the qualitative data collected and entailed a combined process of coding transcripts of interviews and meetings, field notes, and other documents; analyzing and organizing findings; writing of reflective and analytic memos; and designing and diagramming conceptual relationships. The data analysis resulted in the development of the Multi-Function Model for Systemic Reform Support. This model organizes systemic reform support into three functions: evaluation, technical assistance, and a third, named here as "systemic perspective." These functions work together to support the project's educational goals as well as a larger goal--building capacity in project participants. This model can now serve as an informed starting point or "blueprint" for strategically supporting systemic reform.

Evaluation through Follow-Up of Students (UGU). Project No. 2119. Current Project 1983:4.

ERIC Educational Resources Information Center

Ljung, Bengt-Olov; Emanuelsson, Ingemar

1983-01-01

This newsletter describes a research design enabling descriptive, comparative, evaluative follow-up studies of individual development in social contexts related to schooling. In addition, it describes a research project focusing mainly on evaluative studies of the ability of the educational system to confer, on different individuals in various…

Insights on Using Developmental Evaluation for Innovating: A Case Study on the Cocreation of an Innovative Program

ERIC Educational Resources Information Center

Lam, Chi Yan; Shulha, Lyn M.

2015-01-01

This article contributes to research on evaluation by examining the capacity and contribution of developmental evaluation for innovating. This case study describes the "preformative development" of an educational program (from conceptualization to pilot implementation) and analyzes the processes of innovation within a developmental…

Meta-analysis of landscape conservation plan evaluations

Treesearch

Michaela Foster; M. Nils Peterson; Frederick Cubbage; Gerard McMahon

2016-01-01

The number of studies evaluating the quality and content of many types of plans have grown in recent decades. Natural resource conservation plans have been included in some of these plan evaluation studies; however, no meta-analysis of natural resource planning literature has been conducted. This focus is needed because natural resource conservation planning differs...

From Curricular Justice to Educational Improvement: What Is the Role of Schools' Self-Evaluation?

ERIC Educational Resources Information Center

Sampaio, Marta; Leite, Carlinda

2017-01-01

This article presents a study that aimed to understand the contributions of self-evaluation (SE) processes towards the development of curricular and social justice and educational improvement. The study focuses on data collected from the schools' external evaluation (SEE) process and from the TEIP programme (Educational Territories of Priority…

An Evaluation of Some Motivational Materials Related to the History of Mathematics.

ERIC Educational Resources Information Center

Fraser, Barry J.; Koop, Anthony J.

This report evaluates motivational materials related to the history of mathematics. Materials produced included plays, written articles, slide-tape presentations, and videotapes. Two of these, plays and written articles, were chosen for intensive study. The study included a survey of teacher opinion, a student questionnaire, and evaluations by…

Evaluability Assessment Thesis and Dissertation Studies in Graduate Professional Degree Programs

ERIC Educational Resources Information Center

Walser, Tamara M.; Trevisan, Michael S.

2016-01-01

Evaluability assessment (EA) has potential as a design option for thesis and dissertation studies, serving as a practical training experience for both technical and nontechnical evaluation skills. Based on a content review of a sample of EA theses and dissertations from graduate professional degree programs, the authors of this article found that…

76 FR 74076 - Notice of Random Assignment Study To Evaluate the YouthBuild Program; Final Notice

Federal Register 2010, 2011, 2012, 2013, 2014

2011-11-30

... Evaluate the YouthBuild Program; Final Notice AGENCY: Employment and Training Administration (ETA), Labor... rigorous, nationally-representative estimates of the net impacts of the YouthBuild program. The Department... study. In the sites randomly selected to participate in this evaluation, all applicants for YouthBuild...

Evaluating the Effectiveness of Cognitive-Behavioral Treatment of Residual Symptoms and Impairment in Schizophrenia

ERIC Educational Resources Information Center

Bradshaw, William; Roseborough, David

2004-01-01

This study evaluated the effectiveness of cognitive-behavioral interventions in the treatment of persons with schizophrenia who experienced significant residual symptoms and impaired functioning despite their adherence to medication. The study used an aggregated AB single-system research design across 22 participants to evaluate change in clinical…

The Cost-Income Compenent of Program Evaluation.

ERIC Educational Resources Information Center

Miner, Norris

Cost-income studies are designed to serve two functions in instructional program evaluation. First, they act as the indicator of the economic value of a program. This economic value in conjunction with the other educational values needed in program evaluation allow for the most realistic appraisal of program worth. Second, if the studies show a…

Which Do Students Prefer to Evaluate Their Essays: Peers or Computer Program

ERIC Educational Resources Information Center

Lai, Yi-hsiu

2010-01-01

The purpose of this study was to investigate problems and potentials of new technologies in English writing education. The effectiveness of automated writing evaluation (AWE) ("MY Access") and of peer evaluation (PE) was compared. Twenty-two English as a foreign language (EFL) learners in Taiwan participated in this study. They submitted…

Evaluation of Student Engagement Assessment in Colorado State University's Warner College of Natural Resources

ERIC Educational Resources Information Center

Holman, Debra Kaye

2013-01-01

The purpose of this mixed methods study was to conduct a participatory program evaluation of student engagement assessment in Colorado State University's (CSU) Warner College of Natural Resources (WCNR). The college requested the evaluation after completing two pilot studies of undergraduate engagement which led them to consider establishing the…

Electrification Futures Study: A Technical Evaluation of the Impacts of an

Science.gov Websites

Technical Evaluation of the Impacts of an Electrified U.S. Energy System Electrification Futures Study: A Technical Evaluation of the Impacts of an Electrified U.S. Energy System Illustration showing various impacts of widespread electrification in the United States. In addition to NREL, the research team

Evaluation of the Centres of Excellence in Higher Education

ERIC Educational Resources Information Center

Kettunen, Juha Matti

2011-01-01

This study presents an evaluation of the centres of excellence in higher education in Finland. This approach is an example of enhancement-led evaluation aiming to improve the long-term development of education. The study presents the Degree Programme in Civil Engineering of the Turku University of Applied Sciences, which was awarded the…

Entropy of Linkography: Evaluating the Creativity of Short Animation

ERIC Educational Resources Information Center

Chou, Shyan-Bin; Chou, Huey-Wen; Chen, Yui-Liang

2013-01-01

There are important research issues surrounding the best methods for evaluating creativity of animation. This study proposed an approach to evaluate the creativity of short animation by adopting Linkography. This article reported the results of two experiments performed to test this approach. In study 1, participants identified as impressive a set…

Self-Study and Evaluation Guide. Section C-2: Accounting for Services and Finances.

ERIC Educational Resources Information Center

National Accreditation Council for Agencies Serving the Blind and Visually Handicapped, New York, NY.

The self study and evaluation guide, one of 16 documents for accreditation of agencies serving the blind and visually handicapped, focuses on accounting for services and finances. Standards address five areas: planning for accounting for services and finances, service accounting systems, financial accounting systems, evaluation, and public…

Cross-Cultural Differences in the Self-Evaluations of American and Finnish Elementary Teachers.

ERIC Educational Resources Information Center

Tirri, Kirsi

This study investigated the cross-cultural differences in American and Finnish elementary teachers' evaluations of their classroom teaching behaviors. The self-evaluation instrument developed for the study was administered to 167 American elementary teachers from Indiana and Texas and to 172 Finnish teachers (also elementary) from two different…

Educational Software Evaluation Form for Teachers

ERIC Educational Resources Information Center

Kara, Yilmaz

2007-01-01

The purpose of the study was to develop an educational software evaluation form to provide an evaluation and selection instrument of educational software that met the requirements of some balance between mechanics, content and pedagogy that is user friendly. The subjects for the study comprised a group of 32 biology teachers working in secondary…

Propofol Induction's Effect on Cardiac Function

ClinicalTrials.gov

2015-03-31

This Study Was Focused to Evaluate Feasibility of Doppler Tissue Monitoring During the Induction Anesthesia,; and Evaluate Routine Propofol Induction's Effect on Myocardial Tissue Motion, Using Non-invasive Doppler Tissue and 2D Speckle Tracking Imaging.; This is the First Study, to Our Knowledge, Which Has Evaluated the Possible Impact of Propofol Induction on LV Function.

"Evaluation as Illumination: A New Approach to the Study of Innovatory Programs". Occasional Paper.

ERIC Educational Resources Information Center

Parlett, Malcolm; Hamilton, David

Conventional approaches to program evaluation have followed the experimental and psychometric traditions dominant in educational research. Their aim (unfulfilled) of achieving fully objective methods had led to studies that are artificial and restricted in scope. Illuminative evaluation is introduced as a total re-appraisal of the rationale and…

Auditing and Evaluating University-Community Engagement: Lessons from a UK Case Study

ERIC Educational Resources Information Center

Hart, Angie; Northmore, Simon

2011-01-01

The growing importance of community and public engagement activities in universities has led to an increasing emphasis on auditing and evaluating university-community partnerships. However, the development of effective audit and evaluation tools is still at a formative stage. This article presents a case study of the University of Brighton's…

A Proposed Methodology for Contextualised Evaluation in Higher Education

ERIC Educational Resources Information Center

Nygaard, Claus; Belluigi, Dina Zoe

2011-01-01

This paper aims to inspire stakeholders working with quality of higher education (such as members of study boards, study programme directors, curriculum developers and teachers) to critically consider their evaluation methods in relation to a focus on student learning. We argue that many of the existing methods of evaluation in higher education…

Properties of the Multiple Measures in Arizona's Teacher Evaluation Model. REL 2015-050

ERIC Educational Resources Information Center

Lazarev, Valeriy; Newman, Denis; Sharp, Alyssa

2014-01-01

This study explored the relationships among the components of the Arizona Department of Education's new teacher evaluation model, with a particular focus on the extent to which ratings from the state model's teacher observation instrument differentiated higher and lower performance. The study used teacher-level evaluation data collected by the…

Teacher Perceptions Regarding Portfolio-Based Components of Teacher Evaluations

ERIC Educational Resources Information Center

Nagel, Charles I.

2012-01-01

This study reports the results of teachers' and principals' perceptions of the package evaluation process, a process that uses a combination of a traditional evaluation with a portfolio-based assessment tool. In addition, this study contributes to the educational knowledge base by exploring the participants' views on the impact of…

A Historical Reflection on Research Evaluation Studies, Their Recurrent Themes and Challenges. Technical Report

ERIC Educational Resources Information Center

Marjanovic, Sonja; Hanney, Stephen; Wooding, Steven

2009-01-01

This report critically examines studies of how scientific research drives innovation which is then translated into socio-economic benefits. It focuses on research evaluation insights that are relevant not only to the academic community, but also to policymakers and evaluation practitioners--and particularly to biomedical and health research…

An Evaluation of the Agriculture Science Project in Mauritius. African Studies in Curriculum Development & Evaluation No. 102.

ERIC Educational Resources Information Center

Jeeroburkhan, M. Fazal

This study evaluated the Agricultural Curriculum Project which is being implemented in 16 secondary schools in Mauritius. Specific areas examined included: (1) the relevance, appropriateness, and practicability of the project's general objectives; (2) the relevance, balance, and organization of the course content; (3) the effectiveness and…

The Effect of Student Evaluations on Faculty Performance

ERIC Educational Resources Information Center

Hartford, Kenneth M.

2017-01-01

The purpose of this study was to understand the behavior of professors as a consequence of receiving student evaluations. To address this purpose, a single research question directed this study: In what ways and to what extent might faculty compromise good pedagogical practices for positive student evaluations of teaching? A qualitative research…

Refinement, Application, and Evaluation of Cognitive and Affective Chemistry Measures for College Students

ERIC Educational Resources Information Center

Heredia, Keily

2013-01-01

This work describes three case studies conducted to address two major problems in the area of chemistry education research, the lack of reported psychometrics regarding instrument scores, and the need for well-characterized assessments to evaluate college chemistry curricula. The first case study describes a psychometric evaluation of the Colorado…

Implementing Curriculum Evaluation: Case Study of a Generic Undergraduate Degree in Health Sciences

ERIC Educational Resources Information Center

Harris, Lynne; Driscoll, Peter; Lewis, Melinda; Matthews, Lynda; Russell, Cherry; Cumming, Steven

2010-01-01

This case study presents a longitudinal, evidence-based approach to health science curriculum reform and evaluation. Curriculum in higher education must meet the needs of diverse stakeholders and must respond to dynamic local, national and international contexts, and this creates challenges for evaluation. The long lead time prior to the…

EVALUATION OF THE REAL-TIME AIR-QUALITY MODEL USING THE RAPS (REGIONAL AIR POLLUTION STUDY) DATA BASE. VOLUME 1. OVERVIEW

EPA Science Inventory

The theory and programming of statistical tests for evaluating the Real-Time Air-Quality Model (RAM) using the Regional Air Pollution Study (RAPS) data base are fully documented in four report volumes. Moreover, the tests are generally applicable to other model evaluation problem...

Tools Used to Evaluate Written Medicine and Health Information: Document and User Perspectives

ERIC Educational Resources Information Center

Luk, Alice; Aslani, Parisa

2011-01-01

This study aims to identify and review tools used to evaluate consumer-oriented written medicine (WMI) and health (WHI) information from a document and user perspective. Articles that met the following inclusion criteria were reviewed: studies evaluating readability, presentation, suitability, quality of WMI/WHI. A total of 152 articles were…

Self and Peer Evaluation of Writing in the Interactive ESL Classroom: An Exploratory Study.

ERIC Educational Resources Information Center

Rothschild, Dennie; Klingenberg, Felicia

1990-01-01

Investigates self- and peer evaluation of writing in the English-as-a-Second-Language classroom. The investigation is examined in two parts: (1) adapting and implementing an appropriate evaluation scale, and (2) studying the various end-of-term effects use of the scale had on students. (17 references) (GLR)

Continuous Evaluation in Ethics Education: A Case Study.

PubMed

McIntosh, Tristan; Higgs, Cory; Mumford, Michael; Connelly, Shane; DuBois, James

2018-04-01

A great need for systematic evaluation of ethics training programs exists. Those tasked with developing an ethics training program may be quick to dismiss the value of training evaluation in continuous process improvement. In the present effort, we use a case study approach to delineate how to leverage formative and summative evaluation measures to create a high-quality ethics education program. With regard to formative evaluation, information bearing on trainee reactions, qualitative data from the comments of trainees, in addition to empirical findings, can ensure that the training program operates smoothly. Regarding summative evaluation, measures examining trainee cognition, behavior, and organization-level results provide information about how much trainees have changed as a result of taking the ethics training. The implications of effective training program evaluation are discussed.

[Development of an evaluation instrument for service quality in nursing homes].

PubMed

Lee, Jia; Ji, Eun Sun

2011-08-01

The purposes of this study were to identify the factors influencing service quality in nursing homes, and to develop an evaluation instrument for service quality. A three-phase process was employed for the study. 1) The important factors to evaluate the service quality in nursing homes were identified through a literature review, panel discussion and focus group interview, 2) the evaluation instrument was developed, and 3) validity and reliability of the study instrument were tested by factor analysis, Pearson correlation coefficient, Cronbach's α and Cohen's Kappa. Factor analysis showed that the factors influencing service quality in nursing homes were healthcare, diet/assistance, therapy, environment and staff. To improve objectivity of the instrument, quantitative as well as qualitative evaluation approaches were adopted. The study instrument was developed with 30 items and showed acceptable construct validity. The criterion-related validity was a Pearson correlation coefficient of .85 in 151 care facilities. The internal consistency was Cronbach's α=.95. The instrument has acceptable validity and a high degree of reliability. Staff in nursing homes can continuously improve and manage their services using the results of the evaluation instrument.

Meta-analytic Evaluation of a Virtual Field Trip to Connect Middle School Students with University Scientists

NASA Astrophysics Data System (ADS)

Adedokun, Omolola A.; Liu, Jia; Parker, Loran Carleton; Burgess, Wilella

2015-02-01

Although virtual field trips are becoming popular, there are few empirical studies of their impacts on student outcomes. This study reports on a meta-analytic evaluation of the impact of a virtual field trip on student perceptions of scientists. Specifically, the study examined the summary effect of zipTrips broadcasts on evaluation participants' perceptions of scientists, as well as the moderating effect of program type on program impact. The results showed statistically significant effect of each broadcast, as well as statistically significant summary (combined) effect of zipTrips on evaluation participants' perceptions of scientists. Results of the moderation analysis showed that the effect was greater for the students that participated in the evaluation of the 8th grade broadcasts, providing additional insight into the role of program variation in predicting differential program impact. This study illustrates how meta-analysis, a methodology that should be of interest to STEM education researchers and evaluation practitioners, can be used to summarize the effects of multiple offerings of the same program. Other implications for STEM educators are discussed.

Green and sustainable remediation (GSR) evaluation: framework, standards, and tool. A case study in Taiwan.

PubMed

Huang, Wen-Yen; Hung, Weiteng; Vu, Chi Thanh; Chen, Wei-Ting; Lai, Jhih-Wei; Lin, Chitsan

2016-11-01

Taiwan has a large number of poorly managed contaminated sites in need of remediation. This study proposes a framework, a set of standards, and a spreadsheet-based evaluation tool for implementing green and sustainable principles into remediation projects and evaluating the projects from this perspective. We performed a case study to understand how the framework would be applied. For the case study, we used a spreadsheet-based evaluation tool (SEFA) and performed field scale cultivation tests on a site contaminated with total petroleum hydrocarbons (TPHs). The site was divided into two lots: one treated by chemical oxidation and the other by bioremediation. We evaluated five core elements of green and sustainable remediation (GSR): energy, air, water resources, materials and wastes, and land and ecosystem. The proposed evaluation tool and field scale cultivation test were found to efficiently assess the effectiveness of the two remediation alternatives. The framework and related tools proposed herein can potentially be used to support decisions about the remediation of contaminated sites taking into account engineering management, cost effectiveness, and social reconciliation.

A Randomized, Controlled Trial of Intranasal Oxytocin as an Adjunct to Behavioral Therapy for Autism Spectrum Disorder

DTIC Science & Technology

2015-10-01

Fellowship of the Alexander von Humboldt Foundation , MIT Name: Dina Hirshfeld- Becker , PhD Project Role: Independent Evaluator Researcher Identifier...e.g. ORCID ID): Nearest person month worked: 3 Contribution to Project: Dr. Hirshfeld- Becker has been an independent evaluator and has evaluated ...either intranasal oxytocin or placebo. Participants and evaluators will be blind to treatment condition. In year 1 of the study, we set up the study

A case study of evaluating informatics impact on diffusion of scientific knowledge.

PubMed

Katz, Susan B

2008-11-06

This case study poster uses a newly developed framework to evaluate an informatics effort in its public health context. The electronic clearance system being evaluated provides the potential for increasing the speed and quality of scientific diffusion of knowledge, and thus translation of research into practice. A graphical logic model and tabular results of the evaluation are presented. Public health history suggests potential benefits of more timely and coordinated diffusion of scientific information.

Development of an external ceramic insulation for the space shuttle orbiter. Part 2: Optimization

NASA Technical Reports Server (NTRS)

Tanzilli, R. A. (Editor)

1973-01-01

The basic insulation improvement study concentrated upon evaluating variables which could result in significant near-term gains in mechanical behavior and insulation effectiveness of the baseline system. The approaches undertaken included: evaluation of small diameter fibers, optimization of binder: slurry characteristics, evaluation of techniques for controlling fiber orientation, optimization of firing cycle, and the evaluation of methods for improving insulation efficiency. A detailed discussion of these basic insulation improvement studies is presented.

Mindfulness Based Stress Reduction for Academic Evaluation Anxiety: A Naturalistic Longitudinal Study.

PubMed

Dundas, Ingrid; Thorsheim, Torbjørn; Hjeltnes, Aslak; Binder, Per Einar

2016-04-02

Mindfulness based stress reduction (MBSR) for academic evaluation anxiety and self-confidence in 70 help-seeking bachelor's and master's students was examined. A repeated measures analysis of covariance on the 46 students who completed pretreatment and posttreatment measures (median age = 24 years, 83% women) showed that evaluation anxiety and self-confidence improved. A growth curve analysis with all 70 original participants showed reductions in both cognitive and emotional components of evaluation anxiety, and that reduction continued postintervention. Although more research is needed, this study indicates that MBSR may reduce evaluation anxiety.

Mindfulness Based Stress Reduction for Academic Evaluation Anxiety: A Naturalistic Longitudinal Study

PubMed Central

Dundas, Ingrid; Thorsheim, Torbjørn; Hjeltnes, Aslak; Binder, Per Einar

2016-01-01

ABSTRACT Mindfulness based stress reduction (MBSR) for academic evaluation anxiety and self-confidence in 70 help-seeking bachelor’s and master’s students was examined. A repeated measures analysis of covariance on the 46 students who completed pretreatment and posttreatment measures (median age = 24 years, 83% women) showed that evaluation anxiety and self-confidence improved. A growth curve analysis with all 70 original participants showed reductions in both cognitive and emotional components of evaluation anxiety, and that reduction continued postintervention. Although more research is needed, this study indicates that MBSR may reduce evaluation anxiety. PMID:27227169

[Existing laparoscopic simulators and their benefit for the surgeon].

PubMed

Kalvach, J; Ryska, O; Ryska, M

2016-01-01

Nowadays, laparoscopic operations are a common part of surgical practice. However, they have their own characteristics and require a specific method of preparation. Recently, simulation techniques have been increasingly used for the training of skills. The aim of this review is to provide a summary of available literature on the topic of laparoscopic simulators, to assess their contribution to the training of surgeons, and to identify the most effective type of simulation. PubMed database, Web of Science and Cochrane Library were used to search for relevant publications. The keywords "laparoscopy, simulator, surgery, assessment" were used in the search. The search was limited to prospective studies published in the last 5 years in the English language. From a total of 354 studies found, we included in the survey 26 that matched our criteria. Nine studies compared individual simulators to one another. Five studies evaluated "high and low fidelity" (a virtual box simulator) as equally effective (EBM 2a). In three cases the "low fidelity" box simulator was found to be more efficient (EBM 2a3b). Only one study preferred the virtual simulator (VR) (EBM2b).Thirteen studies evaluated the benefits of simulators for practice. Twelve found training on a simulator to be an effective method of preparation (EBM 1b3b). In contrast, one study did not find any difference between the training simulator and traditional preparation (EBM 3b). Nine studies evaluated directly one of the methods of evaluating laparoscopic skills. Three studies evaluated VR simulator as a useful assessment tool. Other studies evaluated as successful the scoring system GOALS-GH. The hand motion analysis model was successful in one case. Most studies were observational (EBM 3b) and only 2 studies were of higher quality (EBM 2b). Simulators are an effective tool for practicing laparoscopic techniques (EBM: 1b). It cannot be determined based on available data which of the simulators is most effective. The virtual simulator, however, still remains the most self-sufficient unit suitable for teaching as well as evaluation of laparoscopic techniques (EBM 2b3b). Further studies are needed to find an effective system and parameters for an objective evaluation of skills. laparoscopy - simulator - surgery assessment.

Ground penetrating radar study--phase I : technology review and evaluation.

DOT National Transportation Integrated Search

2006-08-01

In December 2005 Mississippi Department of Transportation (MDOT) initiated State Study No. 182 on review : and evaluation of ground penetrating radar (GPR) technology. This Phase I study has reviewed GPR equipment : and data interpretation methodolog...

Evaluation of the Ramazzini Foundation Study of Methanol in Rats

EPA Pesticide Factsheets

Evaluation of the Ramazzini Foundation Study of Methanol in Rats: A Comparison of Diagnoses by the RF Study Pathologist and a Recent NTP Review Team, summarized by George Cruzan and submitted to the Methanol Institute

A systematic review of health economic evaluations of vaccines in Brazil.

PubMed

Sartori, Ana Marli Christovam; Rozman, Luciana Martins; Decimoni, Tassia Cristina; Leandro, Roseli; Novaes, Hillegonda Maria Dutilh; de Soárez, Patrícia Coelho

2017-06-03

In Brazil, since 2005, the Ministry of Health requires Health Economic Evaluation (HEE) of vaccines for introduction into the National Immunization Program. To describe and analyze the full HEE on vaccines conducted in Brazil from 1980 to 2013. Systematic review of the literature. We searched multiple databases. Two researchers independently selected the studies and extracted the data. The methodological quality of individual studies was evaluated using CHEERS items. Twenty studies were reviewed. The most evaluated vaccines were pneumococcal (25%) and HPV (15%). The most used types of HEE were cost-effectiveness analysis (45%) and cost-utility analysis (20%). The research question and compared strategies were stated in all 20 studies and the target population was clear in 95%. Nevertheless, many studies did not inform the perspective of analysis or data sources. HEE of vaccines in Brazil has increased since 2008. However, the studies still have methodological deficiencies.

A systematic review of health economic evaluations of vaccines in Brazil

PubMed Central

2017-01-01

ABSTRACT Background: In Brazil, since 2005, the Ministry of Health requires Health Economic Evaluation (HEE) of vaccines for introduction into the National Immunization Program. Objectives: To describe and analyze the full HEE on vaccines conducted in Brazil from 1980 to 2013. Methods: Systematic review of the literature. We searched multiple databases. Two researchers independently selected the studies and extracted the data. The methodological quality of individual studies was evaluated using CHEERS items. Results: Twenty studies were reviewed. The most evaluated vaccines were pneumococcal (25%) and HPV (15%). The most used types of HEE were cost-effectiveness analysis (45%) and cost-utility analysis (20%). The research question and compared strategies were stated in all 20 studies and the target population was clear in 95%. Nevertheless, many studies did not inform the perspective of analysis or data sources. Conclusions: HEE of vaccines in Brazil has increased since 2008. However, the studies still have methodological deficiencies. PMID:28129026

Evaluation of Life Sciences and Social Sciences Course Books in Term of Societal Sexuality

ERIC Educational Resources Information Center

Aykac, Necdet

2012-01-01

This study aims to evaluate primary school Life Sciences (1st, 2nd, and 3rd grades) and Social Sciences (4th, 5th, and 6th grades) course books in terms of gender discrimination. This study is a descriptive study aiming to evaluate the primary school Life Sciences (1st, 2nd, 3rd grades) and Social Sciences (4th, 5th, and 6th grades) course books…

The Practice of Evaluation Research and the Use of Evaluation Results.

ERIC Educational Resources Information Center

Van den Berg, G.; Hoeben, W. Th. J. G.

1984-01-01

Lack of use of educational evaluation results in the Netherlands was investigated by analyzing 14 curriculum evaluation studies. Results indicated that rational decision making with a technical (empirical) evaluation approach makes utilization of results most likely. Incremental decision making and a conformative approach make utilization least…

Principal Evaluation in Indiana: Practitioners' Perceptions of a New Statewide Model

ERIC Educational Resources Information Center

Andrews, Kelly A.; Boyland, Lori G.; Quick, Marilynn M.

2016-01-01

This study examines administrators' perspectives of a state-developed principal evaluation model adopted by a majority of Indiana school districts after legislation mandated policy reform in educator evaluation. Feedback was gathered from public school superintendents (the evaluators) and principals (those being evaluated), with 364 participants.…

The Evaluator's Perspective: Evaluating the State Capacity Building Program.

ERIC Educational Resources Information Center

Madey, Doren L.

A historical antagonism between the advocates of quantitative evaluation methods and the proponents of qualitative evaluation methods has stymied the recognition of the value to be gained by utilizing both methodologies in the same study. The integration of quantitative and qualitative methods within a single evaluation has synergistic effects in…

Educational Evaluation: Key Characteristics. ACER Research Series No. 102.

ERIC Educational Resources Information Center

Maling-Keepes, Jillian

A set of 13 key characteristics is presented as a framework for educational evaluation studies: (1) program's stage of development when evaluator is appointed; (2) program's openness to revision; (3) program uniformity from site to site; (4) specificity of program objectives; (5) evaluator's independence; (6) evaluator's orientation to value…

Which Features of Spanish Learners' Pronunciation Most Impact Listener Evaluations?

ERIC Educational Resources Information Center

McBride, Kara

2015-01-01

This study explores which features of Spanish as a foreign language (SFL) pronunciation most impact raters' evaluations. Native Spanish speakers (NSSs) from regions with different pronunciation norms were polled: 147 evaluators from northern Mexico and 99 evaluators from central Argentina. These evaluations were contrasted with ratings from…

Integrating Participatory Elements into an Effectiveness Evaluation

ERIC Educational Resources Information Center

Wallace, Tanner LeBaron

2008-01-01

This article describes an effectiveness evaluation of an intensive case management intervention coordinated by a non-profit organization in a midsize Midwest City. As an effectiveness evaluation, the primary evaluation question was causal in nature; the key task of the evaluative study was to establish and probe connections between the…

Complicity Revisited: Balancing Stakeholder Input and Roles in Evaluation Use

ERIC Educational Resources Information Center

Sturges, Keith M.

2015-01-01

Drawing on a qualitative study of an educational reform and its external evaluation, I describe how a well-intentioned but poorly conceptualized evaluation helped perpetuate asymmetries in the generation and use of evaluation findings. This article explores this project's failure to clarify evaluator roles, identify intended users and expected…

Training Evaluation: An Analysis of the Stakeholders' Evaluation Needs

ERIC Educational Resources Information Center

Guerci, Marco; Vinante, Marco

2011-01-01

Purpose: In recent years, the literature on program evaluation has examined multi-stakeholder evaluation, but training evaluation models and practices have not generally taken this problem into account. The aim of this paper is to fill this gap. Design/methodology/approach: This study identifies intersections between methodologies and approaches…

Annual Report. Technical Reports. Evaluation Productivity Project.

ERIC Educational Resources Information Center

Alkin, Marvin; And Others

After outlining the 1984 activities and results of the Center for the Study of Evaluation's (CSE's) Evaluation Productivity Project, this monograph presents three reports. The first, "The Administrator's Role in Evaluation Use," by James Burry, Marvin C. Alkin, and Joan A. Ruskus, describes the factors influencing an evaluation's use…

Quality of Instruction Improved by Evaluation and Consultation of Instructors

ERIC Educational Resources Information Center

Rindermann, Heiner; Kohler, Jurgen; Meisenberg, Gerhard

2007-01-01

One aim of student evaluation of instruction is the improvement of teaching quality, but there is little evidence that student assessment of instruction alone improves teaching. This study tried to improve the effects of evaluation by combining evaluation with individual counselling in an institutional development approach. Evaluation was…

Redesigning and aligning assessment and evaluation for a federally funded math and science teacher educational program.

PubMed

Hardré, Patricia L; Slater, Janis; Nanny, Mark

2010-11-01

This paper examines the redesign of evaluation components for a teacher professional development project funded by the National Science Foundation. It focuses on aligning evaluation instrumentation and strategies with program goals, research goals and program evaluation best practices. The study identifies weaknesses in the original (year 1) program evaluation design and implementation, develops strategies and tracks changes for year 2 implementation, and then reports enhancement of findings and recommendations for year 3. It includes lessons learned about assessment and evaluation over the project lifespan, with implications for research and evaluation of a range of related programs. This study functions as a classic illustration of how critical it is to observe first principles of assessment and evaluation for funded programs, the risks that arise when they are ignored, and the benefits that accrue when they are systematically observed. Copyright (c) 2009. Published by Elsevier Ltd.

Investment Evaluation of RFID TechnologyApplications: An Evolution Perspective

NASA Astrophysics Data System (ADS)

Dimakopoulou, Andriana; Pramatari, Katerina; Karagiannaki, Angeliki; Papadopoulos, George; Paraskevopoulos, Antonis

Prior empirical research on the evaluation of RFID technologytreats and assesses individual RFID applications independently and in isolation from each other. However, literature on investment evaluation of information technologies has recognised and utilised the significance of evaluating "interdependent" information systems (IS) projects with synergies. Moreover, previous studies when appraising the business value of an RFID investment ignore its opportunity to offer and evolve into additional follow-on investments in the future. Nevertheless, the importance of this notion has been acknowledged by the pertinent literature for the evaluation of other information technologies. This chapter proposes an approach for the investment evaluation of RFID applications considering them rather as a bundle of interdependent and sequential investments than as stand-alone ones. The results from a case study demonstrate how the proposed approach can be employed for the evaluation of RFID projects and offering an additional insight into evaluating investments in RFID applications.

Usability Evaluation Methods for Gesture-Based Games: A Systematic Review.

PubMed

Simor, Fernando Winckler; Brum, Manoela Rogofski; Schmidt, Jaison Dairon Ebertz; Rieder, Rafael; De Marchi, Ana Carolina Bertoletti

2016-10-04

Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user's age and limitations. Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology and a test protocol may offer the user more comfort, welfare, and confidence.

Evaluation of several ultrasonography scoring systems for synovitis and comparison to clinical examination: results from a prospective multicentre study of rheumatoid arthritis.

PubMed

Dougados, Maxime; Jousse-Joulin, Sandrine; Mistretta, Frederic; d'Agostino, Maria-Antonietta; Backhaus, Marina; Bentin, Jacques; Chalès, Gérard; Chary-Valckenaere, Isabelle; Conaghan, Philip; Etchepare, Fabien; Gaudin, Philippe; Grassi, Walter; van der Heijde, Désirée; Sellam, Jérémie; Naredo, Esperanza; Szkudlarek, Marcin; Wakefield, Richard; Saraux, Alain

2010-05-01

To evaluate different global ultrasonographic (US) synovitis scoring systems as potential outcome measures of rheumatoid arthritis (RA) according to the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) filter. To study selected global scoring systems, for the clinical, B mode and power Doppler techniques, the following joints were evaluated: 28 joints (28-joint Disease Activity Score (DAS28)), 20 joints (metacarpophalangeals (MCPs) + metatarsophalangeals (MTPs)) and 38 joints (28 joints + MTPs) using either a binary (yes/no) or a 0-3 grade. The study was a prospective, 4-month duration follow-up of 76 patients with RA requiring anti-tumour necrosis factor (TNF) therapy (complete follow-up data: 66 patients). Intraobserver reliability was evaluated using the intraclass correlation coefficient (ICC), construct validity was evaluated using the Cronbach alpha test and external validity was evaluated using level of correlation between scoring system and C reactive protein (CRP). Sensitivity to change was evaluated using the standardised response mean. Discriminating capacity was evaluated using the standardised mean differences in patients considered by the doctor as significantly improved or not at the end of the study. Different clinimetric properties of various US scoring systems were at least as good as the clinical scores with, for example, intraobserver reliability ranging from 0.61 to 0.97 versus from 0.53 to 0.82, construct validity ranging from 0.76 to 0.89 versus from 0.76 to 0.88, correlation with CRP ranging from 0.28 to 0.34 versus from 0.28 to 0.35 and sensitivity to change ranging from 0.60 to 1.21 versus from 0.96 to 1.36 for US versus clinical scoring systems, respectively. This study suggests that US evaluation of synovitis is an outcome measure at least as relevant as physical examination. Further studies are required in order to achieve optimal US scoring systems for monitoring patients with RA in clinical trials and in clinical practice.

Building a patient-centered and interprofessional training program with patients, students and care professionals: study protocol of a participatory design and evaluation study.

PubMed

Vijn, Thomas W; Wollersheim, Hub; Faber, Marjan J; Fluit, Cornelia R M G; Kremer, Jan A M

2018-05-30

A common approach to enhance patient-centered care is training care professionals. Additional training of patients has been shown to significantly improve patient-centeredness of care. In this participatory design and evaluation study, patient education and medical education will be combined by co-creating a patient-centered and interprofessional training program, wherein patients, students and care professionals learn together to improve patient-centeredness of care. In the design phase, scientific literature regarding interventions and effects of student-run patient education will be synthesized in a scoping review. In addition, focus group studies will be performed on the preferences of patients, students, care professionals and education professionals regarding the structure and content of the training program. Subsequently, an intervention plan of the training program will be constructed by combining these building blocks. In the evaluation phase, patients with a chronic disease, that is rheumatoid arthritis, diabetes and hypertension, and patients with an oncologic condition, that is colonic cancer and breast cancer, will learn together with medical students, nursing students and care professionals in training program cycles of three months. Process and effect evaluation will be performed using the plan-do-study-act (PDSA) method to evaluate and optimize the training program in care practice and medical education. A modified control design will be used in PDSA-cycles to ensure that students who act as control will also benefit from participating in the program. Our participatory design and evaluation study provides an innovative approach in designing and evaluating an intervention by involving participants in all stages of the design and evaluation process. The approach is expected to enhance the effectiveness of the training program by assessing and meeting participants' needs and preferences. Moreover, by using fast PDSA cycles and a modified control design in evaluating the training program, the training program is expected to be efficiently and rapidly implemented into and adjusted to care practice and medical education.

Usability Evaluation Methods for Gesture-Based Games: A Systematic Review

PubMed Central

Simor, Fernando Winckler; Brum, Manoela Rogofski; Schmidt, Jaison Dairon Ebertz; De Marchi, Ana Carolina Bertoletti

2016-01-01

Background Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. Objective This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. Methods The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. Results In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user’s age and limitations. Conclusions Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology and a test protocol may offer the user more comfort, welfare, and confidence. PMID:27702737

ACCP: economic evaluations of clinical pharmacy services: 2001-2005.

PubMed

Perez, Alexandra; Doloresco, Fred; Hoffman, James M; Meek, Patrick D; Touchette, Daniel R; Vermeulen, Lee C; Schumock, Glen T

2009-01-01

The objectives of this review were to summarize and evaluate studies that measured the economic impact of clinical pharmacy services published between 2001 and 2005 (inclusive) and to provide guidance on methodologic considerations to individuals performing such research in the future. A systematic literature search using the MEDLINE and International Pharmaceutical Abstracts databases was conducted to identify published economic evaluations of clinical pharmacy services. Studies were screened and then randomly assigned to reviewers, who reassessed inclusion and exclusion criteria and abstracted prespecified data from each study. Among the many characteristics examined in each study were study design and type of economic evaluation, setting and type of clinical pharmacy service, study quality, and results. Ninety-three articles were included in the final analysis. These studies were published in 43 different journals, most of which (68 [73.1%]) were pharmacy-based. Most studies were performed in hospitals (40 [43.0%]), ambulatory care clinics or physician's offices (20 [21.5%]), or community pharmacies (16 [17.2%]). The most common types of clinical pharmacy services evaluated were general pharmacotherapeutic monitoring services (32 [34.4%]), target drug programs (27 [29%]), and disease state-management services (21 [22.6%]). Full economic evaluations were performed in just less than half (45 [48.4%]) of the studies, and a positive economic benefit associated with clinical pharmacy services was noted in 31 (69%) of the 45 studies. Among 15 studies reporting data necessary to determine a benefit:cost ratio, the pooled median value was 4.81:1-meaning that for every $1 invested in clinical pharmacy services, $4.81 was achieved in reduced costs or other economic benefits. The quality of studies varied widely, with less than one half considered to be good to fair (40 [43.0%]); however, the proportion of studies using appropriate study designs increased compared with previous reviews. Based on the evidence examined in this review, clinical pharmacy services continue to provide a significant return on investment, but improvements are still needed in the methods used to evaluate the economic impact of these services.

Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance.

PubMed

Hallingberg, Britt; Turley, Ruth; Segrott, Jeremy; Wight, Daniel; Craig, Peter; Moore, Laurence; Murphy, Simon; Robling, Michael; Simpson, Sharon Anne; Moore, Graham

2018-01-01

Evaluations of complex interventions in public health are frequently undermined by problems that can be identified before the effectiveness study stage. Exploratory studies, often termed pilot and feasibility studies, are a key step in assessing the feasibility and value of progressing to an effectiveness study. Such studies can provide vital information to support more robust evaluations, thereby reducing costs and minimising potential harms of the intervention. This systematic review forms the first phase of a wider project to address the need for stand-alone guidance for public health researchers on designing and conducting exploratory studies. The review objectives were to identify and examine existing recommendations concerning when such studies should be undertaken, questions they should answer, suitable methods, criteria for deciding whether to progress to an effectiveness study and appropriate reporting. We searched for published and unpublished guidance reported between January 2000 and November 2016 via bibliographic databases, websites, citation tracking and expert recommendations. Included papers were thematically synthesized. The search retrieved 4095 unique records. Thirty papers were included, representing 25 unique sources of guidance/recommendations. Eight themes were identified: pre-requisites for conducting an exploratory study, nomenclature, guidance for intervention assessment, guidance surrounding any future evaluation study design, flexible versus fixed design, progression criteria to a future evaluation study, stakeholder involvement and reporting of exploratory studies. Exploratory studies were described as being concerned with the intervention content, the future evaluation design or both. However, the nomenclature and endorsed methods underpinning these aims were inconsistent across papers. There was little guidance on what should precede or follow an exploratory study and decision-making surrounding this. Existing recommendations are inconsistent concerning the aims, designs and conduct of exploratory studies, and guidance is lacking on the evidence needed to inform when to proceed to an effectiveness study. PROSPERO 2016, CRD42016047843.

Evaluation of clinical practice guidelines.

PubMed Central

Basinski, A S

1995-01-01

Compared with the current focus on the development of clinical practice guidelines the effort devoted to their evaluation is meagre. Yet the ultimate success of guidelines depends on routine evaluation. Three types of evaluation are identified: evaluation of guidelines under development and before dissemination and implementation, evaluation of health care programs in which guidelines play a central role, and scientific evaluation, through studies that provide the scientific knowledge base for further evolution of guidelines. Identification of evaluation and program goals, evaluation design and a framework for evaluation planning are discussed. PMID:7489550

Undergraduate medical education programme renewal: a longitudinal context, input, process and product evaluation study.

PubMed

Mirzazadeh, Azim; Gandomkar, Roghayeh; Hejri, Sara Mortaz; Hassanzadeh, Gholamreza; Koochak, Hamid Emadi; Golestani, Abolfazl; Jafarian, Ali; Jalili, Mohammad; Nayeri, Fatemeh; Saleh, Narges; Shahi, Farhad; Razavi, Seyed Hasan Emami

2016-02-01

The purpose of this study was to utilize the Context, Input, Process and Product (CIPP) evaluation model as a comprehensive framework to guide initiating, planning, implementing and evaluating a revised undergraduate medical education programme. The eight-year longitudinal evaluation study consisted of four phases compatible with the four components of the CIPP model. In the first phase, we explored the strengths and weaknesses of the traditional programme as well as contextual needs, assets, and resources. For the second phase, we proposed a model for the programme considering contextual features. During the process phase, we provided formative information for revisions and adjustments. Finally, in the fourth phase, we evaluated the outcomes of the new undergraduate medical education programme in the basic sciences phase. Information was collected from different sources such as medical students, faculty members, administrators, and graduates, using various qualitative and quantitative methods including focus groups, questionnaires, and performance measures. The CIPP model has the potential to guide policy makers to systematically collect evaluation data and to manage stakeholders' reactions at each stage of the reform in order to make informed decisions. However, the model may result in evaluation burden and fail to address some unplanned evaluation questions.

Evaluations of health promoting schools: a review of nine studies.

PubMed

Mũkoma, Wanjirũ; Flisher, Alan J

2004-09-01

The concept of 'health promoting schools' has been embraced internationally as an effective way of promoting the health of children, adolescents, and the wider school community. It is only recently that attempts have been made to evaluate health promoting schools. This paper reviews evaluations of health promoting schools and draws useful evaluation methodology lessons. The review is confined to school-based interventions that are founded explicitly on the concept of the health promoting school and employ the concept beyond one school domain. We included nine evaluations in this review. Seven of these were published in the peer reviewed scientific literature. Two were unpublished reports. One study was a randomized controlled trial, while a quasi-experimental research design with comparison schools was used in three studies. With three exceptions, combinations of quantitative and qualitative data were collected. There was evidence that the health promoting school has some influence on various domains of health for the school community. It is also possible to integrate health promotion into the school curriculum and policies successfully. However, the evaluation of health promoting schools is complex. We discuss some of the methodological challenges of evaluating health promoting schools and make suggestions for improving future evaluations.

Cost-effectiveness of Occupational Therapy in Older People: Systematic Review of Randomized Controlled Trials.

PubMed

Nagayama, Hirofumi; Tomori, Kounosuke; Ohno, Kanta; Takahashi, Kayoko; Yamauchi, Keita

2016-06-01

A systematic review of the cost-effectiveness of occupational therapy for older people was conducted. MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library, OT seeker and unpublished trials registers were searched. Reference lists of all potentially eligible studies were searched with no language restrictions. We included trial-based full economic evaluations that considered both costs and outcomes in occupational therapy for older people compared with standard care (i.e. other therapy) or no intervention. We reviewed each trial for methodological quality using the Cochrane risk of bias tool and assessed the quality of economic evaluations using a Drummond checklist. In the results of this review, we included five eligible studies (1-5) that were randomized controlled trials with high-quality economic evaluation. Two studies were full economic evaluations of interventions for fall prevention (1 and 2); two studies were full economic evaluations of preventive occupational therapy interventions (3 and 4; one was a comparison of an occupational therapy group with a social work group); one study was a full economic evaluation of occupational therapy for individuals with dementia (5). Two of the studies (one was preventive occupational therapy [3] and the other was occupational therapy for dementia [5]) found a significant effect and confirmed the cost-effectiveness of occupational therapy for older people compared with the control group. These studies found that occupational therapy for older people was clinically effective and cost-effective in comparison with standard care or other therapies. With reference to their clinical implication, these intervention studies (using a client-centred approach) suggested potentially cost-effective means to motivate clients to maintain their own health. However, this review has limitations because of the high heterogeneity of the reviewed studies on full economic evaluations of occupational therapy for older people. Future studies on the cost-effectiveness of occupational therapy in older people are strongly warranted. Copyright © 2015 John Wiley & Sons, Ltd. Copyright © 2015 John Wiley & Sons, Ltd.

Sustaining and Improving Study Abroad Experiences Through Comparative Evaluation.

PubMed

Johanson, Linda S

Researchers have related participation in study abroad experiences to many positive outcomes for nursing students; however, educators are faced with the task of not only developing meaningful study abroad opportunities but sustaining and improving them as well. Educators can evaluate repeat study abroad programs by comparing experiences, looking for trends, and conjecturing rationales. To illustrate this process, an example of a study abroad opportunity that has been repeated over 11 years is presented. The first six years have been compared to the most recent five years, revealing three categories of change for evaluation and the resulting course improvements.

When less is more: the consequences of affective primacy for subliminal priming effects.

PubMed

Stapel, Diederik A; Koomen, Willem

2005-09-01

This research investigates the consequences of the notion that one can distinguish early-evaluative (when exposure is short) and late-descriptive reactions (when exposure is long) to subliminally primed trait concepts. In three studies, it was found that the evaluative effects instigated by short exposure to primed concepts were bigger than the evaluative + descriptive effects instigated by long exposure: Less is more. Only when exposure was short, target interpretations were accompanied by evaluative inferences (Studies 1 and 3). Similarly, only when exposure was short, descriptively inapplicable trait primes affected the interpretation of an ambiguous target (Studies 2 and 3).

Interventions to reduce injuries among older workers in agriculture: A review of evaluated intervention projects.

PubMed

Nilsson, Kerstin

2016-10-17

The number of older workers is increasing throughout the industrialised world and older workers are known to be more frequent in the injury-prone agricultural sector. This paper sought to extend knowledge by reviewing evaluated intervention studies intended to decrease risks and work injuries among older workers in agriculture. A systematic literature review regarding: evaluated intervention projects on injury prevention, including participants aged 55 years and older, and working in agriculture. This review identified evaluated intervention projects regarding: i) intervention in injury prevention; ii) interventions to increase knowledge in health and safety tasks and practice; and iii) interventions to increase the use of safety equipment in work. The evaluations reviewed showed that the interventions were less successful in involving older agricultural workers than their younger counterparts. The evaluations also showed that the outcome of interventions was generally less positive or brought about no significant difference in risk awareness and behaviour change among older agricultural workers. Many articles and statistics describe injuries in agriculture. Especially older farm workers are one of the groups with most work injuries and deaths. Despite this, an important finding in this review was shortage of implemented and evaluated intervention studies orientated toward reduce injuries among older workers in agriculture. This review also found that no intervention project in the evaluations studied had a clear positive effect. Many intervention studies have problems with or lack of evaluation in the study design. Based on the results in this review, important future research tasks are to improve the design of interventions, devise implementation methods and formulate appropriate evaluation methods to measure the outcome of the interventions. Intervention programmes also need to involve older workers specific physical and cognitive age aspects in the design to increases their willingness to participate and to be successful to reduce injuries.

Organizational determinants of evaluation practice in Australian prevention agencies.

PubMed

Schwarzman, J; Bauman, A; Gabbe, B; Rissel, C; Shilton, T; Smith, B J

2018-06-01

Program evaluation is essential to inform decision making, contribute to the evidence base for strategies, and facilitate learning in health promotion and disease prevention organizations. Theoretical frameworks of organizational learning, and studies of evaluation capacity building describe the organization as central to evaluation capacity. Australian prevention organizations recognize limitations to current evaluation effectiveness and are seeking guidance to build evaluation capacity. This qualitative study identifies organizational facilitators and barriers to evaluation practice, and explores their interactions in Australian prevention organizations. We conducted semi-structured interviews with 40 experienced practitioners from government and non-government organizations. Using thematic analysis, we identified seven key themes that influence evaluation practice: leadership, organizational culture, organizational systems and structures, partnerships, resources, workforce development and training and recruitment and skills mix. We found organizational determinants of evaluation to have multi-level interactions. Leadership and organizational culture influenced organizational systems, resource allocation and support of staff. Partnerships were important to overcome resource deficits, and systems were critical to embed evaluation within the organization. Organizational factors also influenced the opportunities for staff to develop skills and confidence. We argue that investment to improve these factors would allow organizations to address evaluation capacity at multiple levels, and ultimately facilitate effective evaluation practice.

Evaluation utilization research--developing a theory and putting it to use.

PubMed

Neuman, Ari; Shahor, Neria; Shina, Ilan; Sarid, Anat; Saar, Zehava

2013-02-01

This article presents the findings of a two-stage study that had two key objectives: to develop a theory about evaluation utilization in an educational organization and to apply this theory to promote evaluation utilization within the organization. The first stage involved a theoretical conceptualization using a participatory method of concept mapping. This process identified the modes of evaluation utilization within the organization, produced a representation of the relationship between them and led to a theory. The second stage examined the practical implications of this conceptualization in terms of how different stakeholders in the organization perceive the actual and preferable state of evaluation utilization within the organization (i.e. to what extent is evaluation utilized and to what extent should it be utilized). The participatory process of the study promoted the evaluation utilization by involving stakeholders, thus giving them a sense of ownership and improving communication between the evaluation unit and the stakeholders. In addition, understanding the evaluation needs of the stakeholders in the organization helped generate relevant and realizable evaluation processes. On a practical level, the results are currently shaping the evaluation plan and the place of evaluations within the organization. Copyright © 2012 Elsevier Ltd. All rights reserved.

A synthesis study of noncontact nondestructive evaluation of top-down cracking in asphalt pavements.

DOT National Transportation Integrated Search

2013-12-01

The primary objective of this research study was to conduct an extensive literature review on top-down : cracking evaluation studies, prepare a synthesis of findings, and recommend a follow up phase for a : pilot study in Mississippi with a candidate...

75 FR 32474 - National Institute of Child Health and Human Development; Revision to Proposed Collection...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-06-08

... purpose of the proposed methodological study is to evaluate the feasibility, acceptability, and cost of... inform the design of the Main Study of the National Children's Study. This data collection will evaluate...

Experimental study and evaluation of radioprotective drugs

NASA Technical Reports Server (NTRS)

Smith, D. E.; Thomson, J. F.

1968-01-01

Experimental study evaluates radioprotective drugs administered before exposure either orally or intravenously. Specifically studied are the sources of radiation, choice of radiation dose, choice of animals, administration of drugs, the toxicity of protective agents and types of protective drug.

Home Start Evaluation Study. Interim Case Studies IIa.

ERIC Educational Resources Information Center

Fein, Robert

This formative evaluation study of Home Start uses a case study approach. A brief case study focuses on the administrative structure and staff resources and responsibilities of National Home Start. Also included are reports on seven local programs developed after two field visits had been made to each program. In the first visit, objectives chosen…

Discovering the Future of the Case Study Method in Evaluation Research.

ERIC Educational Resources Information Center

Yin, Robert K.

1994-01-01

It is assumed that evaluators of the future will still be interested in case study methodology. Scenarios that ignore a case study method, that look back to a distinctive case study method, and that see the case study method as an integrating force in the qualitative-quantitative debate are explored. (SLD)

Self-esteem and evaluative beliefs in paranoia.

PubMed

Valiente, Carmen; Cantero, Dolores; Sánchez, Alvaro; Provencio, María; Wickham, Sophie

2014-06-01

Psychological models have implicated negative self-esteem as an important factor underlying paranoia. However, research investigating the role of self-esteem in paranoia suffers from poor conceptual and methodological understanding, resulting in conflicting findings. Central to this problem is the use of measures investigating global self-esteem and self-evaluative beliefs interchangeably. In the present study we aimed to analyze differences in self-esteem domains and self-evaluation. The present study used interviews and questionnaires to compare a clinical sample of participants who were currently paranoid (n = 55) with healthy controls (n = 57) on global self-esteem domains and negative evaluative beliefs, in order to investigate the multi-faceted role of "the self". There was no significant difference in self-esteem domains between groups, highlighting that self-esteem is preserved in currently paranoid individuals. However, the paranoid group had significantly more negative evaluative beliefs. Interestingly, our global measures of self-esteem and measures of negative evaluative beliefs were uncorrelated, highlighting the importance of understanding the differences underlying these concepts. This study does not address dynamic aspects of self-esteem and self-evaluation. The present study provides undeniable evidence to investigate self-concept dimensions separately. These findings must be considered by researchers interested in the role of the self in the onset and maintenance of paranoia. Copyright © 2014 Elsevier Ltd. All rights reserved.

Do health economic evaluations using observational data provide reliable assessment of treatment effects?

PubMed Central

2013-01-01

Economic evaluation in modern health care systems is seen as a transparent scientific framework that can be used to advance progress towards improvements in population health at the best possible value. Despite the perceived superiority that trial-based studies have in terms of internal validity, economic evaluations often employ observational data. In this review, the interface between econometrics and economic evaluation is explored, with emphasis placed on highlighting methodological issues relating to the evaluation of cost-effectiveness within a bivariate framework. Studies that satisfied the eligibility criteria exemplified the use of matching, regression analysis, propensity scores, instrumental variables, as well as difference-in-differences approaches. All studies were reviewed and critically appraised using a structured template. The findings suggest that although state-of-the-art econometric methods have the potential to provide evidence on the causal effects of clinical and policy interventions, their application in economic evaluation is subject to a number of limitations. These range from no credible assessment of key assumptions and scarce evidence regarding the relative performance of different methods, to lack of reporting of important study elements, such as a summary outcome measure and its associated sampling uncertainty. Further research is required to better understand the ways in which observational data should be analysed in the context of the economic evaluation framework. PMID:24229445

Unleashing creativity: The role of left temporoparietal regions in evaluating and inhibiting the generation of creative ideas.

PubMed

Mayseless, Naama; Aharon-Peretz, Judith; Shamay-Tsoory, Simone

2014-11-01

Human creativity is thought to entail two processes. One is idea generation, whereby ideas emerge in an associative manner, and the other is idea evaluation, whereby generated ideas are evaluated and screened. Thus far, neuroimaging studies have identified several brain regions as being involved in creativity, yet only a handful of studies have examined the neural basis underlying these two processes. We found that an individual with left temporoparietal hemorrhage who had no previous experience as an artist developed remarkable artistic creativity, which diminished as the hemorrhage receded. We thus hypothesized that damage to the evaluation network of creativity during the initial hematoma had a releasing effect on creativity by "freeing" the idea generation system. In line with this hypothesis, we conducted a subsequent fMRI study showing that decreased left temporal and parietal activations among healthy individuals as they evaluated creative ideas selectively predicted higher creativity. The current studies provide converging multi-method evidence suggesting that the left temporoparietal area is part of a neural network involved in evaluating creativity, and that as such may act as inhibitors of creativity. We propose an explanatory model of creativity centered upon the key role of the left temporoparietal regions in evaluating and inhibiting creativity. Copyright © 2014 Elsevier Ltd. All rights reserved.

Economic Evaluation alongside Multinational Studies: A Systematic Review of Empirical Studies

PubMed Central

Oppong, Raymond; Jowett, Sue; Roberts, Tracy E.

2015-01-01

Purpose of the study This study seeks to explore methods for conducting economic evaluations alongside multinational trials by conducting a systematic review of the methods used in practice and the challenges that are typically faced by the researchers who conducted the economic evaluations. Methods A review was conducted for the period 2002 to 2012, with potentially relevant articles identified by searching the Medline, Embase and NHS EED databases. Studies were included if they were full economic evaluations conducted alongside a multinational trial. Results A total of 44 studies out of a possible 2667 met the inclusion criteria. Methods used for the analyses varied between studies, indicating a lack of consensus on how economic evaluation alongside multinational studies should be carried out. The most common challenge appeared to be related to addressing differences between countries, which potentially hinders the generalisability and transferability of results. Other challenges reported included inadequate sample sizes and choosing cost-effectiveness thresholds. Conclusions It is recommended that additional guidelines be developed to aid researchers in this area and that these be based on an understanding of the challenges associated with multinational trials and the strengths and limitations of alternative approaches. Guidelines should focus on ensuring that results will aid decision makers in their individual countries. PMID:26121465

Stimulant Use in the Management of Mild Traumatic Brain Injury: A Qualitative Literature Review.

PubMed

Iaccarino, Mary Alexis; Philpotts, Lisa Liang; Zafonte, Ross; Biederman, Joseph

2018-03-01

Mild traumatic brain injury (mTBI) often presents with cognitive complaints including difficulty with attention and concentration. As these symptoms resemble those of ADHD, stimulants may be a potential treatment for mTBI. This review evaluates the literature on the use of stimulants for the treatment of mTBI. A systematic evaluation of the literature using six databases: Ovidmedline, Pubmed, psychINFO, CINAH, Embase, and Cochrane. Broad search terms were used and studies were included that evaluate the use of stimulant and stimulant-like medications in the mTBI population. Data extracted included stimulant type and dosing, symptoms targeted, outcomes, safety and tolerability, and if the study population had ADHD. Nine studies were identified that met the inclusion criteria. Immediate release methylphenidate and amantadine were used for treatment. Methylphenidate had some impact on attention, fatigue, and depression. However, due to the limited number of studies and heterogeneity of study populations, symptoms targeted, and outcome measures used, meaningful conclusions regarding the effect of stimulants in mTBI could not be made. No study evaluated for the presence of ADHD within the study population, despite stimulants being the mainstay treatment for ADHD. PProspective studies on the use of stimulants in mTBI, that evaluate participants for a diagnosis of ADHD, are needed.

Strategies facilitating practice change in pediatric cancer: a systematic review.

PubMed

Robinson, Paula D; Dupuis, Lee L; Tomlinson, George; Phillips, Bob; Greenberg, Mark; Sung, Lillian

2016-09-01

By conducting a systematic review, we describe strategies to actively disseminate knowledge or facilitate practice change among healthcare providers caring for children with cancer and we evaluate the effectiveness of these strategies. We searched Ovid Medline, EMBASE and PsychINFO. Fully published primary studies were included if they evaluated one or more professional intervention strategies to actively disseminate knowledge or facilitate practice change in pediatric cancer or hematopoietic stem cell transplantation. Data extracted included study characteristics and strategies evaluated. In studies with a quantitative analysis of patient outcomes, the relationship between study-level characteristics and statistically significant primary analyses was evaluated. Of 20 644 titles and abstracts screened, 146 studies were retrieved in full and 60 were included. In 20 studies, quantitative evaluation of patient outcomes was examined and a primary outcome was stated. Eighteen studies were 'before and after' design; there were no randomized studies. All studies were at risk for bias. Interrupted time series was never the primary analytic approach. No specific strategy type was successful at improving patient outcomes. Literature describing strategies to facilitate practice change in pediatric cancer is emerging. However, major methodological limitations exist. Studies with robust designs are required to identify effective strategies to effect practice change. © The Author 2016. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Perceptions of the Principal Evaluation Process and Performance Criteria: A Qualitative Study of the Challenge of Principal Evaluation

ERIC Educational Resources Information Center

Faginski-Stark, Erica; Casavant, Christopher; Collins, William; McCandless, Jason; Tencza, Marilyn

2012-01-01

Recent federal and state mandates have tasked school systems to move beyond principal evaluation as a bureaucratic function and to re-imagine it as a critical component to improve principal performance and compel school renewal. This qualitative study investigated the district leaders' and principals' perceptions of the performance evaluation…

Publicly Available Information Regarding the Evaluation of School District Superintendents in New York State

ERIC Educational Resources Information Center

Powers, Jarett

2017-01-01

The purpose of this qualitative case study from archival material was to evaluate the degree of public transparency in the evaluation methodology and procedures of school superintendents of districts with student populations between 700 and 900 students in the State of New York. This study examined the language of the 71 superintendent employment…

The Influence of Social Style in Evaluating Academic Presentations of Engineering Projects

ERIC Educational Resources Information Center

Ortiz València, Héctor; García Carrillo, Águeda; González Benítez, Margarita

2012-01-01

An individual's social style is determined by behavioral patterns in the interactions with their peers. Some studies suggest that social style may influence the way in which an individual's performance is evaluated. We studied the effects that speakers' and evaluators' social styles have on the marks given for end-of-term presentations in a…

Intensive Evaluation of Head Start Implementation in the Tucson Early Education Model.

ERIC Educational Resources Information Center

Rentfrow, Robert K.

As part of the national Head Start Planned Variation Study, this study used a relatively small sample in an intensive evaluation of program implementation in one field community using the Tucson Early Education Model (TEEM). A modified Solomon four-group research design formed the organization framework. Evaluation of six TEEM classrooms and two…

A Study of How Secondary School Principals in Minnesota Perceive the Evaluation of Their Performance

ERIC Educational Resources Information Center

Muenich, John Andrew

2014-01-01

The purpose for this study was to ascertain the perceptions principals of public secondary schools in Minnesota have in relation to the evaluation of their job performance. Responding principals reported that past evaluations have been fair and consistent but have questioned their value with regard to professional growth. When asked if student…

Developing and Using a Logic Model for Evaluation and Assessment of University Student Affairs Programming: A Case Study

ERIC Educational Resources Information Center

Cooper, Jeff

2009-01-01

This dissertation addresses theory and practice of evaluation and assessment in university student affairs, by applying logic modeling/program theory to a case study. I intend to add knowledge to ongoing dialogue among evaluation scholars and practitioners on student affairs program planning and improvement as integral considerations that serve…

Exploring the Capability of Evaluating Technical Solutions: A Collaborative Study into the Primary Technology Classroom

ERIC Educational Resources Information Center

Björkholm, Eva

2014-01-01

Within the field of technology education, evaluating technical solutions is considered as an important topic. Research indicates that pupils have difficulties in evaluating technical solutions in terms of fitness for purpose, i.e. how effective a technical solution supports its intended function. By using the learning study, which is an iterative…

A Brief History of the Evaluation of VET in Australia. Review of Research.

ERIC Educational Resources Information Center

McDonald, Rod; Hayton, Geoff

The history and current state of evaluation of vocational education and training (VET) in Australia was examined. The study's main conclusion was that, despite its important potential, evaluation has thus far been little used for the improvement of VET in Australia. Other conclusions of the study were as follows: (1) little development of…

Third National Even Start Evaluation: Follow-Up Findings from the Experimental Design Study. NCEE 2005-3002

ERIC Educational Resources Information Center

Ricciuti, Anne E., St.Pierre, Robert G.; Lee, Wang; Parsad, Amanda

2004-01-01

Citation: Ricciuti, A.E., R.G. St.Pierre, W. Lee, A. Parsad & T. Rimdzius. Third National Even Start Evaluation: Follow-Up Findings From the Experimental Design Study. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Washington, D.C., 2004. Background: The Even…

Evaluation Bias against Work in Progress: Evidence from a Spanish Marketing Conference

ERIC Educational Resources Information Center

Calderon-Martinez, Aurora; Ruiz-Conde, Enar; Parreno-Selva, Josefa; Mas-Ruiz, Francisco Jose

2010-01-01

Traditionally, the literature has analyzed the existence of bias in the evaluation of papers submitted to scientific journals, but it has not considered any possible bias in the evaluation of studies submitted to conferences. In this sense we can ask whether the conferences provide an adequate service for researchers. This study examines whether…

The Effects of Head Start Health Services: Report of the Head Start Health Evaluation. Volume I and II.

ERIC Educational Resources Information Center

Fosburg, Linda B.; And Others

In 1977, a longitudinal study was initiated to assess the effectiveness of health services provided by Head Start. The study provided for 10 domains: pediatric health examinations, health history recordings, dental evaluation, anthropometric assessment, diet and nutrition assessment, and hematology evaluations, as well as for developmental,…

Construction and Validation of a Holistic Education School Evaluation Tool Using Montessori Erdkinder Principles

ERIC Educational Resources Information Center

Setari, Anthony Philip

2016-01-01

The purpose of this study was to construct a holistic education school evaluation tool using Montessori Erdkinder principles, and begin the validation process of examining the proposed tool. This study addresses a vital need in the holistic education community for a school evaluation tool. The tool construction process included using Erdkinder…

A Qualitative Approach to the Evaluation of Expert Systems Shells.

ERIC Educational Resources Information Center

Slawson, Dean A.; And Others

This study explores an approach to the evaluation of expert system shells using case studies. The methodology and some of the results of an evaluation of the prototype development of an expert system using the shell "M1" are detailed, including a description of the participants and the project, the data collection process and materials,…

Teacher Evaluation: A Qualitative Study on Growth-Producing Practices in K-8 Schools

ERIC Educational Resources Information Center

Manso, Phoebe Hechanova

2017-01-01

Literature on teacher evaluation describes a flawed system that evokes wide-ranging attempts to "get it right." This qualitative study is another effort that seeks to understand its legal and traditional constructs. The goal of this research is to identify growth-producing practices that will transform teacher evaluation into an organic…

A Study of Developing an Attitude Scale towards Authentic Learning Environments and Evaluation

ERIC Educational Resources Information Center

Çetinkaya, Murat

2018-01-01

The aim of the research is to improve a valid and reliable attributing scale which identifies authentic learning environments and evaluation attributes of the science teacher candidates. The study has been designed on the base of validity and reliability of the scale developed to evaluate the authentic learning environments. The research group is…

A Longitudinal Study of the Impact of Parental Divorce on Adolescents' Evaluations of Self and Parents.

ERIC Educational Resources Information Center

Parish, Thomas S.; Wigle, Stanley E.

1985-01-01

Obtained children's (N=639) evaluations of their parents and themselves over a three-year period. Compared children whose parents divorced prior to or during the study with the children whose families remained intact during that time. Results showed children in intact families evaluated themselves and their parents more positively. (BH)

Student Use of Mobile Devices in Course Evaluation: A Longitudinal Study

ERIC Educational Resources Information Center

Champagne, Matthew V.

2013-01-01

A 2012 survey of higher education found that 27% of colleges and universities were "mobile ready", that is, allowing students to complete course evaluations via mobile devices, and 26% of schools planned to allow the use of mobile devices for course evaluations within the next year. The purpose of this study was to prepare for this…

77 FR 5850 - Notice of Random Assignment Study To Evaluate Workforce Investment Act Adult and Dislocated...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-02-06

...: One commenter was concerned about 50 percent of the participants being placed in a control group. The... impact methodology for the evaluation. The design of the study was described as follows: The evaluation... eligible for intensive services would be randomly assigned to one of three groups. The three research...

Evaluation of flue-gas desulfurization gypsum in poultry litter as a substrate component for greenhouse horticultural crops

USDA-ARS?s Scientific Manuscript database

A study was conducted to evaluate the growth response and consumer preference of three plant species to substrate blends containing flue gas desulfurization gypsum (FGDG). Substrate blends used in this study were derived from a previous experiment that evaluated the use of FGD Gas a bedding material...

Consequences of ignoring geologic variation in evaluating grazing impacts

Treesearch

Jonathan W. Long; Alvin L. Medina

2006-01-01

The geologic diversity of landforms in the Southwest complicates efforts to evaluate impacts of land uses such as livestock grazing. We examined a research study that evaluated relationships between trout biomass and stream habitat in the White Mountains of east-central Arizona. That study interpreted results of stepwise regressions and a nonparametric test of “grazed...

Evaluation of Mathematics Curriculum in Primary Teacher Training Institute in Somalia. African Studies in Curriculum Development & Evaluation.

ERIC Educational Resources Information Center

Jama, Mohamed A. F.

This study sought to evaluate the mathematics curriculum of the Halane Teacher Training Institute in Somalia with a view toward: (1) determining its weaknesses and recommending measures for improvement; (2) examining its relevance to the present needs of the Somali society; (3) determining the suitability of instructional materials and other…

Teacher Perspectives of the Use of Student Performance Data in Teacher Evaluations

ERIC Educational Resources Information Center

Hopkins, Paul Thomas

2013-01-01

The purpose of this study was to determine how K-12 public school teachers perceive the use of student performance data in teacher evaluations. The proprietary, utility, feasibility, and accuracy standards created by the Joint Committee on Standards for Education Evaluation (JCSEE) served as a framework for the study. An online survey was deployed…

Teacher Evaluation in China: Latest Trends and Future Directions

ERIC Educational Resources Information Center

Liu, Shujie; Zhao, Decheng

2013-01-01

With the implementation of teacher performance pay in 2009 in China, teacher performance evaluation has become a heated topic. This research study follows up on two previous studies of teacher evaluation in China and continues the dialog by analyzing the latest trends in the context of teacher performance pay. There were two sources of information…

Technology Evaluation Tools and Teacher Performance in Public Schools

ERIC Educational Resources Information Center

Stonehouse, Pauline; Keengwe, Jared

2013-01-01

The purpose of this study was, (a) to describe the introduction of mVAL software and Charlotte Danielson Rubrics (CDR) as teacher evaluation tools; (b) to compare the process and outcomes of the new initiative with traditional systems; and (c) to evaluate the software from the perspective of participants in the system. This study highlights the…

Evaluating the Accessibility of Web-Based Instruction for Students with Disabilities.

ERIC Educational Resources Information Center

Hinn, D. Michelle

This paper presents the methods and results of a year-long evaluation study, conducted for the purpose of determining disability accessibility barriers and potential solutions for those barriers found in four World Wide Web-based learning environments. The primary questions used to frame the evaluation study were: (1) Are there any features of the…

A Study of Kindergarten Teachers' Evaluation of Order-Managing Skills Used in Group Activities by Early Childhood Student Teachers in Taiwan.

ERIC Educational Resources Information Center

Chen, Ya-Mei

This study analyzed how Taiwanese kindergarten teachers evaluated events of various classroom management skills used by early childhood student teachers in group activities. The kindergarten teachers evaluated effectiveness, appropriateness, frequency of practicing similar skills themselves, and the need to modify any of the above. A questionnaire…

Empirical Evaluation Indicators in Thai Higher Education: Theory-Based Multidimensional Learners' Assessment

ERIC Educational Resources Information Center

Sritanyarat, Dawisa; Russ-Eft, Darlene

2016-01-01

This study proposed empirical indicators which can be validated and adopted in higher education institutions to evaluate quality of teaching and learning, and to serve as an evaluation criteria for human resource management and development of higher institutions in Thailand. The main purpose of this study was to develop empirical indicators of a…

Learning when Serious: Psychophysiological Evaluation of a Technology-Enhanced Learning Game

ERIC Educational Resources Information Center

Cowley, Ben; Fantato, Martino; Jennett, Charlene; Ruskov, Martin; Ravaja, Niklas

2014-01-01

We report an evaluation study for a novel learning platform, motivated by the growing need for methods to do assessment of serious game efficacy. The study was a laboratory experiment combining evaluation methods from the fields of learning assessment and psychophysiology. 15 participants used the TARGET game platform for 25 minutes, while the…

Evaluation Services from Needs Assessment to Follow-up: A Case Study.

ERIC Educational Resources Information Center

Broadbooks, Wendy J.

This paper describes the nature and scope of evaluation services provided within the training division of Arthur Andersen & Company, and highlights some of the evaluation results. The cycle of assessment began with a needs assessment study at the curriculum level. Curriculum planning was undertaken for first-year trainees in the Tax Division.…

A Mandatory, High-Stakes National Teacher Evaluation System: Perceptions and Attributions of Teachers Who Actively Refuse to Participate

ERIC Educational Resources Information Center

Tornero, Bernardita; Taut, Sandy

2010-01-01

This study examines why some public elementary school teachers openly refuse participation in a mandatory national, standards-based teacher evaluation program. We describe the perceptions these "rebel" teachers have of the evaluation system, studying their open resistance based on the meanings they construct, and elaborated an…

EVALUATION OF THE REAL-TIME AIR-QUALITY MODEL USING THE RAPS (REGIONAL AIR POLLUTION STUDY) DATA BASE. VOLUME 3. PROGRAM USER'S GUIDE

EPA Science Inventory

The theory and programming of statistical tests for evaluating the Real-Time Air-Quality Model (RAM) using the Regional Air Pollution Study (RAPS) data base are fully documented in four volumes. Moreover, the tests are generally applicable to other model evaluation problems. Volu...

How Analogue Research Can Advance Descriptive Evaluation Theory: Understanding (and Improving) Stakeholder Dialogue

ERIC Educational Resources Information Center

Campbell, Bernadette; Mark, Melvin M.

2015-01-01

Evaluation theories can be tested in various ways. One approach, the experimental analogue study, is described and illustrated in this article. The approach is presented as a method worthy to use in the pursuit of what Alkin and others have called descriptive evaluation theory. Drawing on analogue studies conducted by the first author, we…

Are They Paying Attention? Students' Lack of Motivation and Attention Potentially Threaten the Utility of Course Evaluations

ERIC Educational Resources Information Center

Bassett, Jonathan; Cleveland, Amanda; Acorn, Deborah; Nix, Marie; Snyder, Timothy

2017-01-01

Student evaluations are a common source of information used by instructors and administrators, but their utility depends on students' motivation and attention. This paper presents evidence from two studies indicating that insufficient effort responding and lack of motivation may be problems in course evaluations. In the first study, approximately…