Using GIS to evaluate a fire safety program in North Carolina.

PubMed

Dudley, Thomas; Creppage, Kathleen; Shanahan, Meghan; Proescholdbell, Scott

2013-10-01

Evaluating program impact is a critical aspect of public health. Utilizing Geographic Information Systems (GIS) is a novel way to evaluate programs which try to reduce residential fire injuries and deaths. The purpose of this study is to demonstrate the application of GIS within the evaluation of a smoke alarm installation program in North Carolina. This approach incorporates national fire incident data which, when linked with program data, provides a clear depiction of the 10 years impact of the Get Alarmed, NC! program and estimates the number of potential lives saved. We overlapped Get Alarmed, NC! program installation data with national information on fires using GIS to identify homes that experienced a fire after an alarm was installed and calculated potential lives saved based on program documentation and average housing occupancy. We found that using GIS was an efficient and quick way to match addresses from two distinct sources. From this approach we estimated that between 221 and 384 residents were potentially saved due to alarms installed in their homes by Get Alarmed, NC!. Compared with other program evaluations that require intensive and costly participant telephone surveys and/or in-person interviews, the GIS approach is inexpensive, quick, and can easily analyze large disparate datasets. In addition, it can be used to help target the areas most at risk from the onset. These benefits suggest that by incorporating previously unutilized data, the GIS approach has the potential for broader applications within public health program evaluation.

Charting the Impact of Federal Spending for Education Research: A Bibliometric Approach

ERIC Educational Resources Information Center

Milesi, Carolina; Brown, Kevin L.; Hawkley, Louise; Dropkin, Eric; Schneider, Barbara L.

2014-01-01

Impact evaluation plays a critical role in determining whether federally funded research programs in science, technology, engineering, and mathematics are wise investments. This paper develops quantitative methods for program evaluation and applies this approach to a flagship National Science Foundation-funded education research program, Research…

Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods

PubMed Central

2010-01-01

Background Schools are key settings for health promotion (HP) but the development of suitable approaches for evaluating HP in schools is still a major topic of discussion. This article presents a research protocol of a program developed to evaluate HP. After reviewing HP evaluation issues, the various possible approaches are analyzed and the importance of a realistic evaluation framework and a mixed methods (MM) design are demonstrated. Methods/Design The design is based on a systemic approach to evaluation, taking into account the mechanisms, context and outcomes, as defined in realistic evaluation, adjusted to our own French context using an MM approach. The characteristics of the design are illustrated through the evaluation of a nationwide HP program in French primary schools designed to enhance children's social, emotional and physical health by improving teachers' HP practices and promoting a healthy school environment. An embedded MM design is used in which a qualitative data set plays a supportive, secondary role in a study based primarily on a different quantitative data set. The way the qualitative and quantitative approaches are combined through the entire evaluation framework is detailed. Discussion This study is a contribution towards the development of suitable approaches for evaluating HP programs in schools. The systemic approach of the evaluation carried out in this research is appropriate since it takes account of the limitations of traditional evaluation approaches and considers suggestions made by the HP research community. PMID:20109202

Novel approaches and application of contemporary sensory evaluation practices in iron fortification programs

NASA Technical Reports Server (NTRS)

Bovell-Benjamin, Adelia C.; Guinard, Jean-Xavier

2003-01-01

Iron deficiency is the leading nutritional deficiency in the U.S. and the rest of the world, with its highest prevalences in the developing world. Iron fortification of food has been proposed as a strategy to reduce the high prevalence of iron deficiency. Poor consumer acceptance, unacceptable taste, and discoloration of the iron-fortified foods have been frequently listed as causes of unsuccessful iron fortification programs. An excellent prospect for improving consumer acceptance of iron-fortified foods is the incorporation of a thorough, organized, and unified approach to sensory evaluation practices into iron fortification programs for product optimization. The information gained from systematic sensory evaluation allows for the manipulation of the sensory attributes, and thus improvement of the sensory properties of the fortified food. However, iron fortification programs have not systematically measured the effect of fortification on the sensory quality of the food. Because sensory evaluation is an important criterion in successful iron fortification, an integrated approach is necessary. Therefore, nutritionists and sensory scientists should work closely with each other to select the most suitable sensory tests and methods. The objectives of this article are to: (1) critically review and discuss some traditional and contemporary approaches and applications of sensory evaluation practices in iron fortification programs, and (2) demonstrate the importance of incorporating a multidisciplinary, systematic sensory evaluation approach in iron fortification programs.

A Qualitative Program Evaluation of a Structured Leadership Mentoring Program at a Large Aerospace Corporation

ERIC Educational Resources Information Center

Teller, Romney P.

2011-01-01

The researcher utilized a qualitative approach to conduct a program evaluation of the organization where he is employed. The study intended to serve as a program evaluation for the structured in-house mentoring program at a large aerospace corporation (A-Corp). This program evaluation clarified areas in which the current mentoring program is…

Evaluation of an individualized sanctioning program for DWI offenders

DOT National Transportation Integrated Search

1998-12-01

Evaluates two different approaches to sentencing Driving While Intoxicated (DWI) offenders in Georgia. One approach (called the "Todd Program" after the judge who developed it) imposed individually-tailored combinations of traditional and alternative...

A Participatory Action Research Approach To Evaluating Inclusive School Programs.

ERIC Educational Resources Information Center

Dymond, Stacy K.

2001-01-01

This article proposes a model for evaluating inclusive schools. Key elements of the model are inclusion of stakeholders in the evaluation process through a participatory action research approach, analysis of program processes and outcomes, use of multiple methods and measures, and obtaining perceptions from diverse stakeholder groups. (Contains…

An Evaluation System for Training Programs: A Case Study Using a Four-Phase Approach

ERIC Educational Resources Information Center

Lingham, Tony; Richley, Bonnie; Rezania, Davar

2006-01-01

Purpose: With the increased importance of training in organizations, creating important and meaningful programs are critical to an organization and its members. The purpose of this paper is to suggest a four-phase systematic approach to designing and evaluating training programs that promotes collaboration between organizational leaders, trainers,…

Resident Evaluation and Remediation: A Comprehensive Approach

PubMed Central

Wu, Jim S.; Siewert, Bettina; Boiselle, Phillip M.

2010-01-01

Background A comprehensive evaluation and remediation program is an essential component of any residency program. The evaluation system should identify problems accurately and early and allow residents with problems to be assigned to a remediation program that effectively deals with them. Elements of a proactive remediation program include a process for outlining deficiencies, providing resources for improvement, communicating clear goals for acceptable performance, and reevaluating performance against these goals. Intervention In recognition of the importance of early detection and prompt remediation of the struggling resident, we sought to develop a multifaceted approach to resident evaluation with the aim of early identification and prompt remediation of difficulties. This article describes our comprehensive evaluation program and remediation program, which uses resources within our radiology department and institutional graduate medical education office. Discussion An effective evaluation system should identify problems accurately and early, whereas a proactive remediation program should effectively deal with issues once they are identified. PMID:21975628

The bottom-up approach to integrative validity: a new perspective for program evaluation.

PubMed

Chen, Huey T

2010-08-01

The Campbellian validity model and the traditional top-down approach to validity have had a profound influence on research and evaluation. That model includes the concepts of internal and external validity and within that model, the preeminence of internal validity as demonstrated in the top-down approach. Evaluators and researchers have, however, increasingly recognized that in an evaluation, the over-emphasis on internal validity reduces that evaluation's usefulness and contributes to the gulf between academic and practical communities regarding interventions. This article examines the limitations of the Campbellian validity model and the top-down approach and provides a comprehensive, alternative model, known as the integrative validity model for program evaluation. The integrative validity model includes the concept of viable validity, which is predicated on a bottom-up approach to validity. This approach better reflects stakeholders' evaluation views and concerns, makes external validity workable, and becomes therefore a preferable alternative for evaluation of health promotion/social betterment programs. The integrative validity model and the bottom-up approach enable evaluators to meet scientific and practical requirements, facilitate in advancing external validity, and gain a new perspective on methods. The new perspective also furnishes a balanced view of credible evidence, and offers an alternative perspective for funding. Copyright (c) 2009 Elsevier Ltd. All rights reserved.

Measuring Success: Evaluating Educational Programs

ERIC Educational Resources Information Center

Fisher, Yael

2010-01-01

This paper reveals a new evaluation model, which enables educational program and project managers to evaluate their programs with a simple and easy to understand approach. The "index of success model" is comprised of five parameters that enable to focus on and evaluate both the implementation and results of an educational program. The…

Program Fair Evaluation--Summative Appraisal of Instructional Sequences with Dissimilar Objectives.

ERIC Educational Resources Information Center

Popham, W. James

A comparative evaluation involving two instructional programs is given, although the approach can easily serve to compare more than two programs. The steps involved in conducting a program fair evaluation of two instructional programs are: (1) Identify objectives (a) common to both programs, (b) unique to one program, and (c) unique to the other…

Evaluating Cross-Cutting Approaches to Chronic Disease Prevention and Management: Developing a Comprehensive Evaluation

PubMed Central

Jernigan, Jan; Barnes, Seraphine Pitt; Shea, Pat; Davis, Rachel; Rutledge, Stephanie

2017-01-01

We provide an overview of the comprehensive evaluation of State Public Health Actions to Prevent and Control Diabetes, Heart Disease, Obesity and Associated Risk Factors and Promote School Health (State Public Health Actions). State Public Health Actions is a program funded by the Centers for Disease Control and Prevention to support the statewide implementation of cross-cutting approaches to promote health and prevent and control chronic diseases. The evaluation addresses the relevance, quality, and impact of the program by using 4 components: a national evaluation, performance measures, state evaluations, and evaluation technical assistance to states. Challenges of the evaluation included assessing the extent to which the program contributed to changes in the outcomes of interest and the variability in the states’ capacity to conduct evaluations and track performance measures. Given the investment in implementing collaborative approaches at both the state and national level, achieving meaningful findings from the evaluation is critical. PMID:29215974

The Discrepancy Evaluation Model: A Systematic Approach for the Evaluation of Career Planning and Placement Programs.

ERIC Educational Resources Information Center

Buttram, Joan L.; Covert, Robert W.

The Discrepancy Evaluation Model (DEM), developed in 1966 by Malcolm Provus, provides information for program assessment and program improvement. Under the DEM, evaluation is defined as the comparison of an actual performance to a desired standard. The DEM embodies five stages of evaluation based upon a program's natural development: program…

"Evaluation as Illumination: A New Approach to the Study of Innovatory Programs". Occasional Paper.

ERIC Educational Resources Information Center

Parlett, Malcolm; Hamilton, David

Conventional approaches to program evaluation have followed the experimental and psychometric traditions dominant in educational research. Their aim (unfulfilled) of achieving fully objective methods had led to studies that are artificial and restricted in scope. Illuminative evaluation is introduced as a total re-appraisal of the rationale and…

Evaluation of the focused approach to pedestrian safety program

DOT National Transportation Integrated Search

2009-02-02

This report summarizes the results of an evaluation of the Federal Highway Administration (FHWA) Focused Approach to Pedestrian Safety Program. The study was done by the Volpe National Transportation Systems Center at the request of the FHWA Office o...

Program Theory and Quality Matter: Changing the Course of Extension Program Evaluation

ERIC Educational Resources Information Center

Arnold, Mary E.; Cater, Melissa

2016-01-01

As internal evaluators for the 4-H program in two states, we simultaneously yet independently began to change the way we approached our evaluation practices, turning from evaluation capacity building (ECB) efforts that prepared educators to define and measure program outcomes to strategies that engage educators in defining and measuring program…

Integrating Program Theory and Systems-Based Procedures in Program Evaluation: A Dynamic Approach to Evaluate Educational Programs

ERIC Educational Resources Information Center

Grammatikopoulos, Vasilis

2012-01-01

The current study attempts to integrate parts of program theory and systems-based procedures in educational program evaluation. The educational program that was implemented, called the "Early Steps" project, proposed that physical education can contribute to various educational goals apart from the usual motor skills improvement. Basic…

Responsive Meta-Evaluation: A Participatory Approach to Enhancing Evaluation Quality

ERIC Educational Resources Information Center

Sturges, Keith M.; Howley, Caitlin

2017-01-01

In an era of ever-deepening budget cuts and a concomitant demand for substantiated programs, many organizations have elected to conduct internal program evaluations. Internal evaluations offer advantages (e.g., enhanced evaluator program knowledge and ease of data collection) but may confront important challenges, including credibility threats,…

Qualitative evaluation: A critical and interpretative complementary approach to improve health programs and services

PubMed Central

Tayabas, Luz María Tejada; León, Teresita Castillo; ESPINO, JOEL MONARREZ

2014-01-01

This short essay aims at commenting on the origin, development, rationale, and main characteristics of qualitative evaluation (QE), emphasizing the value of this methodological tool to evaluate health programs and services. During the past decades, different approaches have come to light proposing complementary alternatives to appraise the performance of public health programs, mainly focusing on the implementation process involved rather than on measuring the impact of such actions. QE is an alternative tool that can be used to illustrate and understand the process faced when executing health programs. It can also lead to useful suggestions to modify its implementation from the stakeholders’ perspectives, as it uses a qualitative approach that considers participants as reflective subjects, generators of meanings. This implies that beneficiaries become involved in an active manner in the evaluated phenomena with the aim of improving the health programs or services that they receive. With this work we want to encourage evaluators in the field of public health to consider the use of QE as a complementary tool for program evaluation to be able to identify areas of opportunity to improve programs’ implementation processes from the perspective of intended beneficiaries. PMID:25152220

Applying Program Theory-Driven Approach to Design and Evaluate a Teacher Professional Development Program

ERIC Educational Resources Information Center

Lin, Su-ching; Wu, Ming-sui

2016-01-01

This study was the first year of a two-year project which applied a program theory-driven approach to evaluating the impact of teachers' professional development interventions on students' learning by using a mix of methods, qualitative inquiry, and quasi-experimental design. The current study was to show the results of using the method of…

Integrating Data Mining in Program Evaluation of K-12 Online Education

ERIC Educational Resources Information Center

Hung, Jui-Long; Hsu, Yu-Chang; Rice, Kerry

2012-01-01

This study investigated an innovative approach of program evaluation through analyses of student learning logs, demographic data, and end-of-course evaluation surveys in an online K-12 supplemental program. The results support the development of a program evaluation model for decision making on teaching and learning at the K-12 level. A case study…

The development of a streamlined, coordinated and sustainable evaluation methodology for a diverse chronic disease management program.

PubMed

Berlowitz, David J; Graco, Marnie

2010-05-01

The Northern Alliance Hospital Admission Risk Program-Chronic Disease Management comprises 13 services delivering care to those with chronic disease and older people with complex care needs, who are frequent hospital users. To develop and implement a system-wide approach to the evaluation of this existing program. The Northern Clinical Research Centre audited all existing, routinely collected administrative data within the program and then met with each service to develop service specific outcome measures. The evaluators then developed and implemented a system-wide evaluation approach to measure performance in terms of: client profile; access and entry; service efficiency; client outcomes; and hospital demand. Data are collected electronically and more than 80% are derived from existing, administrative datasets, minimising staff and client burden. Additional data include client outcomes and a health related quality of life measure. The preliminary twelve month data suggest that clients have the equivalent of 'fair' or 'poor' self-reported health status (n = 862) and the average health utility scores are significantly (P < 0.05) worse than population control data. These analyses reveal, for the first time, that the Northern Alliance Hospital Admission Risk Program-Chronic Disease Management program is targeting appropriate clients. This methodology will enable many prospective assessments to be performed including; client outcome evaluation, service model comparisons, and cost-utility analyses. This evaluation approach demonstrates the feasibility of a highly coordinated 'whole of system' evaluation. Such an approach may ultimately contribute to the development of evidence-based policy.

Evaluation of training programs and entry-level qualifications for nuclear-power-plant control-room personnel based on the systems approach to training

DOE Office of Scientific and Technical Information (OSTI.GOV)

Haas, P M; Selby, D L; Hanley, M J

1983-09-01

This report summarizes results of research sponsored by the US Nuclear Regulatory Commission (NRC) Office of Nuclear Regulatory Research to initiate the use of the Systems Approach to Training in the evaluation of training programs and entry level qualifications for nuclear power plant (NPP) personnel. Variables (performance shaping factors) of potential importance to personnel selection and training are identified, and research to more rigorously define an operationally useful taxonomy of those variables is recommended. A high-level model of the Systems Approach to Training for use in the nuclear industry, which could serve as a model for NRC evaluation of industrymore » programs, is presented. The model is consistent with current publically stated NRC policy, with the approach being followed by the Institute for Nuclear Power Operations, and with current training technology. Checklists to be used by NRC evaluators to assess training programs for NPP control-room personnel are proposed which are based on this model.« less

Model Energy Efficiency Program Impact Evaluation Guide

EPA Pesticide Factsheets

Find guidance on model approaches for calculating energy, demand, and emissions savings resulting from energy efficiency programs. It describes several standard approaches that can be used in order to make these programs more efficient.

Assessment of Learning and Program Evaluation in Health Professions Education Programs

ERIC Educational Resources Information Center

Moore, Donald E., Jr.

2018-01-01

This chapter proposes approaches for assessing learners and evaluating courses and curriculum that could be used by directors of health professions education (HPE) programs to determine the effectiveness and impact of their programs.

Building a community-based culture of evaluation.

PubMed

Janzen, Rich; Ochocka, Joanna; Turner, Leanne; Cook, Tabitha; Franklin, Michelle; Deichert, Debbie

2017-12-01

In this article we argue for a community-based approach as a means of promoting a culture of evaluation. We do this by linking two bodies of knowledge - the 70-year theoretical tradition of community-based research and the trans-discipline of program evaluation - that are seldom intersected within the evaluation capacity building literature. We use the three hallmarks of a community-based research approach (community-determined; equitable participation; action and change) as a conceptual lens to reflect on a case example of an evaluation capacity building program led by the Ontario Brian Institute. This program involved two community-based groups (Epilepsy Southwestern Ontarioand the South West Alzheimer Society Alliance) who were supported by evaluators from the Centre for Community Based Research to conduct their own internal evaluation. The article provides an overview of a community-based research approach and its link to evaluation. It then describes the featured evaluation capacity building initiative, including reflections by the participating organizations themselves. We end by discussing lessons learned and their implications for future evaluation capacity building. Our main argument is that organizations that strive towards a community-based approach to evaluation are well placed to build and sustain a culture of evaluation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Guidelines "PEP," Peer Evaluation Program. A Systematic Approach for Evaluating Educational Programs, 1973-1974.

ERIC Educational Resources Information Center

Grotsky, Jeffery N.; And Others

The Peer Evaluation Program (PEP) has been instituted by the Division of Special Education, Pennsylvania State Department of Education, to allow intermediate units an opportunity to continuously improve their programs. The advantages of the PEP system are: (1) it is a self-improvememt system of program development; (2) PEP allows local autonomy as…

Evaluating Social Programs at the State and Local Level. The JTPA Evaluation Design Project.

ERIC Educational Resources Information Center

Blalock, Ann Bonar, Ed.; And Others

This book on evaluating social programs is an outcome of the Job Training Partnership Act (JTPA) Evaluation Design Project, which produced a set of 10 guides for the evaluation of state and local JTPA programs. This book distills ideas from these guides and applies them to a larger context. Part 1 presents a general approach to program evaluation…

Program Evaluation: Two Management-Oriented Samples

ERIC Educational Resources Information Center

Alford, Kenneth Ray

2010-01-01

Two Management-Oriented Samples details two examples of the management-oriented approach to program evaluation. Kenneth Alford, a doctorate candidate at the University of the Cumberlands, details two separate program evaluations conducted in his school district and seeks to compare and contrast the two evaluations based upon the characteristics of…

Motivation for Evaluation: A roadmap for Improving Program Efficacy

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D. F.; Welti, R.; Davis, H. B.

2016-12-01

Over the past year, the Incorporated Research Institutions for Seismology (IRIS) Education and Public Outreach (EPO) program has undertaken a new effort to increase the rigor with which it evaluates its programs and products. More specifically we sought to make evaluation an integral part of our EPO staff's work, enable staff to demonstrate why we do the activities we do, enhance the impact or our products and programs, and empower staff to be able to make evidence-based claims. The challenges we faced included a modest budget, finding an applicable approach to both new and legacy programs ranging from formal and informal education to public outreach, and implementing the process without overwhelming staff. The Collaborative Impact Analysis Method (IAM; Davis and Scalice, 2015) was selected as it allowed us to combine the EPO staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make incremental improvements to the evaluation of programs over time. We have found that this approach promotes the development of staff knowledge and skills regarding evaluation, provides a common language among staff, increases enthusiasm to collect and share data, encourages discussions of evaluative approaches when planning new activities, and improves each program's ability to capture the intended and unintended effects on the behaviors, attitudes, skills, interests, and/or knowledge of users/participants. We will share the initial IAM Scores for products and programs in the EPO portfolio, along with examples of the action plans for several key products and programs, and the impact that implementing those actions plans has had on our evaluations. Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis Method (Invited). Abstract ED53D-0871 presented at 2015 Fall Meeting, AGU, San Francisco, Calif., 14 - 18 Dec.

A Mixed-Methods Exploration of an Environment for Learning Computer Programming

ERIC Educational Resources Information Center

Mather, Richard

2015-01-01

A mixed-methods approach is evaluated for exploring collaborative behaviour, acceptance and progress surrounding an interactive technology for learning computer programming. A review of literature reveals a compelling case for using mixed-methods approaches when evaluating technology-enhanced-learning environments. Here, ethnographic approaches…

The Evaluation of Industry Practical of Mechanical Engineering in Vocational Education: A CIPP Model Approach

NASA Astrophysics Data System (ADS)

Kamaludin, M.; Munawar, W.; Mahdan, D.; Simanjuntak, M. V.; Wendi, H. F.

2018-02-01

The learning system is not only studied on campus but also practicing in the world of work. Industry Practical aims to enable students to develop their skills in accordance with the real world of work. To know the success of the implementation of industry practical program then held evaluation. The evaluation of the program in this study used the CIPP evaluation approach (Context, Input, Process, Product). The purpose of this research is to know the extent of achievement and success of industry practical program at vocational school in Bandung with descriptive research method using mix method approach. The sample in this research is students majoring in mechanical engineering in the city of Bandung who have done industry practical.

A Comparison of Methodological Approaches for Evaluating the Quality of Basic Skills Programs. AIR Forum 1982 Paper.

ERIC Educational Resources Information Center

Dumont, Richard G.; And Others

A multiple-indicator and multiple-design approach for evaluating the quality of basic skills programs in reading, writing, and mathematics that has been implemented within a statewide system of higher education in the southeastern United States was assessed. Several of the evaluation methodologies are compared in order to determine whether results…

Parts on Demand: Evaluation of Approaches to Achieve Flexible Manufacturing Systems for Navy Parts on Demand. Volume 2. Appendices

DTIC Science & Technology

1984-02-01

of Cost Analysis Worksheets * POD Program-Economic Analysis & Methodology - Economic Evaluation Procedures for POD Investment Program - System...Approval Considerations - POD Investment Program - Potential Improvement Areas for POD - Example Cost Categories and Determinants Appendix E Long Range R&D...Funding Profiles * Investment Strategy for Integrated Circuits Diminishing Sources of Supply - Problem Scope - Approach - Alternatives - Proposed

Evaluating programs that address ideological issues: ethical and practical considerations for practitioners and evaluators.

PubMed

Lieberman, Lisa D; Fagen, Michael C; Neiger, Brad L

2014-03-01

There are important practical and ethical considerations for organizations in conducting their own, or commissioning external, evaluations and for both practitioners and evaluators, when assessing programs built on strongly held ideological or philosophical approaches. Assessing whether programs "work" has strong political, financial, and/or moral implications, particularly when expending public dollars, and may challenge objectivity about a particular program or approach. Using a case study of the evaluation of a school-based abstinence-until-marriage program, this article discusses the challenges, lessons learned, and ethical responsibilities regarding decisions about evaluation, specifically associated with ideologically driven programs. Organizations should consider various stakeholders and views associated with their program to help identify potential pitfalls in evaluation. Once identified, the program or agency needs to carefully consider its answers to two key questions: Do they want the answer and are they willing to modify the program? Having decided to evaluate, the choice of evaluator is critical to assuring that ethical principles are maintained and potential skepticism or criticism of findings can be addressed appropriately. The relationship between program and evaluator, including agreements about ownership and eventual publication and/or promotion of data, should be addressed at the outset. Programs and organizations should consider, at the outset, their ethical responsibility when findings are not expected or desired. Ultimately, agencies, organizations, and programs have an ethical responsibility to use their data to provide health promotion programs, whether ideologically founded or not, that appropriately and effectively address the problems they seek to solve.

A multidimensional evaluation of a nursing information-literacy program.

PubMed Central

Fox, L M; Richter, J M; White, N E

1996-01-01

The goal of an information-literacy program is to develop student skills in locating, evaluating, and applying information for use in critical thinking and problem solving. This paper describes a multidimensional evaluation process for determining nursing students' growth in cognitive and affective domains. Results indicate improvement in student skills as a result of a nursing information-literacy program. Multidimensional evaluation produces a well-rounded picture of student progress based on formal measurement as well as informal feedback. Developing new educational programs can be a time-consuming challenge. It is important, when expending so much effort, to ensure that the goals of the new program are achieved and benefits to students demonstrated. A multidimensional approach to evaluation can help to accomplish those ends. In 1988, The University of Northern Colorado School of Nursing began working with a librarian to integrate an information-literacy component, entitled Pathways to Information Literacy, into the curriculum. This article describes the program and discusses how a multidimensional evaluation process was used to assess program effectiveness. The evaluation process not only helped to measure the effectiveness of the program but also allowed the instructors to use several different approaches to evaluation. PMID:8826621

Progress and Accountability in Family Literacy: Lessons from a Collaborative Approach.

ERIC Educational Resources Information Center

Ryan, Katherine E.; And Others

1996-01-01

The implementation of a collaborative approach to evaluating family literacy programs was studied using a conceptual framework and applied to 36 family literacy programs from a midwestern state. Evaluation participants learned how results could be used to develop curriculum in addition to reporting to funding agencies. (SLD)

Physiotherapists supporting self-management through health coaching: a mixed methods program evaluation.

PubMed

Dufour, Sinéad Patricia; Graham, Shane; Friesen, Josh; Rosenblat, Michael; Rous, Colin; Richardson, Julie

2015-01-01

To evaluate a program in support of chronic disease self-management (CDSM) that is founded on a health coaching (HC) approach, includes supervised exercise and mindfulness-based stress reduction components and is delivered within a private practice physiotherapy setting. An explanatory mixed method design, framed by theory-based program evaluation, was employed to evaluate an eight-week group-based program. Standardized self-rated and performance measures were evaluated pre- and post intervention. Additionally, participant focus groups were conducted following the intervention period. An inductive thematic approach was undertaken to analyze the qualitative data. Seventeen participants (N = 17) completed the study. Improvements were seen in both self-report and performance outcomes. Participants explained how and why they felt the program was beneficial. Six themes were generated: (1) group dynamic; (2) learning versus doing; (3) holism and comprehensive care; (4) self-efficacy and empowerment; (5) previous solutions versus new management strategies; and (6) healthcare provider support. This study established that a group program in support of CDSM founded on a HC approach demonstrated potential value from participants as well as favorable outcomes. A pragmatic randomized control trial is required to determine efficacy of this intervention.

Programs to improve adolescent sexual and reproductive health in the US: a review of the evidence.

PubMed

Manlove, Jennifer; Fish, Heather; Moore, Kristin Anderson

2015-01-01

US adolescents have high rates of teen pregnancy, childbearing, and sexually transmitted infections (STIs), highlighting the need to identify and implement effective programs that will help improve teen sexual and reproductive health. This review identified 103 random-assignment evaluations of 85 programs that incorporated intent-to-treat analyses and assessed impacts on pregnancy, childbearing, STIs, and their key determinants - sexual activity, number of sexual partners, condom use, and other contraceptive use - among teens. This review describes the evidence base for five broad program approaches, including abstinence education, comprehensive sex education, clinic-based programs, youth development programs, and parent-youth relationship programs. We also describe programs with impacts on key outcomes, including pregnancy/childbearing, STIs, and those that found impacts on both sexual activity and contraceptive use. Our review identified 52 effective programs: 38 with consistent impacts on reproductive health outcomes, and 14 with mixed findings (across subpopulations, follow-ups, or multiple measures of a single outcome). We found that a variety of program approaches produced impacts on sexual and reproductive health outcomes. Parent-youth relationship programs and clinic-based program evaluations more frequently showed impacts than other program approaches, although we also identified a number of abstinence-education, comprehensive sex education, and youth-development programs with impacts on sexual and reproductive health outcomes. Overall, we identified nine program evaluations with impacts on teen pregnancies or births, five with impacts on reducing STIs, and 15 with impacts on both delaying/reducing sexual activity and increasing contraceptive use (including condom use). Future efforts should conduct replications of existing program evaluations, identify implementation components linked to impacts, rigorously evaluate programs that appear promising, and expand the evidence base on programs that impact hormonal and long-acting contraceptive method use.

Evaluation of Career Development Programs from an Action Perspective.

ERIC Educational Resources Information Center

Young, Richard A.; Valach, Ladislav

1994-01-01

Presents action-theoretical approach to evaluation of career development programs based on constructionist epistemology. Propositions from action-theoretical perspective center around career and action as related, interpretative constructs. Propositions give rise to implications for evaluation of career programs that address ongoing nature of…

Evaluation of STD/AIDS prevention programs: a review of approaches and methodologies.

PubMed

da Cruz, Marly Marques; dos Santos, Elizabeth Moreira; Monteiro, Simone

2007-05-01

The article presents a review of approaches and methodologies in the evaluation of STD/AIDS prevention programs, searching for theoretical and methodological support for the institutionalization of evaluation and decision-making. The review included the MEDLINE, SciELO, and ISI Web of Science databases and other sources like textbooks and congress abstracts from 1990 to 2005, with the key words: "evaluation", "programs", "prevention", "STD/AIDS", and similar terms. The papers showed a predominance of quantitative outcome or impact evaluative studies with an experimental or quasi-experimental design. The main use of evaluation is accountability, although knowledge output and program improvement were also identified in the studies. Only a few evaluative studies contemplate process evaluation and its relationship to the contexts. The review aimed to contribute to the debate on STD/AIDS, which requires more effective, consistent, and sustainable decisions in the field of prevention.

Adapting the Empowerment Evaluation Model: A Mental Health Drop-In Center Case Example

ERIC Educational Resources Information Center

Sullins, Carolyn D.

2003-01-01

Empowerment evaluation involves a program's stakeholders in designing and implementing an evaluation of their own program, thus contributing to the program's improvement and self-determination (Fetterman, 1994, 1996). It appeared to be an appropriate approach for evaluating a mental health drop-in center, which had congruent goals of collaboration…

Evaluating disease management program effectiveness: an introduction to survival analysis.

PubMed

Linden, Ariel; Adams, John L; Roberts, Nancy

2004-01-01

Currently, the most widely used method in the disease management industry for evaluating program effectiveness is the "total population approach." This model is a pretest-posttest design, with the most basic limitation being that without a control group, there may be sources of bias and/or competing extraneous confounding factors that offer plausible rationale explaining the change from baseline. Survival analysis allows for the inclusion of data from censored cases, those subjects who either "survived" the program without experiencing the event (e.g., achievement of target clinical levels, hospitalization) or left the program prematurely, due to disenrollement from the health plan or program, or were lost to follow-up. Additionally, independent variables may be included in the model to help explain the variability in the outcome measure. In order to maximize the potential of this statistical method, validity of the model and research design must be assured. This paper reviews survival analysis as an alternative, and more appropriate, approach to evaluating DM program effectiveness than the current total population approach.

Spacelab Level 4 programmatic Implementation Assessment Study. Volume 3: Optimization and programmatics

NASA Technical Reports Server (NTRS)

1978-01-01

Alternate level 4 integration approaches were synthesized and evaluated to establish the most cost effective experiment integration approach. Program baseline system trade studies are described, as well as Spacelab equipment utilization. Programmatic analysis of the baseline program was evaluated; the 2/3 and 1/3 traffic models were also considered.

Determining Safety Inspection Thresholds for Employee Incentives Programs on Construction Sites

PubMed Central

Sparer, Emily; Dennerlein, Jack

2017-01-01

The goal of this project was to evaluate approaches of determining the numerical value of a safety inspection score that would activate a reward in an employee safety incentive program. Safety inspections are a reflection of the physical working conditions at a construction site and provide a safety score that can be used in incentive programs to reward workers. Yet it is unclear what level of safety should be used when implementing this kind of program. This study explored five ways of grouping safety inspection data collected during 19 months at Harvard University-owned construction projects. Each approach grouped the data by one of the following: owner, general contractor, project, trade, or subcontractor. The median value for each grouping provided the threshold score. These five approaches were then applied to data from a completed project in order to calculate the frequency and distribution of rewards in a monthly safety incentive program. The application of each approach was evaluated qualitatively for consistency, competitiveness, attainability, and fairness. The owner-specific approach resulted in a threshold score of 96.3% and met all of the qualitative evaluation goals. It had the most competitive reward distribution (only 1/3 of the project duration) yet it was also attainable. By treating all workers equally and maintaining the same value throughout the project duration, this approach was fair and consistent. The owner-based approach for threshold determination can be used by owners or general contractors when creating leading indicator incentives programs and by researchers in future studies on incentive program effectiveness. PMID:28638178

Lazy evaluation of FP programs: A data-flow approach

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wei, Y.H.; Gaudiot, J.L.

1988-12-31

This paper presents a lazy evaluation system for the list-based functional language, Backus` FP in data-driven environment. A superset language of FP, called DFP (Demand-driven FP), is introduced. FP eager programs are transformed into DFP lazy programs which contain the notions of demands. The data-driven execution of DFP programs has the same effects of lazy evaluation. DFP lazy programs have the property of always evaluating a sufficient and necessary result. The infinite sequence generator is used to demonstrate the eager-lazy program transformation and the execution of the lazy programs.

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program.

PubMed

Jaafaripooyan, Ebrahim

2014-09-01

Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs' over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs.

Native Language Reading Approach Program, 1982-1983. O.E.E. Final Evaluation Report.

ERIC Educational Resources Information Center

Keyes, Jose Luis; And Others

The Native Language Reading Approach Program in New York City was designed as an exemplary approach to on-site training of classroom teachers and their assistants in how to help students transfer reading skills from their native language to English. Program components included support services, teacher training, material/curriculum development,…

Building a patient-centered and interprofessional training program with patients, students and care professionals: study protocol of a participatory design and evaluation study.

PubMed

Vijn, Thomas W; Wollersheim, Hub; Faber, Marjan J; Fluit, Cornelia R M G; Kremer, Jan A M

2018-05-30

A common approach to enhance patient-centered care is training care professionals. Additional training of patients has been shown to significantly improve patient-centeredness of care. In this participatory design and evaluation study, patient education and medical education will be combined by co-creating a patient-centered and interprofessional training program, wherein patients, students and care professionals learn together to improve patient-centeredness of care. In the design phase, scientific literature regarding interventions and effects of student-run patient education will be synthesized in a scoping review. In addition, focus group studies will be performed on the preferences of patients, students, care professionals and education professionals regarding the structure and content of the training program. Subsequently, an intervention plan of the training program will be constructed by combining these building blocks. In the evaluation phase, patients with a chronic disease, that is rheumatoid arthritis, diabetes and hypertension, and patients with an oncologic condition, that is colonic cancer and breast cancer, will learn together with medical students, nursing students and care professionals in training program cycles of three months. Process and effect evaluation will be performed using the plan-do-study-act (PDSA) method to evaluate and optimize the training program in care practice and medical education. A modified control design will be used in PDSA-cycles to ensure that students who act as control will also benefit from participating in the program. Our participatory design and evaluation study provides an innovative approach in designing and evaluating an intervention by involving participants in all stages of the design and evaluation process. The approach is expected to enhance the effectiveness of the training program by assessing and meeting participants' needs and preferences. Moreover, by using fast PDSA cycles and a modified control design in evaluating the training program, the training program is expected to be efficiently and rapidly implemented into and adjusted to care practice and medical education.

The Positive Alternative Credit Experience (PACE) Program a Quantitative Comparative Study

ERIC Educational Resources Information Center

Warren, Rebecca Anne

2011-01-01

The purpose of this quantitative comparative study was to evaluate the Positive Alternative Credit Experience (PACE) Program using an objectives-oriented approach to a formative program evaluation. The PACE Program was a semester-long high school alternative education program designed to serve students at-risk for academic failure or dropping out…

Qualitative Evaluation of Emotional Intelligence In-Service Program for Secondary School Teachers

ERIC Educational Resources Information Center

Fer, Seval

2004-01-01

This paper is an attempt to evaluate the Emotional Intelligence (EQ) In-Service Program on the basis of experiences of 20 secondary school teachers who attended the program in a private school in Turkey. A phenomenological approach, with a focus group method was used. The first objective of this study was to evaluate EQ program on the basis of…

Assessment of the Teaching Behavior of the Instructors of an Out-of-School Program

ERIC Educational Resources Information Center

Rodríguez-Naveiras, Elena; Borges, África

2015-01-01

Out-of-school programs for students with high abilities are especially relevant when their needs are not covered in formal education. The evaluation of these programs is essential and it can be carried out from different evaluative approaches. The evaluation of the behavior of the people who implement the programs is an important aspect in the…

Program Evaluation: A Review and Synthesis.

ERIC Educational Resources Information Center

Webber, Charles F.

This paper reviews models of program evaluation. Major topics and issues found in the evaluation literature include quantitative versus qualitative approaches, identification and involvement of stakeholders, formulation of research questions, collection of data, analysis and interpretation of data, reporting of results, evaluation utilization, and…

Evaluation as a critical factor of success in local public health accreditation programs.

PubMed

Tremain, Beverly; Davis, Mary; Joly, Brenda; Edgar, Mark; Kushion, Mary L; Schmidt, Rita

2007-01-01

This article presents the variety of approaches used to conduct evaluations of performance improvement or accreditation systems, while illustrating the complexity of conducting evaluations to inform local public health practice. We, in addition, hope to inform the Exploring Accreditation Program about relevant experiences involving accreditation and performance assessment processes, specifically evaluation, as it debates and discusses a national voluntary model. A background of each state is given. To further explore these issues, interviews were conducted with each state's evaluator to gain more in-depth information on the many different evaluation strategies and approaches used. On the basis of the interviews, the authors provide several overall themes, which suggest that evaluation is a critical tool and success factor for performance assessment or accreditation programs.

Students' Reactions to the Use of Animals as Instructional Tools for Science

ERIC Educational Resources Information Center

Gurzau, Vickie Anne

2013-01-01

Teaching science to students with special needs requires different approaches. A program at the study site supplemented packaged science kits with animal-assisted instruction, but such an approach was never evaluated. The purpose of this study was to provide a formative program evaluation documenting students' reactions to the use of animals…

Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development.

PubMed

Archibald, Thomas; Sharrock, Guy; Buckley, Jane; Cook, Natalie

2016-12-01

Unexamined and unjustified assumptions are the Achilles' heel of development programs. In this paper, we describe an evaluation capacity building (ECB) approach designed to help community development practitioners work more effectively with assumptions through the intentional infusion of evaluative thinking (ET) into the program planning, monitoring, and evaluation process. We focus specifically on one component of our ET promotion approach involving the creation and analysis of theory of change (ToC) models. We describe our recent efforts to pilot this ET ECB approach with Catholic Relief Services (CRS) in Ethiopia and Zambia. The use of ToC models, plus the addition of ET, is a way to encourage individual and organizational learning and adaptive management that supports more reflective and responsive programming. Copyright © 2016 Elsevier Ltd. All rights reserved.

The Nursing Leadership Institute program evaluation: a critique

PubMed Central

Havaei, Farinaz; MacPhee, Maura

2015-01-01

A theory-driven program evaluation was conducted for a nursing leadership program, as a collaborative project between university faculty, the nurses’ union, the provincial Ministry of Health, and its chief nursing officers. A collaborative logic model process was used to engage stakeholders, and mixed methods approaches were used to answer evaluation questions. Despite demonstrated, successful outcomes, the leadership program was not supported with continued funding. This paper examines what happened during the evaluation process: What factors failed to sustain this program? PMID:29355180

A responsive evaluation of an Aboriginal nursing education access program.

PubMed

Curran, Vernon; Solberg, Shirley; LeFort, Sandra; Fleet, Lisa; Hollett, Ann

2008-01-01

Nursing education access programs have been introduced in a number of countries to address the shortage of healthcare providers of Aboriginal descent. An evaluation study of a nursing education access program in Labrador, Canada, was undertaken using a Responsive Evaluation approach. Interviews and focus groups with program stakeholders were conducted. Program effectiveness was influenced by culturally relevant curriculum, experiential and authentic learning opportunities, academic and social support, and the need for partnership building between stakeholders. The authors report key findings resulting from the Responsive Evaluation.

Using System Dynamics as an Evaluation Tool: Experience from a Demonstration Program

ERIC Educational Resources Information Center

Fredericks, Kimberly A.; Deegan, Michael; Carman, Joanne G.

2008-01-01

Evaluators are often faced with many challenges in the design and implementation of a program's evaluation. Because programs are entangled in complex networks of structures and stakeholders, they can be challenging to understand, and they often pose issues of competing and conflicting goals. However, by using a systems mapping approach to…

Handbook in Evaluating with Photography.

ERIC Educational Resources Information Center

Templin, Patricia A.

This handbook is intended to help educational evaluators use still photography in designing, conducting, and reporting evaluations of educational programs. It describes techniques for using a visual documentary approach to program evaluation that features data collected with a camera. The emphasis is on the aspects of educational evaluation…

Detecting selection effects in community implementations of family-based substance abuse prevention programs.

PubMed

Hill, Laura G; Goates, Scott G; Rosenman, Robert

2010-04-01

To calculate valid estimates of the costs and benefits of substance abuse prevention programs, selection effects must be identified and corrected. A supplemental comparison sample is typically used for this purpose, but in community-based program implementations, such a sample is often not available. We present an evaluation design and analytic approach that can be used in program evaluations of real-world implementations to identify selection effects, which in turn can help inform recruitment strategies, pinpoint possible selection influences on measured program outcomes, and refine estimates of program costs and benefits. We illustrate our approach with data from a multisite implementation of a popular substance abuse prevention program. Our results indicate that the program's participants differed significantly from the population at large.

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program

PubMed Central

Jaafaripooyan, Ebrahim

2014-01-01

Background: Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. Methods: A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Results: Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs’ over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Conclusion: Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs. PMID:25279381

Faculty-led faculty development: evaluation and reflections on a distributed educational leadership model.

PubMed

Elzubeir, Margaret

2011-01-01

This report describes and explores the impact of a series of faculty-led faculty development programs underpinned by principles of distributed educational leadership. We aimed to prepare faculty for their roles as facilitators and assessors in a newly implemented problem-based (PBL) graduate entry medical program. We asked participants attending a series of faculty development programs to evaluate workshops attended using an in-house designed survey. Overall descriptive statistics for all workshops and qualitative feedback for PBL workshops alone were examined. It was concluded that clinical faculty who are not specialized in medical education can offer high-quality, well-accepted training for their peers. Faculty development, underpinned by a distributed leadership approach which supports learning organization tenets, imaginative, flexible and democratic approaches to developing and nurturing expertise at all levels of the organization, is likely to lead to improvements in medical education. Despite the limitations of the survey approach to evaluation of faculty development programs, the information provided is useful both as a basis for decision making and program improvement.

A Value-Added Approach to Selecting the Best Master of Business Administration (MBA) Program

ERIC Educational Resources Information Center

Fisher, Dorothy M.; Kiang, Melody; Fisher, Steven A.

2007-01-01

Although numerous studies rank master of business administration (MBA) programs, prospective students' selection of the best MBA program is a formidable task. In this study, the authors used a linear-programming-based model called data envelopment analysis (DEA) to evaluate MBA programs. The DEA model connects costs to benefits to evaluate the…

Historical context for the creation of the Office of Adolescent Health and the Teen Pregnancy Prevention Program.

PubMed

Kappeler, Evelyn M; Farb, Amy Feldman

2014-03-01

In Fiscal Year 2010, Federal funds were dedicated to support evidence-based approaches to effectively target teen pregnancy prevention and resulted in the establishment of the Office of Adolescent Health (OAH) and the Teen Pregnancy Prevention (TPP) Program. Through the tiered TPP Program, OAH supports replication and evaluation of programs using models whose effectiveness has been demonstrated through rigorous evaluation and the development and testing of promising or innovative pregnancy prevention strategies and approaches. This article documents the creation of OAH and the development of the TPP Program, the identification of a TPP evidence base, current program and evaluation efforts at OAH, and government coordination and partnerships related to reducing teen pregnancy. This article is of interest to those working to improve the health and wellbeing of adolescents. Published by Elsevier Inc.

Assumptions at the philosophical and programmatic levels in evaluation.

PubMed

Mertens, Donna M

2016-12-01

Stakeholders and evaluators hold a variety of levels of assumptions at the philosophical, methodological, and programmatic levels. The use of a transformative philosophical framework is presented as a way for evaluators to become more aware of the implications of various assumptions made by themselves and program stakeholders. The argument is examined and demonstrated that evaluators who are aware of the assumptions that underlie their evaluation choices are able to provide useful support for stakeholders in the examination of the assumptions they hold with regard to the nature of the problem being addressed, the program designed to solve the problem, and the approach to evaluation that is appropriate in that context. Such an informed approach has the potential for development of more appropriate and culturally responsive programs being implemented in ways that lead to the desired impacts, as well as to lead to evaluation approaches that support effective solutions to intransigent social problems. These arguments are illustrated through examples of evaluations from multiple sectors; additional challenges are also identified. Copyright © 2016 Elsevier Ltd. All rights reserved.

Programs to improve adolescent sexual and reproductive health in the US: a review of the evidence

PubMed Central

Manlove, Jennifer; Fish, Heather; Moore, Kristin Anderson

2015-01-01

Background US adolescents have high rates of teen pregnancy, childbearing, and sexually transmitted infections (STIs), highlighting the need to identify and implement effective programs that will help improve teen sexual and reproductive health. Materials and methods This review identified 103 random-assignment evaluations of 85 programs that incorporated intent-to-treat analyses and assessed impacts on pregnancy, childbearing, STIs, and their key determinants – sexual activity, number of sexual partners, condom use, and other contraceptive use – among teens. This review describes the evidence base for five broad program approaches, including abstinence education, comprehensive sex education, clinic-based programs, youth development programs, and parent–youth relationship programs. We also describe programs with impacts on key outcomes, including pregnancy/childbearing, STIs, and those that found impacts on both sexual activity and contraceptive use. Results Our review identified 52 effective programs: 38 with consistent impacts on reproductive health outcomes, and 14 with mixed findings (across subpopulations, follow-ups, or multiple measures of a single outcome). We found that a variety of program approaches produced impacts on sexual and reproductive health outcomes. Parent–youth relationship programs and clinic-based program evaluations more frequently showed impacts than other program approaches, although we also identified a number of abstinence-education, comprehensive sex education, and youth-development programs with impacts on sexual and reproductive health outcomes. Overall, we identified nine program evaluations with impacts on teen pregnancies or births, five with impacts on reducing STIs, and 15 with impacts on both delaying/reducing sexual activity and increasing contraceptive use (including condom use). Conclusion Future efforts should conduct replications of existing program evaluations, identify implementation components linked to impacts, rigorously evaluate programs that appear promising, and expand the evidence base on programs that impact hormonal and long-acting contraceptive method use. PMID:25897271

Guide Lines for Evaluation of Continuing Education Programs in Mental Health.

ERIC Educational Resources Information Center

Miller, Norma; And Others

Suggestions for program administrators and training program directors to develop comprehensive plans based on principles of community involvement, education, administration and finance, and the disciplines being taught are broadly outlined. Three accompanying charts illustrate approach to evaluation planning. (NF)

Title I of the Higher Education Act of 1965: Evaluation of the Present Program: Recommendations for the Future.

ERIC Educational Resources Information Center

Whipple, James B.

In this document, which points out weaknesses in evaluation procedures and offers a new approach to the subject, it is suggested that in the area of the United States studied, the Title 1 program is drifting without direction, leadership, or system. This makes evaluation impossible. Evaluation is sometimes a description of a program and often…

Pitfalls in Evaluating the Effectiveness of Case Management Programs for Homeless Persons: Lessons from the NIAAA Community Demonstration Program.

ERIC Educational Resources Information Center

Orwin, Robert G.; And Others

1994-01-01

Evaluation findings of the National Institute on Alcohol Abuse and Alcoholism (NIAAA) Community Demonstration Program show few clear indications of the effectiveness of case management approaches. This article identifies phenomena that explain the lack of positive effects and suggests that evaluation problems may explain apparent negative…

Improving Programs for the Preparation of Teachers through the Use of Program Evaluation--The Second Phase.

ERIC Educational Resources Information Center

Berney, Mary F.

Plans for the field-testing and implementation of the Accreditation Plus Model (APM) for evaluation of teacher preparation programs are described. Field-testing is taking place in the Center for Teacher Education Evaluation at Tennessee Technological University (Cookeville). The APM is based on the professional judgment approach to program…

A Mixed Methods Approach to Understanding School Counseling Program Evaluation: High School Counselors' Methods and Perceptions

ERIC Educational Resources Information Center

Aucoin, Jennifer Mangrum

2013-01-01

The purpose of this mixed methods concurrent triangulation study was to examine the program evaluation practices of high school counselors. A total of 294 high school counselors in Texas were assessed using a mixed methods concurrent triangulation design. A researcher-developed survey, the School Counseling Program Evaluation Questionnaire…

Evaluation of the Leadership Institute: A Program to Build Individual and Organizational Capacity through Emotional Intelligence

ERIC Educational Resources Information Center

Chen, Claire Yueh-Ti; King, Jeff; Cochran, Graham R.; Argabright, Karen J.

2014-01-01

The purpose of the study reported here was to evaluate the impact of Leadership Institute, a program designed to strengthen leadership capacity through developing individuals' emotional intelligence (EQ). A pre-and posttest approach was used to collect data from two workshops with identical EQ content, program structure, and evaluation.…

Cancer Education Program Evaluation: A Responsive Approach to Planning an Evaluation and Initial Results.

ERIC Educational Resources Information Center

Pearsol, James A.

This paper describes evaluation planning for the Cancer Education Program (CEP) at Ohio State University (OSU). The three-year OSU CEP project was designed as a multidisciplinary cancer education program. A responsive method, which trades off some measurement precision in order to increase the usefulness of the findings, was employed in the…

Supplemental Literacy Instruction for Students with Down Syndrome: A Program Evaluation

ERIC Educational Resources Information Center

Regan, Lisa Michelle

2013-01-01

The study utilizes an inductive, qualitative approach to program evaluation to understand the nature of an afterschool literacy tutoring program for students with Down syndrome. Two research questions guide this study: (a) What are the curricular and instructional elements of the Let's Read Now (LRN) literacy tutoring program for students…

A Review of Elementary School-Based Substance Use Prevention Programs: Identifying Program Attributes

ERIC Educational Resources Information Center

Hopfer, S.; Davis, D.; Kam, J. A.; Shin, Y.; Elek, E.; Hecht, M. L.

2010-01-01

This article takes a systematic approach to reviewing substance use prevention programs introduced in elementary school (K-6th grade). Previous studies evaluating such programs among elementary school students showed mixed effects on subsequent substance use and related psychosocial factors. Thirty published evaluation studies of 24 elementary…

Training Evaluation as an Integral Component of Training for Performance.

ERIC Educational Resources Information Center

Lapp, H. J., Jr.

A training evaluation system should address four major areas: reaction, learning, behavior, and results. The training evaluation system at GPU Nuclear Corporation addresses each of these areas through practical approaches such as course and program evaluation. GPU's program evaluation instrument uses a Likert-type scale to assess task development,…

Evaluation Options for Family Resource Centers.

ERIC Educational Resources Information Center

Horsch, Karen, Ed.; Weiss, Heather B., Ed.

Family resource centers (FRC) are emerging as a promising program approach to solving urgent social problems. Evaluation plays an important role in learning how these programs work, what their impact is, and whether they should be expanded. However, FRCs pose unique challenges to evaluation. This report considers the challenges to evaluating FRCs,…

Evaluation: An Imperative for Public Education and Training Programs. Professorial Inaugural Lecture Series.

ERIC Educational Resources Information Center

McCaslin, N. L.

Evaluation is important to public education and training programs if the United States is to be competitive in a global economy. Six major factors have had an impact on evaluation efforts: decreased financial resources, increased public dissatisfaction, changed management approaches, enhanced evaluation methods and procedures, redesigned…

Evaluating a physician leadership development program - a mixed methods approach.

PubMed

Throgmorton, Cheryl; Mitchell, Trey; Morley, Tom; Snyder, Marijo

2016-05-16

Purpose - With the extent of change in healthcare today, organizations need strong physician leaders. To compensate for the lack of physician leadership education, many organizations are sending physicians to external leadership programs or developing in-house leadership programs targeted specifically to physicians. The purpose of this paper is to outline the evaluation strategy and outcomes of the inaugural year of a Physician Leadership Academy (PLA) developed and implemented at a Michigan-based regional healthcare system. Design/methodology/approach - The authors applied the theoretical framework of Kirkpatrick's four levels of evaluation and used surveys, observations, activity tracking, and interviews to evaluate the program outcomes. The authors applied grounded theory techniques to the interview data. Findings - The program met targeted outcomes across all four levels of evaluation. Interview themes focused on the significance of increasing self-awareness, building relationships, applying new skills, and building confidence. Research limitations/implications - While only one example, this study illustrates the importance of developing the evaluation strategy as part of the program design. Qualitative research methods, often lacking from learning evaluation design, uncover rich themes of impact. The study supports how a PLA program can enhance physician learning, engagement, and relationship building throughout and after the program. Physician leaders' partnership with organization development and learning professionals yield results with impact to individuals, groups, and the organization. Originality/value - Few studies provide an in-depth review of evaluation methods and outcomes of physician leadership development programs. Healthcare organizations seeking to develop similar in-house programs may benefit applying the evaluation strategy outlined in this study.

Evaluation of the Meaning of Life Program in Israel

ERIC Educational Resources Information Center

Kasler, Jonathan; White, Gwyne W.; Elias, Maurice J.

2013-01-01

During the 2009-2010 academic year, 10 schools participated in the Meaning of Life educational program, an adaption of the popular U.S. Laws of Life program. The program sought to encourage each participant to develop a personal approach to finding meaning in life. To evaluate the success of the program, we conducted a study to compare measures of…

A practical assessment of physician biopsychosocial performance.

PubMed

Margalit, Alon Pa; Glick, Shimon M; Benbassat, Jochanan; Cohen, Ayala; Margolis, Carmi Z

2007-10-01

A biopsychosocial approach to care seems to improve patient satisfaction and health outcomes. Nevertheless, this approach is not widely practiced, possibly because its precepts have not been translated into observable skills. To identify the skill components of a biopsychosocial consultation and develop an tool for their evaluation. We approached three e-mail discussion groups of family physicians and pooled their responses to the question "what types of observed physician behavior would characterize a biopsychosocial consultation?" We received 35 responses describing 37 types of behavior, all of which seemed to cluster around one of three aspects: patient-centered interview; system-centered and family-centered approach to care; or problem-solving orientation. Using these categories, we developed a nine-item evaluation tool. We used the evaluation tool to score videotaped encounters of patients with two types of doctors: family physicians who were identified by peer ratings to have a highly biopsychosocial orientation (n = 9) or a highly biomedical approach (n = 4); and 44 general practitioners, before and after they had participated in a program that taught a biopsychosocial approach to care. The evaluation tool was found to demonstrate high reliability (alpha = 0.90) and acceptable interobserver variability. The average scores of the physicians with a highly biopsychosocial orientation were significantly higher than those of physicians with a highly biomedical approach. There were significant differences between the scores of the teaching-program participants before and after the program. A biopsychosocial approach to patient care can be characterized using a valid and easy-to-apply evaluation tool.

A Synergistic Approach to Faculty Mentoring

ERIC Educational Resources Information Center

Goodwin, Laura D.

2004-01-01

Following a comparison of two approaches to mentoring--the traditional model and a relatively new "synergistic" or co-mentoring model--a new formal mentoring program for faculty in the School of Education at the University of Colorado at Denver, based on the synergistic approach, is described. First-year program evaluation data revealed…

New approach in the evaluation of a fitness program at a worksite.

PubMed

Shirasaya, K; Miyakawa, M; Yoshida, K; Tanaka, C; Shimada, N; Kondo, T

1999-03-01

The most common methods for the economic evaluation of a fitness program at a worksite are cost-effectiveness, cost-benefit, and cost-utility analyses. In this study, we applied a basic microeconomic theory, "neoclassical firm's problems," as the new approach for it. The optimal number of physical-exercise classes that constitute the core of the fitness program are determined using the cubic health production function. The optimal number is defined as the number that maximizes the profit of the program. The optimal number corresponding to any willingness-to-pay amount of the participants for the effectiveness of the program is presented using a graph. For example, if the willingness-to-pay is $800, the optimal number of classes is 23. Our method can be applied to the evaluation of any health care program if the health production function can be estimated.

Developing a comprehensive resident education evaluation system in the era of milestone assessment.

PubMed

Gardner, Aimee K; Scott, Daniel J; Choti, Michael A; Mansour, John C

2015-01-01

In an effort to move training programs toward competency-based education, the Accreditation Council for Graduate Medical Education (ACGME) introduced the Next Accreditation System (NAS), which organizes specific milestones regarding resident skills, knowledge, and abilities along a continuum. In order to foster innovation and creativity, the ACGME has provided programs with minimal guidelines regarding the optimal way to approach these milestones. The education team at UT Southwestern embraced the milestones and developed a process in which performance assessment methods were critically evaluated, mapped onto an extrapolated performance list corresponding to the areas required by the ACGME milestones, and filled gaps in the previous system by modifying evaluation tools and creating new program components. Although the authors are early in the evolution of applying the new milestones system, this approach has thus far allowed them to comprehensively evaluate the residents and the program in an efficient and effective fashion, with notable improvements compared to the prior approach. The authors hope that these experiences can inform others embarking upon similar journeys with the milestones. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Empowerment evaluation with programs designed to prevent first-time male perpetration of sexual violence.

PubMed

Noonan, Rita K; Gibbs, Deborah

2009-01-01

This special issue captures several threads in the ongoing evolution of sexual violence prevention. The articles that follow examine an empowerment evaluation process with four promising programs dedicated to preventing first-time male perpetration of sexual violence, as well as evaluation findings. Both the evaluation approach and the programs examined shed light on how sexual violence prevention can continue to be improved in the future.

The Practice of Health Program Evaluation.

PubMed

Lewis, Sarah R

2017-11-01

The Practice of Health Program Evaluation provides an overview of the evaluation process for public health programs while diving deeper to address select advanced concepts and techniques. The book unfolds evaluation as a three-phased process consisting of identification of evaluation questions, data collection and analysis, and dissemination of results and recommendations. The text covers research design, sampling methods, as well as quantitative and qualitative approaches. Types of evaluation are also discussed, including economic assessment and systems research as relative newcomers. Aspects critical to conducting a successful evaluation regardless of type or research design are emphasized, such as stakeholder engagement, validity and reliability, and adoption of sound recommendations. The book encourages evaluators to document their approach by developing an evaluation plan, a data analysis plan, and a dissemination plan, in order to help build consensus throughout the process. The evaluative text offers a good bird's-eye view of the evaluation process, while offering guidance for evaluation experts on how to navigate political waters and advocate for their findings to help affect change.

An e-Learning Collaborative Filtering Approach to Suggest Problems to Solve in Programming Online Judges

ERIC Educational Resources Information Center

Toledo, Raciel Yera; Mota, Yailé Caballero

2014-01-01

The paper proposes a recommender system approach to cover online judge's domains. Online judges are e-learning tools that support the automatic evaluation of programming tasks done by individual users, and for this reason they are usually used for training students in programming contest and for supporting basic programming teachings. The…

Bystander intervention and litter control: Evaluation of an appeal-to-help program.

Treesearch

Harriet H. Christensen

1981-01-01

Managers of public recreation areas are concerned about increases in vandalism and disregard for regulations and the rights of others by some users. Other than the litter incentive program, none of the approaches for reducing violations have been evaluated or proved effective. The present study evaluated a program to increase involvement of campers in the management of...

The Floating Lab Research Project: An Approach to Evaluating Field Programs.

ERIC Educational Resources Information Center

Brody, Michael J.

This report explains an evaluative study of the conceptual and affective development of students associated with the Floating Lab Program, an experiential field project sponsored by the University of New Hampshire and the Maine Sea Grant Program. The field program involved an opportunity for students to have hands-on experiences aboard a 65-foot…

A Comparison of Linear and Systems Thinking Approaches for Program Evaluation Illustrated Using the Indiana Interdisciplinary GK-12

ERIC Educational Resources Information Center

Dyehouse, Melissa; Bennett, Deborah; Harbor, Jon; Childress, Amy; Dark, Melissa

2009-01-01

Logic models are based on linear relationships between program resources, activities, and outcomes, and have been used widely to support both program development and evaluation. While useful in describing some programs, the linear nature of the logic model makes it difficult to capture the complex relationships within larger, multifaceted…

Applied Pluralism in the Evaluation of Employee Counselling.

ERIC Educational Resources Information Center

Goss, Stephen; Mearns, Dave

1997-01-01

Outlines the method, findings, and philosophical approach taken in a 22-month evaluation of an Employee Assistance Program. The program offered free counseling sessions, telephone support, in-service training, and conciliation work. Using an integrated pluralist evaluation, found that clients reported high satisfaction. Reduced absenteeism…

A Holistic Approach to Evaluating Vocational Education: Traditional Chinese Physicians (TCP) Model.

ERIC Educational Resources Information Center

Lee, Lung-Sheng; Chang, Liang-Te

Conventional approaches to evaluating vocational education have often been criticized for failing to deal holistically with the institution or program being evaluated. Integrated quantitative and qualitative evaluation methods have documented benefits; therefore, it would be useful to consider possibility of developing a model for evaluating…

Design Exception In-Service Monitoring Program Development

DOT National Transportation Integrated Search

2017-05-01

This study evaluates various possible program designs for in-service monitoring of design exceptions (DEs) for the Georgia Department of Transportation. The study recommends a multitier stepwise approach to the evaluation of DEs. Specifically, the pr...

Pre- and Post-Planned Evaluation: Which Is Preferable?

ERIC Educational Resources Information Center

Strasser, Stephen; Deniston, O. Lynn

1978-01-01

Factors involved in pre-planned and post-planned evaluation of program effectiveness are compared: (1) reliability and cost of data; (2) internal and external validity; (3) obtrusiveness and threat; (4) goal displacement and program direction. A model to help program administrators decide which approach is more appropriate is presented. (Author/MH)

Evaluation of Children's After-School Programs in Taiwan: FAHP Approach

ERIC Educational Resources Information Center

Lee, Amy H. I.; Yang, Chih-Neng; Lin, Chun-Yu

2012-01-01

The need of after-school programs has become urgent for school-age children in many industrialized countries due to social structure changes. This research develops a hierarchical framework to evaluate after-school programs from two distinct aspects--service quality from parents' perspectives and marketing strategy from operators'…

Supporting School Leaders in Blended Learning with Blended Learning

ERIC Educational Resources Information Center

Acree, Lauren; Gibson, Theresa; Mangum, Nancy; Wolf, Mary Ann; Kellogg, Shaun; Branon, Suzanne

2017-01-01

This study provides a mixed-methods case-study design evaluation of the Leadership in Blended Learning (LBL) program. The LBL program uses blended approaches, including face-to-face and online, to prepare school leaders to implement blended learning initiatives in their schools. This evaluation found that the program designers effectively…

Footprints: Strategies for Non-Traditional Program Evaluation.

ERIC Educational Resources Information Center

Frechtling, Joy A., Ed.

Papers in this collection explore alternative and nontraditional approaches to evaluation. They provide options, speculations, and propositions that affect each thinker's ideas on how to trace the impact of National Science Foundation-supported programs. Organized around a central theme of footprints as evidence of a program's impact, the papers…

The State of the Empirical Research Literature on Stakeholder Involvement in Program Evaluation

ERIC Educational Resources Information Center

Brandon, Paul R.; Fukunaga, Landry L.

2014-01-01

Evaluators widely agree that stakeholder involvement is a central aspect of effective program evaluation. With the exception of articles on collaborative evaluation approaches, however, a systematic review of the breadth and depth of the literature on stakeholder involvement has not been published. In this study, we examine peer-reviewed empirical…

Alternative management and funding options for aeronautics programs, Task 1

NASA Technical Reports Server (NTRS)

1975-01-01

Research and technology will be at lower program levels with basic military research for aviation decreasing as fewer aircraft programs are initiated and the present new aircraft programs move into the prototype and production status. The key question is can industry take on the management and financing role and meet the criteria and characteristics considered essential for a viable research and technology program. The criteria for evaluating alternative approaches include an examination of the nature of the product to be provided, responsiveness to changing needs, efficiency in terms of costs, ability to provide leadership, and to provide impartial and independent evaluation of approaches, and to provide technological inputs for regulating functions.

A participatory approach to evaluating a national training and institutional change initiative: the BUILD longitudinal evaluation.

PubMed

Davidson, Pamela L; Maccalla, Nicole M G; Afifi, Abdelmonem A; Guerrero, Lourdes; Nakazono, Terry T; Zhong, Shujin; Wallace, Steven P

2017-01-01

The National Institutes of Health (NIH) funds training programs to increase the numbers and skills of scientists who obtain NIH research grants, but few programs have been rigorously evaluated. The sizeable recent NIH investment in developing programs to increase the diversity of the NIH-funded workforce, implemented through the Diversity Program Consortium (DPC), is unusual in that it also funds a Consortium-wide evaluation plan, which spans the activities of the 10 BUilding Infrastructure Leading to Diversity (BUILD) awardees and the National Research Mentoring Network (NRMN). The purpose of this article is to describe the evaluation design and innovations of the BUILD Program on students, faculty, and institutions of the 10 primarily undergraduate BUILD sites. Our approach to this multi-methods quasi-experimental longitudinal evaluation emphasizes stakeholder participation and collaboration. The evaluation plan specifies the major evaluation questions and key short- to long-term outcome measures (or Hallmarks of Success). The Coordination and Evaluation Center (CEC) embarked on a comprehensive evaluation strategy by developing a set of logic models that incorporate the Hallmarks of Success and other outcomes that were collaboratively identified by the DPC. Data were collected from each BUILD site through national surveys from the Higher Education Research Institute at UCLA (HERI), annual followup surveys that align with the HERI instruments, site visits and case studies, program encounter data ("tracker" data), and institutional data. The analytic approach involves comparing changes in Hallmarks (key outcomes) within institutions for biomedical students who participated versus those who did not participate in the BUILD program at each institution, as well as between institution patterns of biomedical students at the BUILD sites, and matched institutions that were not BUILD grantees. Case studies provide insights into the institutionalization of these new programs and help to explain the processes that lead to the observed outcomes. Ultimately, the results of the consortium-wide evaluation will be used to inform national policy in higher education and will provide relevant examples of institutional and educational programmatic changes required to diversify the biomedical workforce in the USA.

Evaluating Program Effectiveness or, If the Program is Competency-Based, How Come the Evaluation Is Costing So Much?

ERIC Educational Resources Information Center

Scriven, Michael

The concern with competency-based education is not whether it is any different from the more traditional approaches but whether it is worth the considerable effort it involves. There are several aspects of a competency-based program to be considered in its evaluation. The first is whether or not there is a justifiable need for the specified…

Policy Forum Report: Statewide Evaluation of Programs and Services for Students with Disabilities (Arlington, Virginia, August 30-31, 1994). Final Report. Project FORUM.

ERIC Educational Resources Information Center

National Association of State Directors of Special Education, Alexandria, VA.

This report reviews the outcomes of a forum convened to examine policy and practice issues surrounding the annual evaluation of effectiveness of programs and services for students with disabilities. At the forum, alternative approaches being used by states to implement program evaluation were discussed, along with issues in gathering…

Evaluation Criteria for Micro-CAI: A Psychometric Approach

PubMed Central

Wallace, Douglas; Slichter, Mark; Bolwell, Christine

1985-01-01

The increased use of microcomputer-based instructional programs has resulted in a greater need for third-party evaluation of the software. This in turn has prompted the development of micro-CAI evaluation tools. The present project sought to develop a prototype instrument to assess the impact of CAI program presentation characteristics on students. Data analysis and scale construction was conducted using standard item reliability analyses and factor analytic techniques. Adequate subscale reliabilities and factor structures were found, suggesting that a psychometric approach to CAI evaluation may possess some merit. Efforts to assess the utility of the resultant instrument are currently underway.

Development and Evaluation of the Efficiency of In-Service Training Program with the Theme of Peace Education

ERIC Educational Resources Information Center

Basaran, Semra Demir; Karakurt, Sevgi Özden

2017-01-01

In this article, an in-service training program was developed and evaluated to improve the peace education competencies of primary school teachers. This program, named as In-Service Training Program with the Theme of Peace Education for Primary Teachers (BEHEP), was based on the system approach. The implementation was completed in 28 hours with…

Evaluating an Enrichment Program in Early Childhood: A Multi-Methods Approach

ERIC Educational Resources Information Center

van Aswegen, Christa; Pendergast, Donna

2015-01-01

This article reports on the evaluation of one topic in an enrichment program designed for children in their early years of learning. The program is responsive to an increased understanding of the benefits for very young children of programs that not only take advantage of the sensitive periods for learning but that also assist parents to a take a…

A merged model of quality improvement and evaluation: maximizing return on investment.

PubMed

Woodhouse, Lynn D; Toal, Russ; Nguyen, Trang; Keene, DeAnna; Gunn, Laura; Kellum, Andrea; Nelson, Gary; Charles, Simone; Tedders, Stuart; Williams, Natalie; Livingood, William C

2013-11-01

Quality improvement (QI) and evaluation are frequently considered to be alternative approaches for monitoring and assessing program implementation and impact. The emphasis on third-party evaluation, particularly associated with summative evaluation, and the grounding of evaluation in the social and behavioral science contrast with an emphasis on the integration of QI process within programs or organizations and its origins in management science and industrial engineering. Working with a major philanthropic organization in Georgia, we illustrate how a QI model is integrated with evaluation for five asthma prevention and control sites serving poor and underserved communities in rural and urban Georgia. A primary foundation of this merged model of QI and evaluation is a refocusing of the evaluation from an intimidating report card summative evaluation by external evaluators to an internally engaged program focus on developmental evaluation. The benefits of the merged model to both QI and evaluation are discussed. The use of evaluation based logic models can help anchor a QI program in evidence-based practice and provide linkage between process and outputs with the longer term distal outcomes. Merging the QI approach with evaluation has major advantages, particularly related to enhancing the funder's return on investment. We illustrate how a Plan-Do-Study-Act model of QI can (a) be integrated with evaluation based logic models, (b) help refocus emphasis from summative to developmental evaluation, (c) enhance program ownership and engagement in evaluation activities, and (d) increase the role of evaluators in providing technical assistance and support.

A quantitative approach to assessing the efficacy of occupant protection programs: A case study from Montana.

PubMed

Manlove, Kezia; Stanley, Laura; Peck, Alyssa

2015-10-01

Quantitative evaluation of vehicle occupant protection programs is critical for ensuring efficient government resource allocation, but few methods exist for conducting evaluation across multiple programs simultaneously. Here we present an analysis of occupant protection efficacy in the state of Montana. This approach relies on seat belt compliance rates as measured by the National Occupant Protection Usage Survey (NOPUS). A hierarchical logistic regression model is used to estimate the impacts of four Montana Department of Transportation (MDT)-funded occupant protection programs used in the state of Montana, following adjustment for a suite of potential confounders. Activity from two programs, Buckle Up coalitions and media campaigns, are associated with increased seat belt use in Montana, whereas the impact of another program, Selective Traffic Enforcement, is potentially masked by other program activity. A final program, Driver's Education, is not associated with any shift in seat belt use. This method allows for a preliminary quantitative estimation of program impacts without requiring states to obtain any new seat belt use data. This approach provides states a preliminary look at program impacts, and a means for carefully planning future program allocation and investigation. Copyright © 2015 Elsevier Ltd. All rights reserved.

What Theorists Say They Do: A Brief Description of Theorists' Approaches

ERIC Educational Resources Information Center

Christie, Christina A.; Azzam, Tarek

2005-01-01

The purpose of this issue of "New Directions for Evaluation" is to examine, comparatively, the practical application of theorists' approaches to evaluation by examining four evaluations of the same case. The thought is that when asked to evaluate the same program (holding the case constant), the practical distinctions between theorists' approaches…

Widening Our Evaluative Lenses of Formal, Facilitated, and Spontaneous Academic Development

ERIC Educational Resources Information Center

Hoessler, Carolyn; Godden, Lorraine; Hoessler, Brian

2015-01-01

As conceptions of academic development expand to encompass a more diverse and flexible set of supports, traditional approaches to evaluation, including impact studies of formal programming, become insufficient. A program may appear ineffective when evaluation ignores additional supports that interact to counteract implementation or alternatively…

A View of Current Evaluative Practices in Instrumental Music Teacher Education

ERIC Educational Resources Information Center

Peterson, Amber Dahlén

2014-01-01

The purpose of this study was to examine how instrumental music educator skills are being evaluated in current undergraduate programs. While accrediting organizations mandate certain elements of these programs, they provide limited guidance on what evaluative approaches should be used. Instrumental music teacher educators in the College Music…

Statistical Process Control in the Practice of Program Evaluation.

ERIC Educational Resources Information Center

Posavac, Emil J.

1995-01-01

A technique developed to monitor the quality of manufactured products, statistical process control (SPC), incorporates several features that may prove attractive to evaluators. This paper reviews the history of SPC, suggests how the approach can enrich program evaluation, and illustrates its use in a hospital-based example. (SLD)

A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming.

PubMed

Hansen, William B; Derzon, James H; Reese, Eric L

2014-06-01

We propose a method for creating groups against which outcomes of local pretest-posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness. © The Author(s) 2012.

Measuring the Impact of Employment-Related Social Programs. A Primer on the Evaluation of Employment and Training, Vocational Education, Vocational Rehabilitation, and Other Job-Oriented Programs.

ERIC Educational Resources Information Center

Borus, Michael E.

An approach and methodology for the systematic measurement of the impact of employment-related social programs is presented in this primer. Chapter 1 focuses on evaluation as the third step (the first two being planning and operation) in the process of program implementation. Chapter 2 examines the impacts of social programs. Topics include…

Right timing in formative program evaluation.

PubMed

Hall, Jori; Freeman, Melissa; Roulston, Kathy

2014-08-01

Since many educational researchers and program developers have limited knowledge of formative evaluation, formative data may be underutilized during the development and implementation of an educational program. The purpose of this article is to explain how participatory, responsive, educative, and qualitative approaches to formative evaluation can facilitate a partnership between evaluators and educational researchers and program managers to generate data useful to inform program implementation and improvement. This partnership is critical, we argue, because it enables an awareness of when to take appropriate action to ensure successful educational programs or "kairos". To illustrate, we use examples from our own evaluation work to highlight how formative evaluation may facilitate opportune moments to (1) define the substance and purpose of a program, (2) develop understanding and awareness of the cultural interpretations of program participants, and (3) show the relevance of stakeholder experiences to program goals. Copyright © 2014 Elsevier Ltd. All rights reserved.

Stirring up the Mud: Using a Community-Based Participatory Approach to Address Health Disparities through a Faith-Based Initiative

PubMed Central

Kaplan, Sue A.; Ruddock, Charmaine; Golub, Maxine; Davis, Joyce; Foley, Robert; Devia, Carlos; Rosen, Rosa; Berry, Carolyn; Barretto, Brenda; Carter, Toni; Irish-Spencer, Evalina; Marchena, Maria; Purcaro, Ellenrita; Calman, Neil

2011-01-01

This case study provides a mid-course assessment of the Bronx Health REACH faith-based initiative four years into its implementation. The study uses qualitative methods to identify lessons learned and to reflect on the benefits and challenges of using a community-based participatory approach for the development and evaluation of a faith-based program designed to address health disparities. Key findings concern the role of pastoral leadership, the importance of providing a religious context for health promotion and health equality messages, the challenges of creating a bilingual/bi-cultural program, and the need to provide management support to the lay program coordinators. The study also identifies lessons learned about community-based evaluation and the importance of addressing community concern about the balance between evaluation and program. Finally, the study identifies the challenges that lie ahead, including issues of program institution-alization and sustainability. PMID:20168022

Evaluation of an Innovative Post-Arrest Diversion Program: 12-Month Recidivism Analysis

ERIC Educational Resources Information Center

Dembo, Richard; Walters, Wansley; Wareham, Jennifer; Burgos, Catherimarty; Schmeidler, James; Hoge, Robert; Underwood, Lee

2008-01-01

We report the results of an evaluation study of a diversion program, operated by the Miami-Dade County Juvenile Assessment Center in Florida. The Post-Arrest Diversion (PAD) program represents an innovative approach to treatment and intervention within the juvenile justice system that utilizes standardized psychosocial risks and needs assessment…

The Classroom: Insights into Educational Evaluation in School Programs for Emotionally Disturbed Children and Youth.

ERIC Educational Resources Information Center

Friedman, Helen, Ed.

Presented are conference proceedings on evaluation of educational programs for emotionally disturbed children. Raphael F. Simches highlights past and future educational trends in programs for handicapped children. Fritz Redl stresses various motivations that may cause violent behavior. A systematic approach to classroom analysis via taxonomy of…

Design of CIAO, a research program to support the development of an integrated approach to prevent overweight and obesity in the Netherlands.

PubMed

van Koperen, Marije Tm; van der Kleij, Rianne Mjj; Renders, Carry Cm; Crone, Matty Mr; Hendriks, Anna-Marie Am; Jansen, Maria M; van de Gaar, Vivian Vm; Raat, Hein Jh; Ruiter, Emilie Elm; Molleman, Gerard Grm; Schuit, Jantine Aj; Seidell, Jacob Jc

2014-01-01

The aim of this paper is to describe the research aims, concepts and methods of the research Consortium Integrated Approach of Overweight (CIAO). CIAO is a concerted action of five Academic Collaborative Centres, local collaborations between academic institutions, regional public health services, local authorities and other relevant sectors in the Netherlands. Prior research revealed lacunas in knowledge of and skills related to five elements of the integrated approach of overweight prevention in children (based upon the French EPODE approach), namely political support, parental education, implementation, social marketing and evaluation. CIAO aims to gain theoretical and practical insight of these elements through five sub-studies and to develop, based on these data, a framework for monitoring and evaluation. For this research program, mixed methods are used in all the five sub-studies. First, problem specification through literature research and consultation of stakeholders, experts, health promotion specialists, parents and policy makers will be carried out. Based on this information, models, theoretical frameworks and practical instruments will be developed, tested and evaluated in the communities that implement the integrated approach to prevent overweight in children. Knowledge obtained from these studies and insights from experts and stakeholders will be combined to create an evaluation framework to evaluate the integrated approach at central, local and individual levels that will be applicable to daily practice. This innovative research program stimulates sub-studies to collaborate with local stakeholders and to share and integrate their knowledge, methodology and results. Therefore, the output of this program (both knowledge and practical tools) will be matched and form building blocks of a blueprint for a local evidence- and practice-based integrated approach towards prevention of overweight in children. The output will then support various communities to further optimize the implementation and subsequently the effects of this approach.

Evaluating Intervention Programs with a Pretest-Posttest Design: A Structural Equation Modeling Approach

PubMed Central

Alessandri, Guido; Zuffianò, Antonio; Perinelli, Enrico

2017-01-01

A common situation in the evaluation of intervention programs is the researcher's possibility to rely on two waves of data only (i.e., pretest and posttest), which profoundly impacts on his/her choice about the possible statistical analyses to be conducted. Indeed, the evaluation of intervention programs based on a pretest-posttest design has been usually carried out by using classic statistical tests, such as family-wise ANOVA analyses, which are strongly limited by exclusively analyzing the intervention effects at the group level. In this article, we showed how second order multiple group latent curve modeling (SO-MG-LCM) could represent a useful methodological tool to have a more realistic and informative assessment of intervention programs with two waves of data. We offered a practical step-by-step guide to properly implement this methodology, and we outlined the advantages of the LCM approach over classic ANOVA analyses. Furthermore, we also provided a real-data example by re-analyzing the implementation of the Young Prosocial Animation, a universal intervention program aimed at promoting prosociality among youth. In conclusion, albeit there are previous studies that pointed to the usefulness of MG-LCM to evaluate intervention programs (Muthén and Curran, 1997; Curran and Muthén, 1999), no previous study showed that it is possible to use this approach even in pretest-posttest (i.e., with only two time points) designs. Given the advantages of latent variable analyses in examining differences in interindividual and intraindividual changes (McArdle, 2009), the methodological and substantive implications of our proposed approach are discussed. PMID:28303110

Evaluating Intervention Programs with a Pretest-Posttest Design: A Structural Equation Modeling Approach.

PubMed

Alessandri, Guido; Zuffianò, Antonio; Perinelli, Enrico

2017-01-01

A common situation in the evaluation of intervention programs is the researcher's possibility to rely on two waves of data only (i.e., pretest and posttest), which profoundly impacts on his/her choice about the possible statistical analyses to be conducted. Indeed, the evaluation of intervention programs based on a pretest-posttest design has been usually carried out by using classic statistical tests, such as family-wise ANOVA analyses, which are strongly limited by exclusively analyzing the intervention effects at the group level. In this article, we showed how second order multiple group latent curve modeling (SO-MG-LCM) could represent a useful methodological tool to have a more realistic and informative assessment of intervention programs with two waves of data. We offered a practical step-by-step guide to properly implement this methodology, and we outlined the advantages of the LCM approach over classic ANOVA analyses. Furthermore, we also provided a real-data example by re-analyzing the implementation of the Young Prosocial Animation, a universal intervention program aimed at promoting prosociality among youth. In conclusion, albeit there are previous studies that pointed to the usefulness of MG-LCM to evaluate intervention programs (Muthén and Curran, 1997; Curran and Muthén, 1999), no previous study showed that it is possible to use this approach even in pretest-posttest (i.e., with only two time points) designs. Given the advantages of latent variable analyses in examining differences in interindividual and intraindividual changes (McArdle, 2009), the methodological and substantive implications of our proposed approach are discussed.

DECISION-MAKING ALIGNED WITH RAPID-CYCLE EVALUATION IN HEALTH CARE.

PubMed

Schneeweiss, Sebastian; Shrank, William H; Ruhl, Michael; Maclure, Malcolm

2015-01-01

Availability of real-time electronic healthcare data provides new opportunities for rapid-cycle evaluation (RCE) of health technologies, including healthcare delivery and payment programs. We aim to align decision-making processes with stages of RCE to optimize the usefulness and impact of rapid results. Rational decisions about program adoption depend on program effect size in relation to externalities, including implementation cost, sustainability, and likelihood of broad adoption. Drawing on case studies and experience from drug safety monitoring, we examine how decision makers have used scientific evidence on complex interventions in the past. We clarify how RCE alters the nature of policy decisions; develop the RAPID framework for synchronizing decision-maker activities with stages of RCE; and provide guidelines on evidence thresholds for incremental decision-making. In contrast to traditional evaluations, RCE provides early evidence on effectiveness and facilitates a stepped approach to decision making in expectation of future regularly updated evidence. RCE allows for identification of trends in adjusted effect size. It supports adapting a program in midstream in response to interim findings, or adapting the evaluation strategy to identify true improvements earlier. The 5-step RAPID approach that utilizes the cumulating evidence of program effectiveness over time could increase policy-makers' confidence in expediting decisions. RCE enables a step-wise approach to HTA decision-making, based on gradually emerging evidence, reducing delays in decision-making processes after traditional one-time evaluations.

A new evaluation tool to obtain practice-based evidence of worksite health promotion programs.

PubMed

Dunet, Diane O; Sparling, Phillip B; Hersey, James; Williams-Piehota, Pamela; Hill, Mary D; Hanssen, Carl; Lawrenz, Frances; Reyes, Michele

2008-10-01

The Centers for Disease Control and Prevention developed the Swift Worksite Assessment and Translation (SWAT) evaluation method to identify promising practices in worksite health promotion programs. The new method complements research studies and evaluation studies of evidence-based practices that promote healthy weight in working adults. We used nationally recognized program evaluation standards of utility, feasibility, accuracy, and propriety as the foundation for our 5-step method: 1) site identification and selection, 2) site visit, 3) post-visit evaluation of promising practices, 4) evaluation capacity building, and 5) translation and dissemination. An independent, outside evaluation team conducted process and summative evaluations of SWAT to determine its efficacy in providing accurate, useful information and its compliance with evaluation standards. The SWAT evaluation approach is feasible in small and medium-sized workplace settings. The independent evaluation team judged SWAT favorably as an evaluation method, noting among its strengths its systematic and detailed procedures and service orientation. Experts in worksite health promotion evaluation concluded that the data obtained by using this evaluation method were sufficient to allow them to make judgments about promising practices. SWAT is a useful, business-friendly approach to systematic, yet rapid, evaluation that comports with program evaluation standards. The method provides a new tool to obtain practice-based evidence of worksite health promotion programs that help prevent obesity and, more broadly, may advance public health goals for chronic disease prevention and health promotion.

Can Rational Prescribing Be Improved by an Outcome-Based Educational Approach? A Randomized Trial Completed in Iran

ERIC Educational Resources Information Center

Esmaily, Hamideh M.; Silver, Ivan; Shiva, Shadi; Gargani, Alireza; Maleki-Dizaji, Nasrin; Al-Maniri, Abdullah; Wahlstrom, Rolf

2010-01-01

Introduction: An outcome-based education approach has been proposed to develop more effective continuing medical education (CME) programs. We have used this approach in developing an outcome-based educational intervention for general physicians working in primary care (GPs) and evaluated its effectiveness compared with a concurrent CME program in…

Measuring Cooperative Biological Engagement Program (CBEP) Performance: Capacities, Capabilities, and Sustainability Enablers for Biorisk Management and Biosurveillance

DTIC Science & Technology

2014-01-01

valid OMB control number. 1. REPORT DATE 2014 2. REPORT TYPE 3. DATES COVERED 00-00-2014 to 00-00-2014 4. TITLE AND SUBTITLE Measuring...should approaches to monitoring program performance. Recognizing this, Congress requested that the Department of Defense improve metrics for measuring...Cooperative Biological Engagement Program Performance broader community of program evaluation practitioners, the work advances innovative approaches

Opening minds in Canada: background and rationale.

PubMed

Stuart, Heather; Chen, Shu-Ping; Christie, Romie; Dobson, Keith; Kirsh, Bonnie; Knaak, Stephanie; Koller, Michelle; Krupa, Terry; Lauria-Horner, Bianca; Luong, Dorothy; Modgill, Geeta; Patten, Scott B; Pietrus, Mike; Szeto, Andrew; Whitley, Rob

2014-10-01

To summarize the background and rationale of the approach taken by the Mental Health Commission of Canada's Opening Minds (OM) Anti-Stigma Initiative. The approach taken by OM incorporates a grassroots, community development philosophy, has clearly defined target groups, uses contact-based education as the central organizing element across interventions, and has a strong evaluative component, so that best practices can be identified, replicated, and disseminated. Contact-based education occurs when people who have experienced a mental illness share their personal story of recovery and hope. OM has acted as a catalyst to develop partnerships between community groups who are undertaking anti-stigma work and an interdisciplinary team of academic researchers in 5 universities who are evaluating the results of these programs. Building partnerships with existing community programs and promoting systematic evaluation using standardized approaches and instruments have contributed to our understanding of best practices in the field of anti-stigma programming.

Integrating quantitative and qualitative evaluation methods to compare two teacher inservice training programs

NASA Astrophysics Data System (ADS)

Lawrenz, Frances; McCreath, Heather

Qualitative and quantitative evaluation procedures were used to compare two physical-science teacher inservice training programs. The two programs followed the master teacher training model espoused by NSF but used different types of master teachers and types of activities. The two evaluation procedures produced different results and together they provided a much clearer picture of the strengths and weaknesses of the two programs. Using only one approach or the other would have substantially altered the conclusions.

The New Accreditation Council for Graduate Medical Education Next Accreditation System Milestones Evaluation System: What Is Expected and How Are Plastic Surgery Residency Programs Preparing?

PubMed

Sillah, Nyama M; Ibrahim, Ahmed M S; Lau, Frank H; Shah, Jinesh; Medin, Caroline; Lee, Bernard T; Lin, Samuel J

2015-07-01

The Accreditation Council for Graduate Medical Education Next Accreditation System milestones were implemented for plastic surgery programs in July of 2014. Forward progress through the milestones is an indicator of trainee-appropriate development, whereas regression or stalling may indicate the need for concentrated, targeted training. Online software at www.surveymonkey.com was used to create a survey about the program's approaches to milestones and was distributed to program directors and administrators of 96 Accreditation Council for Graduate Medical Education-approved plastic surgery programs. The authors had a 63.5 percent response rate (61 of 96 plastic surgery programs). Most programs report some level of readiness, only 22 percent feel completely prepared for the Next Accreditation System milestones, and only 23 percent are completely satisfied with their planned approach for compliance. Seventy-five percent of programs claim to be using some form of electronic tracking system. Programs plan to use multiple tools to capture and report milestone data. Most programs (44.4 percent) plan to administer evaluations at the end of each rotation. Over 70 percent of respondents believe that the milestones approach would improve the quality of resident training. However, programs were less than confident that their current compliance systems would live up to their full potential. The Next Accreditation System has been implemented nationwide for plastic surgery training programs. Milestone-based resident training is a new paradigm for residency training evaluation; programs are in the process of making this transition to find ways to make milestone data meaningful for faculty and residents.

Using multiple methods to assess learning and outcomes in an online degree-granting dental hygiene program.

PubMed

Springfield, Emily; Gwozdek, Anne E; Peet, Melissa; Kerschbaum, Wendy E

2012-04-01

Program evaluation is a necessary component of curricular change and innovation. It ascertains whether an innovation has met benchmarks and contributes to the body of knowledge about educational methodologies and supports the use of evidence-based practice in teaching. Education researchers argue that rigorous program evaluation should utilize a mixed-method approach, triangulating both qualitative and quantitative methods to understand program effectiveness. This approach was used to evaluate the University of Michigan Dental Hygiene Degree Completion E-Learning (online) Program. Quantitative data included time spent on coursework, grades, publications, course evaluation results, and survey responses. Qualitative data included student and faculty responses in focus groups and on surveys as well as students' portfolio reflections. The results showed the program was academically rigorous, fostering students' ability to connect theory with practice and apply evidence-based practice principles. These results also demonstrated that the students had learned to critically reflect on their practice and develop expanded professional identities; going beyond the role of clinician, they began to see themselves as educators, advocates, and researchers. This evaluation model is easily adaptable and is applicable to any health science or other professional degree program. This study also raised important questions regarding the effect of meta-reflection on student confidence and professional behavior.

Evaluating the Impact of the U.S. National Toxicology Program: A Case Study on Hexavalent Chromium.

PubMed

Xie, Yun; Holmgren, Stephanie; Andrews, Danica M K; Wolfe, Mary S

2017-02-01

Evaluating the impact of federally funded research with a broad, methodical, and objective approach is important to ensure that public funds advance the mission of federal agencies. We aimed to develop a methodical approach that would yield a broad assessment of National Toxicology Program's (NTP's) effectiveness across multiple sectors and demonstrate the utility of the approach through a case study. A conceptual model was developed with defined activities, outputs (products), and outcomes (proximal, intermediate, distal) and applied retrospectively to NTP's research on hexavalent chromium (CrVI). Proximal outcomes were measured by counting views of and requests for NTP's products by external stakeholders. Intermediate outcomes were measured by bibliometric analysis. Distal outcomes were assessed through Web and LexisNexis searches for documents related to legislation or regulation changes. The approach identified awareness of NTP's work on CrVI by external stakeholders (proximal outcome) and citations of NTP's research in scientific publications, reports, congressional testimonies, and legal and policy documents (intermediate outcome). NTP's research was key to the nation's first-ever drinking water standard for CrVI adopted by California in 2014 (distal outcome). By applying this approach to a case study, the utility and limitations of the approach were identified, including challenges to evaluating the outcomes of a research program. This study identified a broad and objective approach for assessing NTP's effectiveness, including methodological needs for more thorough and efficient impact assessments in the future. Citation: Xie Y, Holmgren S, Andrews DMK, Wolfe MS. 2017. Evaluating the impact of the U.S. National Toxicology Program: a case study on hexavalent chromium. Environ Health Perspect 125:181-188; http://dx.doi.org/10.1289/EHP21.

Teaching Note--Keeping It Real: Program Evaluation Projects for an Undergraduate Research Class

ERIC Educational Resources Information Center

John, Aesha; Bang, Eun-Jun

2017-01-01

This article describes a teaching innovation that focused on the redesign of an undergraduate social work research class. Students enrolled in the redesigned class had an opportunity to conduct program evaluation projects in community agencies. The projects included (a) pretest and posttest evaluation of reminiscence approach in improving the…

The Parent-Child Home Program in Western Manitoba: A 20-Year Evaluation

ERIC Educational Resources Information Center

Gfellner, Barbara M.; McLaren, Lorraine; Metcalfe, Arron

2008-01-01

This article is a 20-year evaluation of the Parent-Child Home Program (PCHP) of Child and Family Services in Western Manitoba. Following Levenstein's (1979, 1988) approach, home visitors model parent-child interchanges using books and toys to enhance children's cognitive development through appropriate parenting behaviors. The evaluation provides…

Stepping Stones to Evaluating Your Own School Literacy Program

ERIC Educational Resources Information Center

Levesque, Jeri; Carnahan, Danielle

2005-01-01

Stepping Stones to Literacy is a tool for elementary school improvement teams to evaluate and strengthen their reading programs. Each Stepping Stone is a guided activity to stimulate reflection and guide systematic inquiry. It is a collaborative, active research approach to evaluation (Levesque & Hinton 2001). The goal is to eliminate the gap…

Rationale and Use of Content-Relevant Achievement Tests for the Evaluation of Instructional Programs.

ERIC Educational Resources Information Center

Patalino, Marianne

Problems in current course evaluation methods are discussed and an alternative method is described for the construction, analysis, and interpretation of a test to evaluate instructional programs. The method presented represents a different approach to the traditional overreliance on standardized achievement tests and the total scores they provide.…

Improving Human Services: Methods for Systematic Planning, Evaluation, and Staff Development.

ERIC Educational Resources Information Center

Jones, G. Brian; And Others

This monograph speaks to the basic problems confronting human services programs, and offers suggestions for improving guidance services. Specific concerns addressed are: (1) the need for more systematic program planning and evaluation; (2) an approach to effective planning and evaluation; (3) development of staff skills for use in the…

An Appreciative Inquiry Approach to Evaluating Culture, Structure, and Power in Agricultural Teacher Education Program Reform

ERIC Educational Resources Information Center

Anderson, James C., II.; Thorson, Candi J.; Kelinsky, Lia R.

2016-01-01

This case study outlines an appreciative inquiry approach to program reform using an agricultural teacher education program at a land-grant university that had begun to suffer from a large decline in student enrollment. Documents were analyzed that provided recommendations toward a master plan for reform made by 23 key agents based on their…

Use of net present value analysis to evaluate a publicly funded biomass-to-ethanol research, development, and demonstration program and valuate expected private sector participation.

PubMed

Hinman, N D; Yancey, M A

1998-01-01

One of the functions of government is to invest tax dollars in programs, projects, and properties that will result in greater public benefit than would have resulted from leaving the tax dollars in the private sector or using them to pay off the public debt. This paper describes the use of Net Present Value (NPV) as an approach to analyze and select investment opportunities for government money in public research, development, and demonstration (RD&D) programs and to evaluate potential private sector participation in the programs. This approach is then applied to a specific biomass-to-ethanol opportunity in California.

Use of a business excellence model to improve conservation programs.

PubMed

Black, Simon; Groombridge, Jim

2010-12-01

The current shortfall in effectiveness within conservation biology is illustrated by increasing interest in "evidence-based conservation," whose proponents have identified the need to benchmark conservation initiatives against actions that lead to proven positive effects. The effectiveness of conservation policies, approaches, and evaluation is under increasing scrutiny, and in these areas models of excellence used in business could prove valuable. Typically, conservation programs require years of effort and involve rigorous long-term implementation processes. Successful balance of long-term efforts alongside the achievement of short-term goals is often compromised by management or budgetary constraints, a situation also common in commercial businesses. "Business excellence" is an approach many companies have used over the past 20 years to ensure continued success. Various business excellence evaluations have been promoted that include concepts that could be adapted and applied in conservation programs. We describe a conservation excellence model that shows how scientific processes and results can be aligned with financial and organizational measures of success. We applied the model to two well-documented species conservation programs. In the first, the Po'ouli program, several aspects of improvement were identified, such as more authority for decision making in the field and better integration of habitat management and population recovery processes. The second example, the black-footed ferret program, could have benefited from leadership effort to reduce bureaucracy and to encourage use of best-practice species recovery approaches. The conservation excellence model enables greater clarity in goal setting, more-effective identification of job roles within programs, better links between technical approaches and measures of biological success, and more-effective use of resources. The model could improve evaluation of a conservation program's effectiveness and may be used to compare different programs, for example during reviews of project performance by sponsoring organizations. © 2010 Society for Conservation Biology.

Demonstrating Impact through Replicable Analysis: Implications of an Evaluation of Arkansas's Expanded Food and Nutrition Education Program

ERIC Educational Resources Information Center

Phelps, Josh; Brite-Lane, Allison; Crook, Tina; Hakkak, Reza; Fuller, Serena

2017-01-01

The evaluation described in this article focused on the effectiveness of Arkansas's Extension-based Expanded Food and Nutrition Education Program (EFNEP) but demonstrates an analytic approach that may be useful across Extension programs. We analyzed data from 1,810 Arkansas EFNEP participants' entry and exit Behavior Checklists to assess…

Evaluation of College Reading and Study Skills Programs; Problems and Approaches.

ERIC Educational Resources Information Center

Anderson, William W.

The purpose of this paper was to bring together some of the most outstanding and significant research in recent years relating to the problems of evaluating college reading and study skills programs and the corresponding prospects and implications for improvement. The sections of the paper include: "Criterion Measures of Program Success," which is…

Solar energy program evaluation: an introduction

DOE Office of Scientific and Technical Information (OSTI.GOV)

deLeon, P.

The Program Evaluation Methodology provides an overview of the practice and methodology of program evaluation and defines more precisely the evaluation techniques and methodologies that would be most appropriate to government organizations which are actively involved in the research, development, and commercialization of solar energy systems. Formal evaluation cannot be treated as a single methodological approach for assessing a program. There are four basic types of evaluation designs - the pre-experimental design; the quasi-experimental design based on time series; the quasi-experimental design based on comparison groups; and the true experimental design. This report is organized to first introduce the rolemore » and issues of evaluation. This is to provide a set of issues to organize the subsequent sections detailing the national solar energy programs. Then, these two themes are integrated by examining the evaluation strategies and methodologies tailored to fit the particular needs of the various individual solar energy programs. (MCW)« less

Evaluating Integrative Cancer Clinics With the Claim Assessment Profile: An Example With the InspireHealth Clinic.

PubMed

Hilton, Lara; Elfenbaum, Pamela; Jain, Shamini; Sprengel, Meredith; Jonas, Wayne B

2018-03-01

The evaluation of freestanding integrative cancer clinical programs is challenging and is rarely done. We have developed an approach called the Claim Assessment Profile (CAP) to identify whether evaluation of a practice is justified, feasible, and likely to provide useful information. A CAP was performed in order to (1) clarify the healing claims at InspireHealth, an integrative oncology treatment program, by defining the most important impacts on its clients; (2) gather information about current research capacity at the clinic; and (3) create a program theory and path model for use in prospective research. This case study design incorporates methods from a variety of rapid assessment approaches. Procedures included site visits to observe the program, structured qualitative interviews with 26 providers and staff, surveys to capture descriptive data about the program, and observational data on program implementation. The InspireHealth program is a well-established, multi-site, thriving integrative oncology clinical practice that focuses on patient support, motivation, and health behavior engagement. It delivers patient-centered care via a standardized treatment protocol. There arehigh levels of research interest from staff and resources by which to conduct research. This analysis provides the primary descriptive and claims clarification of an integrative oncology treatment program, an evaluation readiness report, a detailed logic model explicating program theory, and a clinical outcomes path model for conducting prospective research. Prospective evaluation of this program would be feasible and valuable, adding to our knowledge base of integrative cancer therapies.

Continuous Evaluation in Ethics Education: A Case Study.

PubMed

McIntosh, Tristan; Higgs, Cory; Mumford, Michael; Connelly, Shane; DuBois, James

2018-04-01

A great need for systematic evaluation of ethics training programs exists. Those tasked with developing an ethics training program may be quick to dismiss the value of training evaluation in continuous process improvement. In the present effort, we use a case study approach to delineate how to leverage formative and summative evaluation measures to create a high-quality ethics education program. With regard to formative evaluation, information bearing on trainee reactions, qualitative data from the comments of trainees, in addition to empirical findings, can ensure that the training program operates smoothly. Regarding summative evaluation, measures examining trainee cognition, behavior, and organization-level results provide information about how much trainees have changed as a result of taking the ethics training. The implications of effective training program evaluation are discussed.

Multi-criteria group decision making for evaluating the performance of e-waste recycling programs under uncertainty.

PubMed

Wibowo, Santoso; Deng, Hepu

2015-06-01

This paper presents a multi-criteria group decision making approach for effectively evaluating the performance of e-waste recycling programs under uncertainty in an organization. Intuitionistic fuzzy numbers are used for adequately representing the subjective and imprecise assessments of the decision makers in evaluating the relative importance of evaluation criteria and the performance of individual e-waste recycling programs with respect to individual criteria in a given situation. An interactive fuzzy multi-criteria decision making algorithm is developed for facilitating consensus building in a group decision making environment to ensure that all the interest of individual decision makers have been appropriately considered in evaluating alternative e-waste recycling programs with respect to their corporate sustainability performance. The developed algorithm is then incorporated into a multi-criteria decision support system for making the overall performance evaluation process effectively and simple to use. Such a multi-criteria decision making system adequately provides organizations with a proactive mechanism for incorporating the concept of corporate sustainability into their regular planning decisions and business practices. An example is presented for demonstrating the applicability of the proposed approach in evaluating the performance of e-waste recycling programs in organizations. Copyright © 2015 Elsevier Ltd. All rights reserved.

An Ex-Post Facto Examination of Relationships among the Developmental Designs Professional Development Model/Classroom Management Approach, School Leadership, Climate, Student Achievement, Attendance, and Behavior in High Poverty, Middle Grades Schools

ERIC Educational Resources Information Center

Hough, David L.; Schmitt, Vicki L.

2011-01-01

This study reports finding from an ex post facto causal-comparison study utilizing data from a multifaceted program evaluation of a professional development approach to classroom management known as Development Designs 1 and Developmental Designs 2 (DD1 & D2). Data from this program evaluation indicate that teachers implement a number of classroom…

Using baldrige performance excellence program approaches in the pursuit of radiation oncology quality care, patient satisfaction, and workforce commitment.

PubMed

Sternick, Edward S

2011-01-01

The Malcolm Baldrige National Quality Improvement Act was signed into law in 1987 to advance US business competitiveness and economic growth. Administered by the National Institute of Standards and Technology, the Act created the Baldrige National Quality Program, recently renamed the Baldrige Performance Excellence Program. The comprehensive analytical approaches referred to as the Baldrige Healthcare Criteria, are very well-suited for the evaluation and sustainable improvement of radiation oncology management and operations. A multidisciplinary self-assessment approach is used for radiotherapy program evaluation and development in order to generate a fact-based, knowledge-driven system for improving quality of care, increasing patient satisfaction, enhancing leadership effectiveness, building employee engagement, and boosting organizational innovation. This methodology also provides a valuable framework for benchmarking an individual radiation oncology practice's operations and results against guidelines defined by accreditation and professional organizations and regulatory agencies.

Reducing Disparities by way of a Cancer Disparities Research Training Program.

PubMed

Caplan, Lee S; Akintobi, Tabia H; Gordon, Tandeca King; Zellner, Tiffany; Smith, Selina A; Blumenthal, Daniel S

2016-01-01

For minority populations, there is a continuing disparity in the burden of death and illness from cancer. Research to address this disparity should be conducted by investigators who can best understand and address the needs of culturally diverse communities. However, minorities are under-represented in health-related research. The goal of this project was to develop and evaluate an approach to motivating and preparing master's degree students for careers dedicated to cancer disparities research. A Cancer Disparities Research Training Program (CDRTP) was initiated in 2010. The program consists of coursework, practicum experiences, and research opportunities. Assessment of the curriculum is based on monitoring achievement of evaluation indicators and included a mixed-method approach with included both quantitative and qualitative approach. In its first three years, the program graduated 20 trainees, all of whom were minorities (18 African Americans and two Asians). When asked about career goals, two-thirds of the trainees indicated interest in pursuing careers in research in cancer prevention and control. The trainees expressed high satisfaction with the courses, instructor, materials, and curriculum. Although trainees had suggestions about course details, evaluations overall were positive. Across focus groups, three recurrent themes emerged regarding activities to enhance the trainee experience: having a wider variety of topics, more guest speakers, and field trips. The CDRTP was intended to recruit students - primarily African Americans - into research on prevention and control of cancer disparities. Although final evaluation of the program's overall outcome will not be available for several years, this preliminary evaluation indicates early program success.

Engaging partners to initiate evaluation efforts: tactics used and lessons learned from the prevention research centers program.

PubMed

Wright, Demia Sundra; Anderson, Lynda A; Brownson, Ross C; Gwaltney, Margaret K; Scherer, Jennifer; Cross, Alan W; Goodman, Robert M; Schwartz, Randy; Sims, Tom; White, Carol R

2008-01-01

The Centers for Disease Control and Prevention's (CDC's) Prevention Research Centers (PRC) Program underwent a 2-year evaluation planning project using a participatory process that allowed perspectives from the national community of PRC partners to be expressed and reflected in a national logic model. The PRC Program recognized the challenge in developing a feasible, useable, and relevant evaluation process for a large, diverse program. To address the challenge, participatory and utilization-focused evaluation models were used. Four tactics guided the evaluation planning process: 1) assessing stakeholders' communication needs and existing communication mechanisms and infrastructure; 2) using existing mechanisms and establishing others as needed to inform, educate, and request feedback; 3) listening to and using feedback received; and 4) obtaining adequate resources and building flexibility into the project plan to support multifaceted mechanisms for data collection. Participatory methods resulted in buy-in from stakeholders and the development of a national logic model. Benefits included CDC's use of the logic model for program planning and development of a national evaluation protocol and increased expectations among PRC partners for involvement. Challenges included the time, effort, and investment of program resources required for the participatory approach and the identification of whom to engage and when to engage them for feedback on project decisions. By using a participatory and utilization-focused model, program partners positively influenced how CDC developed an evaluation plan. The tactics we used can guide the involvement of program stakeholders and help with decisions on appropriate methods and approaches for engaging partners.

Evaluation and communication: using a communication audit to evaluate organizational communication.

PubMed

Hogard, Elaine; Ellis, Roger

2006-04-01

This article identifies a surprising dearth of studies that explicitly link communication and evaluation at substantive, theoretical, and methodological levels. A three-fold typology of evaluation studies referring to communication is proposed and examples given. The importance of organizational communication in program delivery is stressed and illustrative studies reviewed. It is proposed that organizational communication should be considered in all program evaluations and that this should be approached through communication audit. Communication audits are described with particular reference to established survey questionnaire instruments. Two case studies exemplify the use of such instruments in the evaluation of educational and social programs.

Entrepreneurship education: A strength-based approach to substance use and suicide prevention for American Indian adolescents.

PubMed

Tingey, Lauren; Larzelere-Hinton, Francene; Goklish, Novalene; Ingalls, Allison; Craft, Todd; Sprengeler, Feather; McGuire, Courtney; Barlow, Allison

2016-01-01

American Indian (AI) adolescents suffer the largest disparities in substance use and suicide. Predominating prevention models focus primarily on risk and utilize deficit-based approaches. The fields of substance use and suicide prevention research urge for positive youth development frameworks that are strength based and target change at individual and community levels. Entrepreneurship education is an innovative approach that reflects the gap in available programs. This paper describes the development and evaluation of a youth entrepreneurship education program in partnership with one AI community. We detail the curriculum, process evaluation results, and the randomized controlled trial evaluating its efficacy for increasing protective factors. Lessons learned may be applicable to other AI communities.

Factors Affecting Regression-Discontinuity.

ERIC Educational Resources Information Center

Schumacker, Randall E.

The regression-discontinuity approach to evaluating educational programs is reviewed, and regression-discontinuity post-program mean differences under various conditions are discussed. The regression-discontinuity design is used to determine whether post-program differences exist between an experimental program and a control group. The difference…

Agent-based modeling as a tool for program design and evaluation.

PubMed

Lawlor, Jennifer A; McGirr, Sara

2017-12-01

Recently, systems thinking and systems science approaches have gained popularity in the field of evaluation; however, there has been relatively little exploration of how evaluators could use quantitative tools to assist in the implementation of systems approaches therein. The purpose of this paper is to explore potential uses of one such quantitative tool, agent-based modeling, in evaluation practice. To this end, we define agent-based modeling and offer potential uses for it in typical evaluation activities, including: engaging stakeholders, selecting an intervention, modeling program theory, setting performance targets, and interpreting evaluation results. We provide demonstrative examples from published agent-based modeling efforts both inside and outside the field of evaluation for each of the evaluative activities discussed. We further describe potential pitfalls of this tool and offer cautions for evaluators who may chose to implement it in their practice. Finally, the article concludes with a discussion of the future of agent-based modeling in evaluation practice and a call for more formal exploration of this tool as well as other approaches to simulation modeling in the field. Copyright © 2017 Elsevier Ltd. All rights reserved.

A review of elementary school-based substance use prevention programs: identifying program attributes.

PubMed

Hopfer, S; Davis, D; Kam, J A; Shin, Y; Elek, E; Hecht, M L

2010-01-01

This article takes a systematic approach to reviewing substance use prevention programs introduced in elementary school (K-6th grade). Previous studies evaluating such programs among elementary school students showed mixed effects on subsequent substance use and related psychosocial factors. Thirty published evaluation studies of 24 elementary school-based substance use prevention programs were reviewed. The study selection criteria included searching for program evaluations from 1980 to 2008. Among 27 evaluation studies that examined program effects on substance use, 56% (n = 15) found significant decreases. In addition, programs most often demonstrated effects on increasing negative substance use attitudes, increasing knowledge, decreasing perceptions of prevalence rates (i.e., descriptive norms), and improving resistance skills. These results have implications for the appropriateness and value of introducing substance use prevention programs to youth in elementary school.

Curriculum Development and Evaluation for a Cancer Education Program.

ERIC Educational Resources Information Center

Deslauriers, Marc P.

1980-01-01

The Mid-America Cancer Center Program has developed a comprehensive approach for evaluating the cancer education curriculum at the University of Kansas Medical Center. The project included a review of all cancer-related teaching objectives and the development of an interdepartmental oncology curriculum. (JMD)

INTEGRATION OF SPATIAL DATA: METHODS EVALUATION WITH REGARD TO DATA ISSUES AND ASSESSMENT QUESTIONS

EPA Science Inventory

EPA's Regional Vulnerability Assessment (REVA) Program is developing and demonstrating approaches to assess current and future environmental vulnerabilities at a regional scale. An initial effort within this research program has been to develop and evaluate methods to synthesize ...

Housing First and Photovoice: Transforming Lives, Communities, and Systems

PubMed Central

Barile, John P.; Ogawa, Terry Yasuko; Peralta, Nelson; Bugg, Reumell; Lau, John; Lamberton, Thomas; Hall, Corazon; Mori, Victor

2018-01-01

This article presents findings from a community-based participatory evaluation of a Housing First program on the Island of O’ahu. In this study, clients in a Housing First program used Photovoice to evaluate the program and to advocate for progressive housing policies. Written together by members of the Housing First Photovoice group, this collaborative article describes the outcomes from both the Housing First program and the Photovoice project and demonstrates the ways in which participatory program evaluations can interact with client-driven programs like Housing First to produce a cumulative, transformative impact. Findings suggest that community psychologists hoping to re-engage with community mental health systems through enacting transformative change should consider taking a community-based participatory approach to program evaluation because increased client voice in community mental health programs and their evaluations can have far-reaching, transformative impacts for research, practice, and policy. PMID:29323410

Maximizing a Multilevel Mentoring Model to Improve Research Experiences for Undergraduates Student Outcomes

NASA Astrophysics Data System (ADS)

Moser, F. C.; Allen, M. R.; Clark, J.

2016-12-01

Since 1989 the Maryland Sea Grant REU program's mentoring approach evolved considerably from a near `hands-off' approach to one that explores ways to maximize mentoring effectiveness. Our current model creates a multi-tiered system: the REU - research mentor relationship remains central to setting the student's science project, but greater student growth is supported by REU program leaders, visiting researchers, graduate students, and peer and near-peer mentors. Evaluation of our evolving mentoring program suggests our approach is successful and serves a diversity of students well, but we recognize the challenge of devising an evaluation system, given our limited annual cohort number (15 - 17 students), that fully captures the nuances of student - mentor relationships. We present multiple years of data on student skills, networks of relationships, student goals, and mentor goals using qualitative, quantitative and interview assessments. Further, we explore opportunities to strengthen our efforts and evolve our evaluation approach as we aspire to more accurately identify the components of our multilevel mentoring model that contribute most significantly to student success.

What Works for Whom? A Bayesian Approach to Channeling Big Data Streams for Public Program Evaluation

ERIC Educational Resources Information Center

Finucane, Mariel McKenzie; Martinez, Ignacio; Cody, Scott

2018-01-01

In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data…

Approaches to Work-Integrated Learning and Engaging Industry in Vocational ICT Courses: Evaluation of an Australian Pilot Program

ERIC Educational Resources Information Center

Armatas, Christine; Papadopoulos, Theo

2013-01-01

Evaluation of a government-sponsored program for promoting work-integrated learning (WIL) in information communication technology (ICT) courses offered to vocational education students is discussed in this paper. The program provided the opportunity to incorporate WIL in the curriculum which had not previously been a feature of these ICT courses.…

An Approach to Evaluation of Rural Development Act Title V Programs. Northeast Regional Center for Rural Development Publication 11, February 1976.

ERIC Educational Resources Information Center

Miller, Robert W.

Presenting suggestions for conceptualizing Rural Development Act Title V program evaluations, this paper focuses upon a five-level program outcomes model which encompasses a causal sequence of anticipated shortrun, intermediate, and longrun outcomes. Examples of possible Title V measurement foci are presented for each of five outcome levels as…

Interfacing theories of program with theories of evaluation for advancing evaluation practice: Reductionism, systems thinking, and pragmatic synthesis.

PubMed

Chen, Huey T

2016-12-01

Theories of program and theories of evaluation form the foundation of program evaluation theories. Theories of program reflect assumptions on how to conceptualize an intervention program for evaluation purposes, while theories of evaluation reflect assumptions on how to design useful evaluation. These two types of theories are related, but often discussed separately. This paper attempts to use three theoretical perspectives (reductionism, systems thinking, and pragmatic synthesis) to interface them and discuss the implications for evaluation practice. Reductionism proposes that an intervention program can be broken into crucial components for rigorous analyses; systems thinking view an intervention program as dynamic and complex, requiring a holistic examination. In spite of their contributions, reductionism and systems thinking represent the extreme ends of a theoretical spectrum; many real-world programs, however, may fall in the middle. Pragmatic synthesis is being developed to serve these moderate- complexity programs. These three theoretical perspectives have their own strengths and challenges. Knowledge on these three perspectives and their evaluation implications can provide a better guide for designing fruitful evaluations, improving the quality of evaluation practice, informing potential areas for developing cutting-edge evaluation approaches, and contributing to advancing program evaluation toward a mature applied science. Copyright © 2016 Elsevier Ltd. All rights reserved.

McGraw-Hill Programmed Reading Evaluation.

ERIC Educational Resources Information Center

Norfolk Public Schools, VA.

One of the programs included in "Effective Reading Programs...," this program, begun in 1972, serves about 3,600 disadvantaged first through fourth grade students. The independent learning approach is featured in this program. Placement tests help teachers determine each child's needs, and diagnostic tests monitor student progress.…

The Integration of a Family Systems Approach for Understanding Youth Obesity, Physical Activity, and Dietary Programs

PubMed Central

Wilson, Dawn K.; St. George, Sara M.; Lawman, Hannah; Segal, Michelle; Fairchild, Amanda

2012-01-01

Rates of overweight in youth have reached epidemic proportions and are associated with adverse health outcomes. Family-based programs have been widely used to treat overweight in youth. However, few programs incorporate a theoretical framework for studying a family systems approach in relation to youth health behavior change. Therefore, this review provides a family systems theory framework for evaluating family-level variables in weight loss, physical activity, and dietary approaches in youth. Studies were reviewed and effect sizes were calculated for interventions that manipulated the family system, including components that targeted parenting styles, parenting skills, or family functioning, or which had novel approaches for including the family. Twenty-one weight loss interventions were identified, and 25 interventions related to physical activity and/or diet were identified. Overall, family-based treatment programs that incorporated training for authoritative parenting styles, parenting skills, or child management, and family functioning had positive effects on youth weight loss. Programs to improve physical activity and dietary behaviors that targeted the family system also demonstrated improvements in youth health behaviors; however, direct effects of parent-targeted programming is not clear. Both treatment and prevention programs would benefit from evaluating family functioning and parenting styles as possible mediators of intervention outcomes. Recommendations are provided to guide the development of future family-based obesity prevention and treatment programs for youth. PMID:20689989

Environmental Approaches to Prevention in College Settings

PubMed Central

Saltz, Robert F.

2011-01-01

Because of concerns regarding drinking among college students and its harmful consequences, numerous prevention efforts have been targeted to this population. These include individual-level and community-level interventions, as well as other measures (e.g., online approaches). Community-level interventions whose effects have been evaluated in college populations include programs that were developed for the community at large as well as programs aimed specifically at college students, such as A Matter of Degree, the Southwest DUI Enforcement Project, Neighborhoods Engaging With Students, the Study to Prevent Alcohol-Related Consequences, and Safer California Universities. Evaluations of these programs have found evidence of their effectiveness in reducing college drinking and related consequences. The most effective approaches to reducing alcohol consumption among college students likely will blend individual-, group-, campus-, and community-level prevention components. PMID:22330219

Cookbook Versus Creative Chemistry

ERIC Educational Resources Information Center

Venkatachelam, Chaya; Rudolph, R. W.

1974-01-01

A new approach to a research-oriented general chemistry laboratory is described. Objectives for the laboratory program are specified, details are provided concerning the program design, and the results of an experiment to evaluate the program are reported. (DT)

Granting Teachers the "Benefit of the Doubt" in Performance Evaluations

ERIC Educational Resources Information Center

Rogge, Nicky

2011-01-01

Purpose: This paper proposes a benefit of the doubt (BoD) approach to construct and analyse teacher effectiveness scores (i.e. SET scores). Design/methodology/approach: The BoD approach is related to data envelopment analysis (DEA), a linear programming tool for evaluating the relative efficiency performance of a set of similar units (e.g. firms,…

Geosynchronous platform definition study. Volume 6: Geosynchronous program evaluation and recommendations

NASA Technical Reports Server (NTRS)

Myers, H. L.

1973-01-01

The programmatic analyses conducted to achieve the objectives of the study are presented. The characteristics are examined of alternate geosynchronous programs based on servicing concepts, geosynchronous platform configurations, and equipment definitions which have evolved during the study. The logistics support necessary to carry out programs using these systems is defined considering alternate approaches for on-orbit servicing. The costs of the resultant programs are then determined and the alternate program approaches compared. Conventional programs with expendable satellites are also defined to the extent necessary to permit comparison with on-orbit serviced platform programs.

Building a Simulation Toolkit for Wireless Mesh Clusters and Evaluating the Suitability of Different Families of Ad Hoc Protocols for the Tactical Network Topology

DTIC Science & Technology

2005-03-01

International Conference On Computers Communications and Networks, 153- 161, Lafayette, L.A. Deitel , H.M. and P.J. Deitel . 2003. C++ How to Program ...of this study is to provide an additional performance evaluation technique for the TNT program of Naval Postgraduate School. The current approach...case are the PAMAS and DBTMA protocols. Toh (2002) illustrates how these approaches succeed in solving the problem. In order to address all the

Community-Based Participatory Evaluation: The Healthy Start Approach

PubMed Central

Braithwaite, Ronald L.; McKenzie, Robetta D.; Pruitt, Vikki; Holden, Kisha B.; Aaron, Katrina; Hollimon, Chavone

2013-01-01

The use of community-based participatory research has gained momentum as a viable approach to academic and community engagement for research over the past 20 years. This article discusses an approach for extending the process with an emphasis on evaluation of a community partnership–driven initiative and thus advances the concept of conducting community-based participatory evaluation (CBPE) through a model used by the Healthy Start project of the Augusta Partnership for Children, Inc., in Augusta, Georgia. Application of the CBPE approach advances the importance of bilateral engagements with consumers and academic evaluators. The CBPE model shows promise as a reliable and credible evaluation approach for community-level assessment of health promotion programs. PMID:22461687

Community-based participatory evaluation: the healthy start approach.

PubMed

Braithwaite, Ronald L; McKenzie, Robetta D; Pruitt, Vikki; Holden, Kisha B; Aaron, Katrina; Hollimon, Chavone

2013-03-01

The use of community-based participatory research has gained momentum as a viable approach to academic and community engagement for research over the past 20 years. This article discusses an approach for extending the process with an emphasis on evaluation of a community partnership-driven initiative and thus advances the concept of conducting community-based participatory evaluation (CBPE) through a model used by the Healthy Start project of the Augusta Partnership for Children, Inc., in Augusta, Georgia. Application of the CBPE approach advances the importance of bilateral engagements with consumers and academic evaluators. The CBPE model shows promise as a reliable and credible evaluation approach for community-level assessment of health promotion programs.

Algunos Criterios para Evaluar Programas de Educacion Superior a Nivel de Posgrado: El Caso Particular de la Administracion Publica (Some Criteria to Evaluate Higher Education Programs at the Graduate Level: The Special Case of Public Administration).

ERIC Educational Resources Information Center

Valle, Victor M.

Intended as a contribution to a workshop discussion on program evaluation in higher education, the paper covers five major evaluation issues. First, it deals with evaluation concepts, explaining the purposes of evaluation; pertinent terms; and the sources of evaluation in public health procedures, the scientific method, the systems approach, and…

Evaluation Study of "Language, Literacy, & Vocabulary!" Spring 2006 Pilot

ERIC Educational Resources Information Center

McNabb, Mary

2006-01-01

Purpose: A pilot study of the "Language, Literacy, & Vocabulary!" program was conducted by Learning Gauger, Inc., for National Geographic School Publishing, in spring 2006. The program evaluation examined the classroom adoption approaches used by participating teachers and the subsequent impact of the "Language, Literacy, &…

Evaluating Writing Programs: Paradigms, Problems, Possibilities.

ERIC Educational Resources Information Center

McLeod, Susan H.

1992-01-01

Describes two methodological approaches (qualitative and quantitative) that grow out of two different research examples. Suggests the problems these methods present. Discusses the ways in which an awareness of these problems can help teachers to understand how to work with researchers in designing useful evaluations of writing programs. (PRA)

Using Mixed Methods and Collaboration to Evaluate an Education and Public Outreach Program (Invited)

NASA Astrophysics Data System (ADS)

Shebby, S.; Shipp, S. S.

2013-12-01

Traditional indicators (such as the number of participants or Likert-type ratings of participant perceptions) are often used to provide stakeholders with basic information about program outputs and to justify funding decisions. However, use of qualitative methods can strengthen the reliability of these data and provide stakeholders with more meaningful information about program challenges, successes, and ultimate impacts (Stern, Stame, Mayne, Forss, David & Befani, 2012). In this session, presenters will discuss how they used a mixed methods evaluation to determine the impact of an education and public outreach (EPO) program. EPO efforts were intended to foster more effective, sustainable, and efficient utilization of science discoveries and learning experiences through three main goals 1) increase engagement and support by leveraging of resources, expertise, and best practices; 2) organize a portfolio of resources for accessibility, connectivity, and strategic growth; and 3) develop an infrastructure to support coordination. The evaluation team used a mixed methods design to conduct the evaluation. Presenters will first discuss five potential benefits of mixed methods designs: triangulation of findings, development, complementarity, initiation, and value diversity (Greene, Caracelli & Graham, 2005). They will next demonstrate how a 'mix' of methods, including artifact collection, surveys, interviews, focus groups, and vignettes, was included in the EPO project's evaluation design, providing specific examples of how alignment between the program theory and the evaluation plan was best achieved with a mixed methods approach. The presentation will also include an overview of different mixed methods approaches and information about important considerations when using a mixed methods design, such as selection of data collection methods and sources, and the timing and weighting of quantitative and qualitative methods (Creswell, 2003). Ultimately, this presentation will provide insight into how a mixed methods approach was used to provide stakeholders with important information about progress toward program goals. Creswell, J.W. (2003). Research design: Qualitative, quantitative, and mixed approaches. Thousand Oaks, CA: Sage. Greene, J. C., Caracelli, V. J., & Graham, W. D. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. Stern, E; Stame, N; Mayne, J; Forss, K; Davis, R & Befani, B (2012) Broadening the range of designs and methods for impact evaluation. Department for International Development.

Development and evaluation of the evidence-based medicine program in surgery: a spiral approach

PubMed Central

Elçin, Melih; Turan, Sevgi; Odabaşı, Orhan; Sayek, İskender

2014-01-01

Background Evidence-based medicine (EBM) aims to provide skills that help physicians answer clinically important questions, determine new evidence, and incorporate the acquired knowledge in practice. EBM skills are necessary for the practice of modern medicine, since physicians should use up-to-date knowledge and information to justify their medical decisions. Purpose We aimed to evaluate the EBM program implemented at Hacettepe University School of Medicine. Methods In 2004, a spiral program for the teaching and practice of EBM was developed for the first 3 years of medical school. Following this program, a practice of EBM was included in the fourth year during the surgery clerkship, after an introductory lecture. The students worked within collaborative teams of 3–5 and practiced EBM with actual cases seen in the surgical service in which they were involved. Each student was asked to complete a questionnaire that evaluated the more theoretical program from the first 3 years and the practical application in the fourth year. Results Nearly half of the students stated that the preclinical years of the EBM program were ‘adequate’, but only 30% of the students indicated that the program was practical. They stated that ‘more practical approaches were used in the fourth year, whereas more theory-based approaches were used during the preclinical years’. More than 75% of the students declared that the practice of EBM in the fourth year was useful and appropriate for team-based learning. Conclusions The EBM program was evaluated as ‘adequate’. EBM courses should be included in the entire curriculum in an integrated manner. The students understand the main philosophy of EBM in the clinical year when involved in its practical application with actual patients. PMID:24767706

A Rationale for Participant Evaluation

ERIC Educational Resources Information Center

Boody, Robert M.

2009-01-01

There are many different models or approaches to doing program evaluation. Fitzpatrick, Sanders, and Worthen classify them into five general approaches: (a) objectives oriented, (b) management oriented, (c) consumer oriented, (d) expertise oriented, and (e) participant oriented. Within each of these general categories, of course, reside many…

A Mandala of Faculty Development: Using Theory-Based Evaluation to Explore Contexts, Mechanisms and Outcomes

ERIC Educational Resources Information Center

Onyura, Betty; Ng, Stella L.; Baker, Lindsay R.; Lieff, Susan; Millar, Barbara-Ann; Mori, Brenda

2017-01-01

Demonstrating the impact of faculty development, is an increasingly mandated and ever elusive goal. Questions have been raised about the adequacy of current approaches. Here, we integrate realist and theory-driven evaluation approaches, to evaluate an intensive longitudinal program. Our aim is to elucidate how faculty development can work to…

Evaluating Integrative Cancer Clinics With the Claim Assessment Profile: An Example With the InspireHealth Clinic

PubMed Central

Hilton, Lara; Elfenbaum, Pamela; Jain, Shamini; Sprengel, Meredith; Jonas, Wayne B.

2016-01-01

Background: The evaluation of freestanding integrative cancer clinical programs is challenging and is rarely done. We have developed an approach called the Claim Assessment Profile (CAP) to identify whether evaluation of a practice is justified, feasible, and likely to provide useful information. Objectives: A CAP was performed in order to (1) clarify the healing claims at InspireHealth, an integrative oncology treatment program, by defining the most important impacts on its clients; (2) gather information about current research capacity at the clinic; and (3) create a program theory and path model for use in prospective research. Study Design/Methods: This case study design incorporates methods from a variety of rapid assessment approaches. Procedures included site visits to observe the program, structured qualitative interviews with 26 providers and staff, surveys to capture descriptive data about the program, and observational data on program implementation. Results: The InspireHealth program is a well-established, multi-site, thriving integrative oncology clinical practice that focuses on patient support, motivation, and health behavior engagement. It delivers patient-centered care via a standardized treatment protocol. There arehigh levels of research interest from staff and resources by which to conduct research. Conclusions: This analysis provides the primary descriptive and claims clarification of an integrative oncology treatment program, an evaluation readiness report, a detailed logic model explicating program theory, and a clinical outcomes path model for conducting prospective research. Prospective evaluation of this program would be feasible and valuable, adding to our knowledge base of integrative cancer therapies. PMID:29444602

Evaluation of a preschool nutrition education program based on the theory of multiple intelligences.

PubMed

Cason, K L

2001-01-01

This report describes the evaluation of a preschool nutrition education program based on the theory of multiple intelligences. Forty-six nutrition educators provided a series of 12 lessons to 6102 preschool-age children. The program was evaluated using a pretest/post-test design to assess differences in fruit and vegetable identification, healthy snack choices, willingness to taste foods, and eating behaviors. Subjects showed significant improvement in food identification and recognition, healthy snack identification, willingness to taste foods, and frequency of fruit, vegetable, meat, and dairy consumption. The evaluation indicates that the program was an effective approach for educating preschool children about nutrition.

The promise and challenge of practice-research collaborations: Guiding principles and strategies for initiating, designing, and implementing program evaluation research.

PubMed

Secret, Mary; Abell, Melissa L; Berlin, Trey

2011-01-01

The authors present a set of guiding principles and strategies to facilitate the collaborative efforts of social work researchers and practitioners as they initiate, design, and implement outcome evaluations of human service interventions and programs. Beginning with an exploration of the interpersonal barriers to practice-research collaborations, and building on their experiences in successfully completing a community-based research evaluation, the authors identify specific relationship-focused principles and strategies and illustrate how these approaches can guide practice-research teams through the various sequential activities of the evaluation research process. In particular, it is suggested that practice-research collaborations can be formed, strengthened, and sustained by emphasis on a spirit of discovery and shared leadership at the start of the relationship, use of a comprehensive evaluation model to clarify and frame the evaluation and program goals, beginning where the client is when selecting research methodology and measurement tools, commitment to keeping the program first and recording everything during the implementation and data-collection stages, discussion of emerging findings and presentation of findings in graphic format at the data-analysis stage, and a total team approach at the dissemination stage.

76 FR 14396 - Agency Information Collection Request; 60-Day Public Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2011-03-16

... within 60-days. Proposed Project: Evaluation of Adolescent Pregnancy Prevention Approaches (PPA) and the Impact Evaluation of the Teen Pregnancy Prevention Program Grantees: First Follow-up Data Collection--OMB No. OS-0990--NEW--Office of Adolescent Pregnancy Programs. Abstract: The Office of Adolescent Health...

Evaluating Technology Integration in the Elementary School: A Site-Based Approach.

ERIC Educational Resources Information Center

Mowe, Richard

This book enables educators at the elementary level to conduct formative evaluations of their technology programs in minimum time. Most of the technology is computer related, including word processing, graphics, desktop publishing, spreadsheets, databases, instructional software, programming, and telecommunications. The design of the book is aimed…

Methodological approach to a multidimensional evaluation of food and nutrition policies.

PubMed

Santos, Leonor Maria Pacheco; dos Santos, Sandra Maria Chaves

2003-01-01

Recommendations arising from global conferences and summits, expressed the need to formulate and implement public policies to improve household food security. In the context of maximizing benefits given available resources, comprehensive evaluations of the nutrition policies and programs are needed. One obstacle to overcome was a clear definition of terminology; the words efficiency, effectiveness, efficacy and impact, widely used in the context of program evaluation, are sometimes interchanged and there is no consensus about their precise definition. Another approach to health evaluation is based in the paradigm structure-process-outcome. The level structure contemplated the installations, resources, instruments (physical and discursive), as well as the technical bureaucratic organizational structures. Process encompassed the whole set of intervention activities developed, whereas the dimension of outcome comprised the effects of interventions on the health and nutrition of beneficiaries. Each of the three dimensions constitutes a continuum in the evaluation procedure: the structure only fulfills its purposes if the processes are adequate and conversely, processes cannot alone supersede structural limitations. Moreover, all the three dimensions only reach their ultimate objectives through the completion of outcomes. The methodology proposed here has been tested in the case of Bahia, Northeast Brazil, and it was found to be adequate for this type of analysis. We hope this approach of a comprehensive multidimensional evaluation, constitutes an effective contribution for program planners and program managers, in particular with regard to the obstacles detected, some of which can surely be overcome.

Multigroup Propensity Score Approach to Evaluating an Effectiveness Trial of the New Beginnings Program.

PubMed

Tein, Jenn-Yun; Mazza, Gina L; Gunn, Heather J; Kim, Hanjoe; Stuart, Elizabeth A; Sandler, Irwin N; Wolchik, Sharlene A

2018-06-01

We used a multigroup propensity score approach to evaluate a randomized effectiveness trial of the New Beginnings Program (NBP), an intervention targeting divorced or separated families. Two features of effectiveness trials, high nonattendance rates and inclusion of an active control, make program effects harder to detect. To estimate program effects based on actual intervention participation, we created a synthetic inactive control comprised of nonattenders and assessed the impact of attending the NBP or active control relative to no intervention (inactive control). We estimated propensity scores using generalized boosted models and applied inverse probability of treatment weighting for the comparisons. Relative to the inactive control, NBP strengthened parenting quality as well as reduced child exposure to interparental conflict, parent psychological distress, and child internalizing problems. Some effects were moderated by parent gender, parent ethnicity, or child age. On the other hand, the effects of active versus inactive control were minimal for parenting and in the unexpected direction for child internalizing problems. Findings from the propensity score approach complement and enhance the interpretation of findings from the intention-to-treat approach.

Model for Evaluating Specialized and Traditional Programs in Marketing and Distributive Education. Final Report. University of Kentucky Research Series in Marketing and Distributive Education.

ERIC Educational Resources Information Center

Litchfield, Carolyn G.

A project was conducted to develop a model for evaluating specialized and traditional programs in marketing and distributive education. The project included a review of literature containing information regarding the points of view expressed by advocates of the specialized, traditional, and middle-of-the-road approaches to program planning in…

Disease management interventions: what's in the black box?

PubMed

Linden, Ariel; Roberts, Nancy

2004-01-01

In discussing evaluation techniques to assess disease management (DM) program outcomes, it is often assumed that DM program interventions are premised on sound clinical judgment, an understanding of the disease process, and knowledge of the psychosocial models of behavioral change that must be used to effect those processes and ultimately improve the health outcomes that are being evaluated. This paper describes eight commonly used behavioral change models applied in the healthcare industry today. They represent programs designed to address individual, interpersonal, and community level factors as well as "packaged" comprehensive approaches. These models illustrate the breadth of approaches to consider when designing or assessing DM program interventions. Careful consideration of the type of behavioral change desired and the theories of how to effect such change should be an integral part of designing disease management program interventions.

Improving Culturally Appropriate Care Using a Community-Based Participatory Research Approach: Evaluation of a Multicomponent Cultural Competency Training Program, Arkansas, 2015–2016

PubMed Central

Long, Christopher R.; Rowland, Brett; Moore, Sarah; Wilmoth, Ralph; Ayers, Britni

2017-01-01

Introduction The United States continues to become more racially and ethnically diverse, and racial/ethnic minority communities encounter sociocultural barriers to quality health care, including implicit racial/ethnic bias among health care providers. In response, health care organizations are developing and implementing cultural competency curricula. Using a community-based participatory research (CBPR) approach, we developed and evaluated a cultural competency training program to improve the delivery of culturally appropriate care in Marshallese and Hispanic communities. Methods We used a mixed-methods evaluation approach based on the Kirkpatrick model of training evaluation. We collected quantitative evaluation data immediately after each training session (March 19, 2015–November 30, 2016) and qualitative data about implementation at 2 points: immediately after each session and 6 months after training. Individuals and organizational units provided qualitative data. Results We delivered 1,250 units of in-person training at 25 organizations. Participants reported high levels of changes in knowledge (91.2%), competence (86.6%), and performance (87.2%) as a result of the cultural competency training. Organizations reported making policy and environmental changes. Conclusion Initial outcomes demonstrate the value of developing and implementing cultural competency training programs using a CBPR approach. Additional research is needed to determine the effect on long-term patient outcomes. PMID:28771402

A program wide framework for evaluating data driven teaching and learning - earth analytics approaches, results and lessons learned

NASA Astrophysics Data System (ADS)

Wasser, L. A.; Gold, A. U.

2017-12-01

There is a deluge of earth systems data available to address cutting edge science problems yet specific skills are required to work with these data. The Earth analytics education program, a core component of Earth Lab at the University of Colorado - Boulder - is building a data intensive program that provides training in realms including 1) interdisciplinary communication and collaboration 2) earth science domain knowledge including geospatial science and remote sensing and 3) reproducible, open science workflows ("earth analytics"). The earth analytics program includes an undergraduate internship, undergraduate and graduate level courses and a professional certificate / degree program. All programs share the goals of preparing a STEM workforce for successful earth analytics driven careers. We are developing an program-wide evaluation framework that assesses the effectiveness of data intensive instruction combined with domain science learning to better understand and improve data-intensive teaching approaches using blends of online, in situ, asynchronous and synchronous learning. We are using targeted online search engine optimization (SEO) to increase visibility and in turn program reach. Finally our design targets longitudinal program impacts on participant career tracts over time.. Here we present results from evaluation of both an interdisciplinary undergrad / graduate level earth analytics course and and undergraduate internship. Early results suggest that a blended approach to learning and teaching that includes both synchronous in-person teaching and active classroom hands-on learning combined with asynchronous learning in the form of online materials lead to student success. Further we will present our model for longitudinal tracking of participant's career focus overtime to better understand long-term program impacts. We also demonstrate the impact of SEO optimization on online content reach and program visibility.

Preliminary evaluation of spectral, normal and meteorological crop stage estimation approaches

NASA Technical Reports Server (NTRS)

Cate, R. B.; Artley, J. A.; Doraiswamy, P. C.; Hodges, T.; Kinsler, M. C.; Phinney, D. E.; Sestak, M. L. (Principal Investigator)

1980-01-01

Several of the projects in the AgRISTARS program require crop phenology information, including classification, acreage and yield estimation, and detection of episodal events. This study evaluates several crop calendar estimation techniques for their potential use in the program. The techniques, although generic in approach, were developed and tested on spring wheat data collected in 1978. There are three basic approaches to crop stage estimation: historical averages for an area (normal crop calendars), agrometeorological modeling of known crop-weather relationships agrometeorological (agromet) crop calendars, and interpretation of spectral signatures (spectral crop calendars). In all, 10 combinations of planting and biostage estimation models were evaluated. Dates of stage occurrence are estimated with biases between -4 and +4 days while root mean square errors range from 10 to 15 days. Results are inconclusive as to the superiority of any of the models and further evaluation of the models with the 1979 data set is recommended.

Challenges and solutions in the evaluation of a low back pain disease management program.

PubMed

Kotsos, Thomas; Muldowney, Kevin; Chapa, Griselda; Martin, J Eric; Linares, Antonio

2009-02-01

This paper examines a novel approach to evaluating a nurse-run telephonic low back pain (LBP) disease management (DM) program offered by a fully-insured commercial health plan population with approximately 150,000 members located in the northeastern United States. Members with at least 6 member months of eligibility were identified to have LBP using administrative claims and eligibility data. The LBP program relies on telephonic nurse management augmented by printed materials, adapting and advocating HEDIS imaging guidelines, and drug utilization review. Outcomes of this LBP DM program were assessed using a pre-post population-based approach as recommended in the DMAA Outcomes Guidelines Report. Baseline year and program year populations were segmented into 5 LBP clinical categories and each was weight adjusted using population size. LBP-related medical service utilization and pharmacy utilization also were evaluated. Individuals under active LBP management exhibited a decrease in LBP-related imaging and surgeries. Overall analgesic use also decreased. These data suggest the LBP DM program is associated with a decrease in imaging, surgeries, and analgesic use. The magnitude of the clinically segmented weight-adjusted return on investment (ROI) was lower than the unadjusted ROI, but was directionally positive indicating program impact. This adjustment is necessary in order to gain insight into and consistency of the results of a comprehensive LBP DM program evaluation.

Design and approach of the Living Organ Video Educated Donors (LOVED) program to promote living kidney donation in African Americans.

PubMed

Sieverdes, John C; Price, Matthew; Ruggiero, Kenneth J; Baliga, Prabhakar K; Chavin, Kenneth D; Brunner-Jackson, Brenda; Patel, Sachin; Treiber, Frank A

2017-10-01

To describe the rationale, methodology, design, and interventional approach of a mobile health education program designed for African Americans with end stage renal disease (ESRD) to increase knowledge and self-efficacy to approach others about their need for a living donor kidney transplant (LDKT). The Living Organ Video Educated Donors (LOVED) program is a theory-guided iterative designed, mixed methods study incorporating three phases: 1) a formative evaluation using focus groups to develop program content and approach; 2) a 2-month proof of concept trial (n=27) to primarily investigate acceptability, tolerability and investigate increases of LDKT knowledge and self-efficacy; and 3) a 6-month, 2-arm, 60-person feasibility randomized control trial (RCT) to primarily investigate increases in LDKT knowledge and self-efficacy, and secondarily, to increase the number of living donor inquiries, medical evaluations, and LDKTs. The 8-week LOVED program includes an interactive web-based app delivered on 10″ tablet computer incorporating weekly interactive video education modules, weekly group video chat sessions with an African American navigator who has had LDKT and other group interactions for support and improve strategies to promote their need for a kidney. Phase 1 and 2 have been completed and the program is currently enrolling for the feasibility RCT. Phase 2 experienced 100% retention rates with 91% adherence completing the video modules and 88% minimum adherence to the video chat sessions. We are in the early stages of an RCT to evaluate the LOVED program; to date, we have found high tolerability reported from Phase 2. Copyright © 2017 Elsevier Inc. All rights reserved.

Methodological Considerations for Use of Routine Health Information System Data to Evaluate Malaria Program Impact in an Era of Declining Malaria Transmission

PubMed Central

Ashton, Ruth A.; Bennett, Adam; Yukich, Joshua; Bhattarai, Achuyt; Keating, Joseph; Eisele, Thomas P.

2017-01-01

Abstract. Coverage of malaria control interventions is increasing dramatically across endemic countries. Evaluating the impact of malaria control programs and specific interventions on health indicators is essential to enable countries to select the most effective and appropriate combination of tools to accelerate progress or proceed toward malaria elimination. When key malaria interventions have been proven effective under controlled settings, further evaluations of the impact of the intervention using randomized approaches may not be appropriate or ethical. Alternatives to randomized controlled trials are therefore required for rigorous evaluation under conditions of routine program delivery. Routine health management information system (HMIS) data are a potentially rich source of data for impact evaluation, but have been underused in impact evaluation due to concerns over internal validity, completeness, and potential bias in estimates of program or intervention impact. A range of methodologies were identified that have been used for impact evaluations with malaria outcome indicators generated from HMIS data. Methods used to maximize internal validity of HMIS data are presented, together with recommendations on reducing bias in impact estimates. Interrupted time series and dose-response analyses are proposed as the strongest quasi-experimental impact evaluation designs for analysis of malaria outcome indicators from routine HMIS data. Interrupted time series analysis compares the outcome trend and level before and after the introduction of an intervention, set of interventions or program. The dose-response national platform approach explores associations between intervention coverage or program intensity and the outcome at a subnational (district or health facility catchment) level. PMID:28990915

Implementing and evaluating a program to facilitate chronic disease prevention and screening in primary care: a mixed methods program evaluation.

PubMed

Manca, Donna Patricia; Aubrey-Bassler, Kris; Kandola, Kami; Aguilar, Carolina; Campbell-Scherer, Denise; Sopcak, Nicolette; O'Brien, Mary Ann; Meaney, Christopher; Faria, Vee; Baxter, Julia; Moineddin, Rahim; Salvalaggio, Ginetta; Green, Lee; Cave, Andrew; Grunfeld, Eva

2014-10-08

The objectives of this paper are to describe the planned implementation and evaluation of the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Primary Care (BETTER 2) program which originated from the BETTER trial. The pragmatic trial, informed by the Chronic Care Model, demonstrated the effectiveness of an approach to Chronic Disease Prevention and Screening (CDPS) involving the use of a new role, the prevention practitioner. The desired goals of the program are improved clinical outcomes, reduction in the burden of chronic disease, and improved sustainability of the health-care system through improved CDPS in primary care. The BETTER 2 program aims to expand the implementation of the intervention used in the original BETTER trial into communities across Canada (Alberta, Ontario, Newfoundland and Labrador, the Northwest Territories and Nova Scotia). This proactive approach provides at-risk patients with an intervention from the prevention practitioner, a health-care professional. Using the BETTER toolkit, the prevention practitioner determines which CDPS actions the patient is eligible to receive, and through shared decision-making and motivational interviewing, develops a unique and individualized 'prevention prescription' with the patient. This intervention is 1) personalized; 2) addressing multiple conditions; 3) integrated through linkages to local, regional, or national resources; and 4) longitudinal by assessing patients over time. The BETTER 2 program brings together primary care providers, policy/decision makers and researchers to work towards improving CDPS in primary care. The target patient population is adults aged 40-65. The reach, effectiveness, adoption, implementation, maintain (RE-AIM) framework will inform the evaluation of the program through qualitative and quantitative methods. A composite index will be used to quantitatively assess the effectiveness of the prevention practitioner intervention. The CDPS actions comprising the composite index include the following: process measures, referral/treatment measures, and target/change outcome measures related to cardiovascular disease, diabetes, cancer and associated lifestyle factors. The BETTER 2 program is a collaborative approach grounded in practice and built from existing work (i.e., integration not creation). The program evaluation is designed to provide an understanding of issues impacting the implementation of an effective approach for CDPS within primary care that may be adapted to become sustainable in the non-research setting.

Examining Criteria for Evaluating Educational Programs.

ERIC Educational Resources Information Center

Shapiro, Edna

The negative results to an assessment of a Bank Street-sponsored Follow Through program raised questions about conventional ways of assessing the effects of educational programs. Children in a Follow Through program using the developmental-interaction approach and children in conventional classrooms were observed and tested for comparison. The…

Adult Offenders' Perceptions of Rehabilitation Programs in Africa

ERIC Educational Resources Information Center

Ngozwana, Nomazulu

2017-01-01

This article reflects on adult offenders' perceptions of rehabilitation programs in Africa. It also evaluates whether offenders are consulted when planning rehabilitation programs. Adult education principles were used as a lens to understand offenders' perceptions of rehabilitation programs. Using an interpretive paradigm and qualitative approach,…

Unlocking the Potential of the "What Works" Approach to Policymaking and Practice: Improving Impact Evaluations

ERIC Educational Resources Information Center

Granger, Robert C.; Maynard, Rebecca

2015-01-01

Despite bipartisan support in Washington, DC, which dates back to the mid-1990s, the "what works" approach has yet to gain broad support among policymakers and practitioners. One way to build such support is to increase the usefulness of program impact evaluations for these groups. We describe three ways to make impact evaluations more…

Redesigning an Introductory Language Curriculum: A Backward Design Approach

ERIC Educational Resources Information Center

Paesani, Kate

2017-01-01

In response to calls for curricular change in foreign language programs and institutional requirements to evaluate programmatic effectiveness, this article presents a backward design approach to the redesign of an introductory French curriculum grounded in the framing concept of cultural literacy. In addition, data from student evaluations,…

Academic ROI: What Does the Most Good?

ERIC Educational Resources Information Center

Levenson, Nathan

2012-01-01

The author, who has served as a school board member and district superintendent, advocates that school districts use an academic return on investment approach to evaluate and make decisions about spending, specifically in special education. This approach requires that schools formally evaluate all programs, efforts, and strategies by multiple…

Drug Education: Goals, Approaches, Evaluation, ERS Report.

ERIC Educational Resources Information Center

Bushey, Julia A.

An examination of the extensive drug education literature as well as of specific school drug education programs indicates that the present confusion over drug education results from confusion over definitions, goals, approaches, and evaluation. This report, based on an examination of recent literature and on information and printed materials on…

An evaluation capacity building toolkit for principal investigators of undergraduate research experiences: A demonstration of transforming theory into practice.

PubMed

Rorrer, Audrey S

2016-04-01

This paper describes the approach and process undertaken to develop evaluation capacity among the leaders of a federally funded undergraduate research program. An evaluation toolkit was developed for Computer and Information Sciences and Engineering(1) Research Experiences for Undergraduates(2) (CISE REU) programs to address the ongoing need for evaluation capacity among principal investigators who manage program evaluation. The toolkit was the result of collaboration within the CISE REU community with the purpose being to provide targeted instructional resources and tools for quality program evaluation. Challenges were to balance the desire for standardized assessment with the responsibility to account for individual program contexts. Toolkit contents included instructional materials about evaluation practice, a standardized applicant management tool, and a modulated outcomes measure. Resulting benefits from toolkit deployment were having cost effective, sustainable evaluation tools, a community evaluation forum, and aggregate measurement of key program outcomes for the national program. Lessons learned included the imperative of understanding the evaluation context, engaging stakeholders, and building stakeholder trust. Results from project measures are presented along with a discussion of guidelines for facilitating evaluation capacity building that will serve a variety of contexts. Copyright © 2016. Published by Elsevier Ltd.

Identifying Potential Biasing Variables in Student Evaluation of Teaching in a Newly Accredited Business Program in the UAE

ERIC Educational Resources Information Center

Badri, Masood A.; Abdulla, Mohamed; Kamali, Mohammed A.; Dodeen, Hamzeh

2006-01-01

Purpose: The purpose of this paper is to investigate the effect of many factors on student evaluation of teaching. Design/methodology/approach: The study analyzed 3,185 student evaluations of faculty from a newly accredited business program at the United Arab Emirates University using univariate and multi-analysis of variance (ANOVA and MANOVA).…

Blended learning: strengths, challenges, and lessons learned in an interprofessional training program.

PubMed

Lotrecchiano, G R; McDonald, P L; Lyons, L; Long, T; Zajicek-Farber, M

2013-11-01

This field report outlines the goals of providing a blended learning model for an interdisciplinary training program for healthcare professionals who care for children with disabilities. The curriculum blended traditional face-to-face or on-site learning with integrated online interactive instruction. Credit earning and audited graduate level online coursework, community engagement experiences, and on-site training with maternal and child health community engagement opportunities were blended into a cohesive program. The training approach emphasized adult learning principles in different environmental contexts integrating multiple components of the Leadership Education in Neurodevelopmental and Related Disabilities Program. This paper describes the key principles adopted for this blended approach and the accomplishments, challenges, and lessons learned. The discussion offers examples from training content, material gathered through yearly program evaluation, as well as university course evaluations. The lessons learned consider the process and the implications for the role of blended learning in this type of training program with suggestions for future development and adoption by other programs.

The State Agency--College-Mandated Approach.

ERIC Educational Resources Information Center

Capoor, Madan; Morante, Edward

1990-01-01

Describes two state-mandated programs for institutional accountability in New Jersey (i.e., Basic Skills Assessment Program and College Outcomes Evaluation Program), which function through cooperation between the state's higher education agency and its colleges and universities. Highlights research objectives, procedures, responsibilities,…

Evaluating employee assistance programs. A review of methods, outcomes, and future directions.

PubMed

Jerrell, J M; Rightmyer, J F

1982-01-01

Renewed interest in assisting troubled employees has led to an upsurge in the development of employee assistance programs, coupled with demands for demonstrable effectiveness. This review examines the nature and scope of these programs, their administrative and methodological context, and the types and outcomes of evaluation studies conducted thus far. Proposals for improving future investigations through a number of different approaches and strategies are then made.

A counterfactual impact evaluation of a bilingual program on students' grade point average at a spanish university.

PubMed

Arco-Tirado, J L; Fernández-Martín, F; Ramos-García, A M; Littvay, L; Villoria, J; Naranjo, J A

2018-06-01

This observational study intends to estimate the causal effects of an English as a Medium of Instruction (EMI) program (as predictor) on students Grade Point Average (GPA) (as outcome) at a particular University in Spain by using a Counterfactual Impact Evaluation (CIE). The need to address the crucial question of causal inferences in EMI programs to produce credible evidences of successful interventions contrasts, however, with the absence of experimental or quasi-experimental research and evaluation designs in the field. CIE approach is emerging as a methodologically viable solution to bridge that gap. The program evaluated here consisted in delivering an EMI program in a Primary Education Teacher Training Degree group. After achieving balance on the observed covariates and recreating a situation that would have been expected in a randomized experiment, three matching approaches such as genetic matching, nearest neighbor matching and Coarsened Exact Matching were used to analyze observational data from a total of 1288 undergraduate students, including both treatment and control group. Results show unfavorable effects of the bilingual group treatment condition. Potential interpretations and recommendations are provided in order to strengthen future causal evidences of bilingual education programs' effectiveness in Higher Education. Copyright © 2018 The Authors. Published by Elsevier Ltd.. All rights reserved.

An evaluation of Teaching-Family (Achievement Place) group homes for juvenile offenders.

PubMed

Kirigin, K A; Braukmann, C J; Atwater, J D; Wolf, M M

1982-01-01

Juvenile crime is a serious problem for which treatment approach has been found to be reliably effective. This outcome evaluation assessed during and posttreatment effectiveness of Teaching-Family group home treatment programs for juvenile offenders. The evaluation included the original Achievement Place program, which was the prototype for the development of the Teaching-Family treatment approach, 12 replications of Achievement Place, and 9 comparison group home programs. Primary dependent measures were retrieved from court and police files and included number of alleged offenses, percentage of youths involved in those alleged offenses, and percentage of youths institutionalized. Other dependent measures were subjective ratings of effectiveness obtained from the program consumers, including the group home residents. The results showed difference during treatment favoring the Teaching-Family programs on rate of alleged criminal offenses, percentage of youths involved in those offenses, and consumer ratings of the programs. The consumer ratings provided by the youths and their school teachers were found to be inversely and significantly correlated with the reduction of criminal offenses during treatment. There were no significant differences during treatment on measures of noncriminal offenses (e.g., truancy, runaway, and curfew violations). In the posttreatment year, none of the differences between the groups was significant on any of the outcome measures. The results are discussed in terms of measurement and design issues in the evaluation of delinquency treatment programs and in relation to the evaluation; of Teaching-Family group homes by Richard Jones and his colleagues.

An evaluation of Teaching-Family (Achievement Place) group homes for juvenile offenders.

PubMed Central

Kirigin, K A; Braukmann, C J; Atwater, J D; Wolf, M M

1982-01-01

Juvenile crime is a serious problem for which treatment approach has been found to be reliably effective. This outcome evaluation assessed during and posttreatment effectiveness of Teaching-Family group home treatment programs for juvenile offenders. The evaluation included the original Achievement Place program, which was the prototype for the development of the Teaching-Family treatment approach, 12 replications of Achievement Place, and 9 comparison group home programs. Primary dependent measures were retrieved from court and police files and included number of alleged offenses, percentage of youths involved in those alleged offenses, and percentage of youths institutionalized. Other dependent measures were subjective ratings of effectiveness obtained from the program consumers, including the group home residents. The results showed difference during treatment favoring the Teaching-Family programs on rate of alleged criminal offenses, percentage of youths involved in those offenses, and consumer ratings of the programs. The consumer ratings provided by the youths and their school teachers were found to be inversely and significantly correlated with the reduction of criminal offenses during treatment. There were no significant differences during treatment on measures of noncriminal offenses (e.g., truancy, runaway, and curfew violations). In the posttreatment year, none of the differences between the groups was significant on any of the outcome measures. The results are discussed in terms of measurement and design issues in the evaluation of delinquency treatment programs and in relation to the evaluation; of Teaching-Family group homes by Richard Jones and his colleagues. PMID:7096223

Feasibility Study of Comprehensive School Physical Activity Programs in Appalachian Communities: The McDowell CHOICES Project

ERIC Educational Resources Information Center

Jones, Emily M.; Taliaferro, Andrea R.; Elliott, Eloise M.; Bulger, Sean M.; Kristjansson, Alfgeir L.; Neal, William; Allar, Ishonté

2014-01-01

Increasing rates of childhood obesity has prompted calls for comprehensive approaches to school-based physical activity (PA). The purpose of this study was to evaluate the feasibility of comprehensive school physical activity program (CSPAP) development and related contextual issues within a rural Appalachian county using a Systems Approach. A…

Replication of an Inter-Disciplinary Approach to Early Education of Handicapped Children 0-3 Years.

ERIC Educational Resources Information Center

Smiley, Constance J.; And Others

Presented is the guide to the Illinois project entitled "An Inter-Disciplinary Approach to Early Education of Handicapped Children Ages 0 - 3 Years" which includes information on funding and public awareness, diagnosis and evaluation, child development-home program, speech and language, structuring the day program, job descriptions and training,…

Computer-Assisted, Counselor-Delivered Smoking Cessation Counseling for Community College Students: Intervention Approach and Sample Characteristics

ERIC Educational Resources Information Center

Prokhorov, Alexander V.; Fouladi, Rachel T.; de Moor, Carl; Warneke, Carla L.; Luca, Mario; Jones, Mary Mullin; Rosenblum, Carol; Emmons, Karen M.; Hudmon, Karen Suchanek; Yost, Tracey E.; Gritz, Ellen R.

2007-01-01

This report presents the experimental approach and baseline findings from "Look at Your Health," an ongoing study to develop and evaluate a computer-assisted, counselor-delivered smoking cessation program for community college students. It describes the expert system software program used for data collection and for provision of tailored feedback,…

Empowerment evaluation: An approach that has literally altered the landscape of evaluation.

PubMed

Donaldson, Stewart I

2017-08-01

The quest for credible and actionable evidence to improve decision making, foster improvement, enhance self-determination, and promote social betterment is now a global phenomenon. Evaluation theorists and practitioners alike have responded to and overcome the challenges that limited the effectiveness and usefulness of traditional evaluation approaches primarily focused on seeking rigorous scientific knowledge about social programs and policies. No modern evaluation approach has received a more robust welcome from stakeholders across the globe than empowerment evaluation. Empowerment evaluation has been a leader in the development of stakeholder involvement approaches to evaluation, setting a high bar. In addition, empowerment evaluation's respect for community knowledge and commitment to the people's right to build their own evaluation capacity has influenced the evaluation mainstream, particularly concerning evaluation capacity building. Empowerment evaluation's most significant contributions to the field have been to improving evaluation use and knowledge utilization. Copyright © 2016. Published by Elsevier Ltd.

Instructional Leadership: Using a Research Approach to Evaluate the Impact of a Reading Intervention Program

ERIC Educational Resources Information Center

O'Hare, John M.

2012-01-01

Though campus principals must take on many leadership roles, their primary responsibility is to facilitate effective teaching and learning with the overall mission of enhancing student achievement. One way to provide instructional leadership is to evaluate and select effective instructional programs that align with campus goals of improving…

Assessment of Public Schools' Out-of-School Time Academic Support Programs with Participant-Oriented Evaluation

ERIC Educational Resources Information Center

Berk, Saban

2018-01-01

Using the participants-oriented approach, this study evaluated public schools' out-of-school time academic support programs, corresponding to the corrective/enrichment stage of Bloom's Mastery Learning Model and offered outside formal education's weekday hours and on weekends. Study participants included 50 principals, 110 teachers, 170 students…

Effective Practices in Community Based Family Literacy: Results of a National Research and Evaluation Project.

ERIC Educational Resources Information Center

Association for Community Based Education, Washington, DC.

A field evaluation of 14 community-based family and intergenerational literacy programs identified the most effective strategies, structures, and approaches to reach and teach the "hardest to reach." Information was collected through 90-minute telephone surveys with program coordinators and/or executive directors. Although different in structural…

Community College Dual Enrollment Faculty Orientation: A Utilization-Focused Approach

ERIC Educational Resources Information Center

Charlier, Hara D.; Duggan, Molly H.

2010-01-01

The current climate of accountability demands that institutions engage in data-driven program evaluation. In order to promote quality dual enrollment (DE) programs, institutions must support the adjunct faculty teaching college courses in high schools. This study uses Patton's utilization-focused model (1997) to conduct a formative evaluation of a…

Evaluating Admission Criteria Effects for Under-Represented Groups

ERIC Educational Resources Information Center

Childs, Ruth A.; Ferguson, Amanda K.; Herbert, Monique B.; Broad, Kathryn; Zhang, Jingshun

2016-01-01

The effects that admission criteria may have for under-represented groups are an important concern for programs seeking to improve access to post-secondary education. Using data from a large preservice teacher education program in the Canadian province of Ontario, we demonstrate two approaches to evaluating the effects of admission criteria. The…

Evaluation of a Place-Based Environmental Education Program: From There to Here

ERIC Educational Resources Information Center

Cincera, Jan; Johnson, Bruce; Kovacikova, Silvia

2015-01-01

The article introduces a Czech environmental education middle school program focused on helping students develop place attachment to their community and on increasing their interest in natural area in their region. The evaluation applied a mixed approach combining pretesting/posttesting of the students (N = 158), two group interviews with selected…

Evaluating Quality in Associate Degree Culinary Arts Programs

ERIC Educational Resources Information Center

Hertzman, Jean; Ackerman, Robert

2010-01-01

Purpose: The purpose of this study is to determine which categories and indicators of quality are best suited to evaluating associate degree culinary arts programs (ADCAP). Design/methodology/approach: The researchers surveyed a national sample of culinary educators and industry chefs in the USA. The instrument asked the participants to rate the…

One Kid at a Time: Evaluative Case Studies and Description of the Alaska Youth Initiative Demonstration Project.

ERIC Educational Resources Information Center

Burchard, John D.; And Others

This monograph presents an evaluative description of the Alaska Youth Initiative (AYI), a community-based interagency program serving children and adolescents with severe emotional and behavioral disorders. Principles of the program include a no reject policy and a "wraparound" service delivery approach. The monograph offers information…

Home Start Evaluation Study. Interim Case Studies IIa.

ERIC Educational Resources Information Center

Fein, Robert

This formative evaluation study of Home Start uses a case study approach. A brief case study focuses on the administrative structure and staff resources and responsibilities of National Home Start. Also included are reports on seven local programs developed after two field visits had been made to each program. In the first visit, objectives chosen…

Evaluating the Effectiveness of READ 180: A Reading Program for Struggling Middle School Students

ERIC Educational Resources Information Center

Johnson, Latonya R.

2011-01-01

This study was conducted using a qualitative approach to evaluate the effectiveness of READ 180 Program in improving students reading interests and achievement at West Bolivar Middle School (WBMS). A number of interviews and observations were conducted. Two teachers and nine students were interviewed, and nine classroom observations were…

Model for an Employee Wellness Project.

ERIC Educational Resources Information Center

Love, Marybeth; And Others

1981-01-01

The evaluation of a health promotion program indicated that it had successfully used several approaches, including: psychophysical tests, individual health consultants, group programs, mass media, and wellness-oriented social activities. (CJ)

Economic evaluation: Concepts, selected studies, system costs, and a proposed program

NASA Technical Reports Server (NTRS)

Osterhoudt, F. H. (Principal Investigator)

1979-01-01

The more usual approaches to valuing crop information are reviewed and an integrated approach is recommended. Problems associated with implementation are examined. What has already been accomplished in the economic evaluation of LACIE-type information is reported including various studies of benefits. The costs of the existing and proposed systems are considered. A method and approach is proposed for further studies.

Socioeconomic impact assessment in ex ante evaluations: a case study on the rural development programs of the European Union.

PubMed

Vidueira, Pablo; Díaz-Puente, José M; Rivera, María

2014-08-01

Ex ante impact assessment has become a fundamental tool for effective program management, and thus, a compulsory task when establishing a new program in the European Union (EU). This article aims to analyze benefits from ex ante impact assessment, methodologies followed, and difficulties encountered. This is done through the case study on the rural development programs (RDPs) in the EU. Results regarding methodologies are then contrasted with the international context in order to provide solid insights to evaluators and program managing authorities facing ex ante impact assessment. All European RDPs from the period 2007 through 2013 (a total of 88) and their corresponding available ex ante evaluations (a total of 70) were analyzed focusing on the socioeconomic impact assessment. Only 46.6% of the regions provide quantified impact estimations on socioeconomic impacts in spite of it being a compulsory task demanded by the European Commission (EC). Recommended methods by the EC are mostly used, but there is a lack of mixed method approaches since qualitative methods are used in substitution of quantitative ones. Two main difficulties argued were the complexity of program impacts and the lack of needed program information. Qualitative approaches on their own have been found as not suitable for ex ante impact assessment, while quantitative approaches-such as microsimulation models-provide a good approximation to actual impacts. However, time and budgetary constraints make that quantitative and mixed methods should be mainly applied on the most relevant impacts for the program success. © The Author(s) 2014.

Evaluation of the Florida coordinated school health program pilot schools project.

PubMed

Weiler, Robert M; Pigg, R Morgan; McDermott, Robert J

2003-01-01

The Florida Department of Education, with CDC funding, designed the Florida Coordinated School Health Program Pilot Schools Project (PSP) to encourage innovative approaches to promote coordinated school health programs (CSHP) in Florida schools. Each of eight pilot schools received $15,000 in project funding, three years of technical assistance including on-site and off-site assistance, a project office resource center, mailings of resource materials, needs assessment and evaluation assistance, and three PSP Summer Institutes. Project evaluators created a context evaluation, approaching each school independently as a "case study" to measure the school's progress in meeting goals established at baseline. Data were collected using the How Healthy is Your School? needs assessment instrument, a School Health Portfolio constructed by each school team, a Pilot Schools Project Team Member Survey instrument, midcourse team interviews, final team interviews, and performance indicator data obtained from pilot and control schools. The PSP posed two fundamental questions: "Can financial resources, professional training, and technical assistance enable individual schools to create and sustain a coordinated school health program?" and "What outcomes reasonably can one expect from a coordinated school health program, assuming programs receive adequate support over time?" First, activities at the eight schools confirmed that a coordinated school health programs can be established and sustained. Program strength and sustainability depend on long-term resources, qualified personnel, and administrative support. Second, though coordinated school health programs may improve school performance indicators, the PSP yielded insufficient evidence to support that belief. Future projects should include robust measurement and evaluation designs, thereby producing conclusive evidence about the influence of a coordinated school health program on such outcomes.

Reducing HIV and AIDS through Prevention (RHAP): a theoretically based approach for teaching HIV prevention to adolescents through an exploration of popular music.

PubMed

Boutin-Foster, Carla; McLaughlin, Nadine; Gray, Angela; Ogedegbe, Anthony; Hageman, Ivan; Knowlton, Courtney; Rodriguez, Anna; Beeder, Ann

2010-05-01

Using popular culture to engage students in discussions of HIV prevention is a nontraditional approach that may complement current prevention efforts and enhance the ability to reach youth who are at high risk of contracting HIV and other sexually transmitted infections. Hip-hop or rap music is the dominant genre of music among adolescents, especially Black and Latino youth who are disproportionately impacted by HIV and AIDS. This paper describes the rationale and development of the Reducing HIV and AIDS through Prevention (RHAP) program, a school-based program that uses hip-hop/rap music as a vehicle for raising awareness among adolescents about HIV/AIDS. Constructs from the Social Cognitive Theory and the Sexual Script Theory were used in developing the program. It was piloted and evaluated among 26 middle school students in East Harlem, New York. The lessons learned from a formative evaluation of the program and the implications for developing other programs targeting public health problems are discussed. The RHAP program challenges the traditional pedagogue-student paradigm and provides an alternative approach to teaching about HIV prevention and awareness.

Principles for assessing disease management outcomes.

PubMed

Fitzner, Karen; Sidorov, Jaan; Fetterolf, Don; Wennberg, David; Eisenberg, Edward; Cousins, Michael; Hoffman, Joel; Haughton, John; Charlton, Warwick; Krause, David; Woolf, Allen; Mcdonough, Kenneth; Todd, Warren; Fox, Kathe; Plocher, David; Juster, Iver; Stiefel, Matt; Villagra, Victor; Duncan, Ian

2004-01-01

Disease management (DM) is rapidly becoming an important force in the late 20th and early 21st century as a strategy for managing the chronic illness of large populations. Given the increasing visibility of DM programs, the clinical, economic and financial impact of this support are vital to DM program accountability and its acceptance as a solution to the twin challenges of achieving affordable, quality health care. Measuring and reporting outcomes in DM is difficult. DM programs must adapt to local market conditions and customer desires, which in turn limits generalizability, and still account for the overlapping/interlocking/multifaceted nature of the interventions included in any DM program. The Disease Management Association of America convened a Steering Committee to suggest a preferred approach, not a mandated or standardized approach for DM program evaluation. This paper presents the Steering Committee's "Consensus Statement" and "Guiding Principles" for robust evaluation.

Rethink, Reform, Reenter: An Entrepreneurial Approach to Prison Programming.

PubMed

Keena, Linda; Simmons, Chris

2015-07-01

The purpose of this article was to present a description and first-stage evaluation of the impact of the Ice House Entrepreneurship Program on the learning experience of participating prerelease inmates at a Mississippi maximum-security prison and their perception of the transfer of skills learned in program into securing employment upon reentry. The Ice House Entrepreneurship Program is a 12-week program facilitated by volunteer university professors to inmates in a prerelease unit of a maximum-security prison in Mississippi. Participants' perspectives were examined through content analysis of inmates' answers to program Reflection and Response Assignments and interviews. The analyses were conducted according to the constant comparative method. Findings reveal the emergent of eight life-lessons and suggest that this is a promising approach to prison programming for prerelease inmates. This study discusses three approaches to better prepare inmates for a mindset change. The rethink, reform, and reenter approaches help break the traditional cycle of release, reoffend, and return. © The Author(s) 2014.

Using Instructional Design, Analyze, Design, Develop, Implement, and Evaluate, to Develop e-Learning Modules to Disseminate Supported Employment for Community Behavioral Health Treatment Programs in New York State.

PubMed

Patel, Sapana R; Margolies, Paul J; Covell, Nancy H; Lipscomb, Cristine; Dixon, Lisa B

2018-01-01

Implementation science lacks a systematic approach to the development of learning strategies for online training in evidence-based practices (EBPs) that takes the context of real-world practice into account. The field of instructional design offers ecologically valid and systematic processes to develop learning strategies for workforce development and performance support. This report describes the application of an instructional design framework-Analyze, Design, Develop, Implement, and Evaluate (ADDIE) model-in the development and evaluation of e-learning modules as one strategy among a multifaceted approach to the implementation of individual placement and support (IPS), a model of supported employment for community behavioral health treatment programs, in New York State. We applied quantitative and qualitative methods to develop and evaluate three IPS e-learning modules. Throughout the ADDIE process, we conducted formative and summative evaluations and identified determinants of implementation using the Consolidated Framework for Implementation Research (CFIR). Formative evaluations consisted of qualitative feedback received from recipients and providers during early pilot work. The summative evaluation consisted of levels 1 and 2 (reaction to the training, self-reported knowledge, and practice change) quantitative and qualitative data and was guided by the Kirkpatrick model for training evaluation. Formative evaluation with key stakeholders identified a range of learning needs that informed the development of a pilot training program in IPS. Feedback on this pilot training program informed the design document of three e-learning modules on IPS: Introduction to IPS, IPS Job development, and Using the IPS Employment Resource Book . Each module was developed iteratively and provided an assessment of learning needs that informed successive modules. All modules were disseminated and evaluated through a learning management system. Summative evaluation revealed that learners rated the modules positively, and self-report of knowledge acquisition was high (mean range: 4.4-4.6 out of 5). About half of learners indicated that they would change their practice after watching the modules (range: 48-51%). All learners who completed the level 1 evaluation demonstrated 80% or better mastery of knowledge on the level 2 evaluation embedded in each module. The CFIR was used to identify implementation barriers and facilitators among the evaluation data which facilitated planning for subsequent implementation support activities in the IPS initiative. Instructional design approaches such as ADDIE may offer implementation scientists and practitioners a flexible and systematic approach for the development of e-learning modules as a single component or one strategy in a multifaceted approach for training in EBPs.

Investigating the Gap Between Estimated and Actual Energy Efficiency and Conservation Savings for Public Buildings Projects & Programs in United States

NASA Astrophysics Data System (ADS)

Qaddus, Muhammad Kamil

The gap between estimated and actual savings in energy efficiency and conservation (EE&C) projects or programs forms the problem statement for the scope of public and government buildings. This gap has been analyzed first on impact and then on process-level. On the impact-level, the methodology leads to categorization of the gap as 'Realization Gap'. It then views the categorization of gap within the context of past and current narratives linked to realization gap. On process-level, the methodology leads to further analysis of realization gap on process evaluation basis. The process evaluation criterion, a product of this basis is then applied to two different programs (DESEU and NYC ACE) linked to the scope of this thesis. Utilizing the synergies of impact and process level analysis, it offers proposals on program development and its structure using our process evaluation criterion. Innovative financing and benefits distribution structure is thus developed and will remain part of the proposal. Restricted Stakeholder Crowd Financing and Risk-Free Incentivized return are the products of proposed financing and benefit distribution structure respectively. These products are then complimented by proposing an alternative approach in estimating EE&C savings. The approach advocates estimation based on range-allocation rather than currently utilized unique estimated savings approach. The Way Ahead section thus explores synergy between financial and engineering ranges of energy savings as a multi-discipline approach for future research. Moreover, it provides the proposed program structure with risk aversion and incentive allocation while dealing with uncertainty. This set of new approaches are believed to better fill the realization gap between estimated and actual energy efficiency savings.

Systems design study of the Pioneer Venus spacecraft. Volume 2. Preliminary program development plan

NASA Technical Reports Server (NTRS)

1973-01-01

The preliminary development plan for the Pioneer Venus program is presented. This preliminary plan treats only developmental aspects that would have a significant effect on program cost. These significant development areas were: master program schedule planning; test planning - both unit and system testing for probes/orbiter/ probe bus; ground support equipment; performance assurance; and science integration Various test planning options and test method techniques were evaluated in terms of achieving a low-cost program without degrading mission performance or system reliability. The approaches studied and the methodology of the selected approach are defined.

Promoting a smokers' quitline in Ontario, Canada: an evaluation of an academic detailing approach.

PubMed

Kirst, Maritt; Schwartz, Robert

2015-06-01

This study assesses the impact of an academic detailing quitline promotional outreach program on integration of patient referrals to the quitline by fax in healthcare settings and quitline utilization in Ontario, Canada. The study employed a mixed methods approach for evaluation, with trend analysis of quitline administrative data from the year before program inception (2005) to 2011 and qualitative interviews with quitline stakeholders. Participants in the qualitative interviews included academic detailing program staff, regional tobacco control stakeholders and quitline promotion experts. Quantitative outcomes included the number of fax referral partners and fax referrals received, and quitline reach. Trends in proximal and distal outreach program outcomes were assessed. The qualitative data were analysed through a process of data coding involving the constant comparative technique derived from grounded theory methods. The study identified that the outreach program has had some success in integrating the fax referral program in healthcare settings through evidence of increased fax referrals since program inception. However, organizational barriers to program partner engagement have been encountered. While referral from health professionals through the fax referral programs has increased since the inception of the outreach program, the overall reach of the quitline has not increased. The study findings highlight that an academic detailing approach to quitline promotion can have some success in achieving increased fax referral program integration in healthcare settings. However, findings suggest that investment in a comprehensive promotional strategy, incorporating academic detailing, media and the provision of free cessation medications may be a more effective approach to quitline promotion. © The Author (2013). Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Debugging embedded computer programs. [tactical missile computers

NASA Technical Reports Server (NTRS)

Kemp, G. H.

1980-01-01

Every embedded computer program must complete its debugging cycle using some system that will allow real time debugging. Many of the common items addressed during debugging are listed. Seven approaches to debugging are analyzed to evaluate how well they treat those items. Cost evaluations are also included in the comparison. The results indicate that the best collection of capabilities to cover the common items present in the debugging task occurs in the approach where a minicomputer handles the environment simulation with an emulation of some kind representing the embedded computer. This approach can be taken at a reasonable cost. The case study chosen is an embedded computer in a tactical missile. Several choices of computer for the environment simulation are discussed as well as different approaches to the embedded emulator.

Experimental evaluation of model predictive control and inverse dynamics control for spacecraft proximity and docking maneuvers

NASA Astrophysics Data System (ADS)

Virgili-Llop, Josep; Zagaris, Costantinos; Park, Hyeongjun; Zappulla, Richard; Romano, Marcello

2018-03-01

An experimental campaign has been conducted to evaluate the performance of two different guidance and control algorithms on a multi-constrained docking maneuver. The evaluated algorithms are model predictive control (MPC) and inverse dynamics in the virtual domain (IDVD). A linear-quadratic approach with a quadratic programming solver is used for the MPC approach. A nonconvex optimization problem results from the IDVD approach, and a nonlinear programming solver is used. The docking scenario is constrained by the presence of a keep-out zone, an entry cone, and by the chaser's maximum actuation level. The performance metrics for the experiments and numerical simulations include the required control effort and time to dock. The experiments have been conducted in a ground-based air-bearing test bed, using spacecraft simulators that float over a granite table.

Procedures for Trade and Industrial Program Development.

ERIC Educational Resources Information Center

Campbell, Clifton P.

The instructional systems development (ISD) approach for the development and accomplishment of vocational training programs provides a methodology for gathering and analyzing job information, developing instructional materials in a variety of media, conducting instruction, and evaluating and improving the effectiveness of training programs. This…

Using Implementation and Program Theory to Examine Communication Strategies in National Wildlife Federation's Backyard Wildlife Habitat Program

ERIC Educational Resources Information Center

Palmer, Dain; Dann, Shari L.

2004-01-01

Our evaluative approach used implementation theory and program theory, adapted from Weiss (1998) to examine communication processes and results for a national wildlife habitat stewardship education program. Using a mail survey of 1427 participants certified in National Wildlife Federation's (NWF) Backyard Wildlife Habitat (BWH) program and a study…

Evaluating the Impact of the U.S. National Toxicology Program: A Case Study on Hexavalent Chromium

PubMed Central

Xie, Yun; Holmgren, Stephanie; Andrews, Danica M. K.; Wolfe, Mary S.

2016-01-01

Background: Evaluating the impact of federally funded research with a broad, methodical, and objective approach is important to ensure that public funds advance the mission of federal agencies. Objectives: We aimed to develop a methodical approach that would yield a broad assessment of National Toxicology Program’s (NTP’s) effectiveness across multiple sectors and demonstrate the utility of the approach through a case study. Methods: A conceptual model was developed with defined activities, outputs (products), and outcomes (proximal, intermediate, distal) and applied retrospectively to NTP’s research on hexavalent chromium (CrVI). Proximal outcomes were measured by counting views of and requests for NTP’s products by external stakeholders. Intermediate outcomes were measured by bibliometric analysis. Distal outcomes were assessed through Web and LexisNexis searches for documents related to legislation or regulation changes. Results: The approach identified awareness of NTP’s work on CrVI by external stakeholders (proximal outcome) and citations of NTP’s research in scientific publications, reports, congressional testimonies, and legal and policy documents (intermediate outcome). NTP’s research was key to the nation’s first-ever drinking water standard for CrVI adopted by California in 2014 (distal outcome). By applying this approach to a case study, the utility and limitations of the approach were identified, including challenges to evaluating the outcomes of a research program. Conclusions: This study identified a broad and objective approach for assessing NTP’s effectiveness, including methodological needs for more thorough and efficient impact assessments in the future. Citation: Xie Y, Holmgren S, Andrews DMK, Wolfe MS. 2017. Evaluating the impact of the U.S. National Toxicology Program: a case study on hexavalent chromium. Environ Health Perspect 125:181–188; http://dx.doi.org/10.1289/EHP21 PMID:27483499

An evidence-based approach to organization evaluation and change in human service organizations evaluation and program planning.

PubMed

Schalock, Robert L; Lee, Tim; Verdugo, Miguel; Swart, Kees; Claes, Claudia; van Loon, Jos; Lee, Chun-Shin

2014-08-01

The work described in this article focuses primarily on how human service organizations can use an evidence-based, self-assessment approach to organization evaluation to facilitate continuous quality improvement and organization change. Real-life examples are presented, strengths and challenges discussed, and future conceptual and measurement issues identified. Copyright © 2014 Elsevier Ltd. All rights reserved.

Increasing the use of evaluation data collection in an EPO program

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dordevic, M.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D.; Welti, R.; Davis, H. B.

2017-12-01

Over the past two years, the Incorporated Research Institutions for Seismology Education and Public Outreach (EPO) program has sought to increase the evaluation rigor of its programs and products. Specifically we sought to make evaluation an integral part of our work; enabling staff to demonstrate why we do the activities we do, enhancing the impact or our products/programs, and empowering staff to make evidence-based claims. The Collaborative Impact Analysis Method (Davis and Scalice, 2015) was selected as it allowed us to combine staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make improvements to the programs over time. A key part of the initial action plans has been the collection and analysis of new evaluation data. The most frequently used tools were surveys as they were relatively straightforward to implement and analyze, and could be adapted for different situations. Examples include: brand awareness, value of booth interactions, assessing community interest in a data app, and user surveys of social media and specific web pages. Other evaluation activities included beta testing of new software, and interviews with students and faculty involved in summer field experiences. The surveys have allowed us to document increased impact in some areas, to improve the usability of products and activities, and to provide baseline impact data. The direct involvement of staff in the process has helped staff appreciate the value of evaluation, but there are also challenges to this approach. Since many of the surveys are developed and conducted by EPO staff, rather than being primarily handled by the evaluator, the process takes considerably more staff time to implement. We are still determining how to best manage and present the data and analysis; our current approach is to post evaluation reports on our EPO website so that other groups may be able to benefit from our evaluation results. Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis Method. Abstract ED53D-0871, 2015 Fall Meeting, AGU.

Mentoring Undergraduate Students in Estuarine Research Experiences: Different Strokes for Different Folks

NASA Astrophysics Data System (ADS)

Moser, F. C.; Allen, M. R.; Montoya-Ospina, R. A.; Maldonado, P.; Barberena-Arias, M.; Olivo-Delgado, C.; Harris, L.; Pierson, J. J.; Alvarez, J. P.

2015-12-01

Here we consider how mentoring, both traditional and peer based, contributes to successful student outcomes in undergraduate research programs and we present several approaches to encourage positive mentor-mentee relationships. From several different research mentoring programs with undergraduates in Maryland and in Puerto Rico, we find that some mentoring techniques are universally useful, while others need to be tailored to a specific program and mentee population. Our programs differ in length, student composition, and student expectations, we find that success occurs across-the-board when mentors quickly establish rapport with their students and reach an early joint understanding of the program's requirements and the students' capabilities and needs through immersive orientations early in the program. Alternatively, mentors have to customize their approaches (e.g. simplify presentations of concepts, increase time for questions) when they encounter differences in student knowledge levels and cultural disconnects (e.g. language barriers, unfamiliarity with research labs and academia). Our current approach to improving and evaluating mentoring includes using a system of multiple mentor tiers (peer, near-peer, faculty, and program leaders), multiple qualitative and quantitative evaluations during the program, and post-research experience student outreach, all of which we believe improve student outcomes. Although we have measures of mentee success (e.g., presenting at national meetings, pursuing additional research experiences, applying to graduate school in marine science-related fields, etc.), we continue to look for additional short and long-term evaluation techniques that may help us to distinguish between the influence of mentoring and that of other program attributes (e.g. lab and field experiences, professional development seminars, ethics training, etc.) on student achievement.

Evaluation Program on the Implementation of Industrial Apprenticeship (Prakerin) in Electrical Engineering

NASA Astrophysics Data System (ADS)

Maulana, I.; Sumarto; Nurafiati, P.; Puspita, R. H.

2018-02-01

This research aims to find out the evaluation program of the Industrial apprenticeship (Prakerin) in electrical engineering. This research includes on four variables of CIPP. (1). Context (a). programme planning (b). design. (2). Input (a). readiness of students (b). performance of vocational education teachers (c). Facilities and infrastructure, (3). process (a). performance students (b). performance mentors, (4). Product (a). readiness of student work. This research is a type of program evaluation research with Stake model approach. Data collection methods used are questionnaires with closed questions and frequently asked questions.

Empowerment Evaluation: A Form of Self-Evaluation.

ERIC Educational Resources Information Center

Fetterman, David M.

Empowerment evaluation is an innovative approach that uses evaluation concepts and techniques to foster improvement and self-determination. Empowerment evaluation employs qualitative and quantitative methodologies. Although it can be applied to individuals and organizations, the usual focus is on programs. The value orientation of empowerment…

Creep-fatigue life prediction for engine hot section materials (isotropic)

NASA Technical Reports Server (NTRS)

Moreno, V.

1982-01-01

The objectives of this program are the investigation of fundamental approaches to high temperature crack initiation life prediction, identification of specific modeling strategies and the development of specific models for component relevant loading conditions. A survey of the hot section material/coating systems used throughout the gas turbine industry is included. Two material/coating systems will be identified for the program. The material/coating system designated as the base system shall be used throughout Tasks 1-12. The alternate material/coating system will be used only in Task 12 for further evaluation of the models developed on the base material. In Task II, candidate life prediction approaches will be screened based on a set of criteria that includes experience of the approaches within the literature, correlation with isothermal data generated on the base material, and judgements relative to the applicability of the approach for the complex cycles to be considered in the option program. The two most promising approaches will be identified. Task 3 further evaluates the best approach using additional base material fatigue testing including verification tests. Task 4 consists of technical, schedular, financial and all other reporting requirements in accordance with the Reports of Work clause.

Evaluation of a peer assessment approach for enhancing the organizational capacity of state injury prevention programs.

PubMed

Hunter, Wanda M; Schmidt, Ellen R; Zakocs, Ronda

2005-01-01

To conduct a formative and pilot impact evaluation of the State Technical Assessment Team (STAT) program, a visitation-based (visitatie) peer assessment program designed to enhance the organizational capacity of state health department injury prevention programs. The formative evaluation was based on observational, record review, and key informant interview data collected during the implementation of the first 7 STAT visits. Pilot impact data were derived from semi-structured interviews with state injury prevention personnel one year after the visit. Formative evaluation identified 6 significant implementation problems in the first visits that were addressed by the program planners, resulting in improvements to the STAT assessment protocol. Impact evaluation revealed that after one year, the 7 state injury prevention programs had acted on 81% of the recommendations received during their STAT visits. All programs reported gains in visibility and credibility within the state health department and increased collaboration and cooperation with other units and agencies. Other significant program advancements were also reported. Specific program standards and review procedures are important to the success of peer assessment programs such as STAT. Early impact evaluation suggests that peer assessment protocols using the visitatie model can lead to gains in organizational capacity.

Using Elite Athletes to Promote Drug Abstinence: Evaluation of a Single-Session School-Based Drug Use Prevention Program Delivered by Junior Hockey Players

ERIC Educational Resources Information Center

Wong, Jennifer

2016-01-01

School-based substance use prevention programs are a common method to approaching drug use in youths. Project SOS is a single-session drug prevention program developed by police officers and delivered by elite junior hockey players to students in grades 6 and 7. The current study evaluates the effects of Project SOS at achieving its objectives of…

Instructor Perspectives of Multiple-Choice Questions in Summative Assessment for Novice Programmers

ERIC Educational Resources Information Center

Shuhidan, Shuhaida; Hamilton, Margaret; D'Souza, Daryl

2010-01-01

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with…

Maximizing Student Learning: The Effects of a Comprehensive School-Based Program for Preventing Problem Behaviors.

ERIC Educational Resources Information Center

Nelson, J. Ron; Martella, Ronald M.; Marchand-Martella, Nancy

2002-01-01

A study evaluated a comprehensive school-wide program based on an effective behavioral support approach for preventing disruptive behaviors implemented in seven elementary schools. The program included a school-wide discipline program, tutoring, conflict resolution, and functional behavioral intervention plans. Schools showed positive effects on…

Training of Personnel to Work with Young Children in India.

ERIC Educational Resources Information Center

Kahalakdina, M.

1984-01-01

Describes recruitment procedures for child care workers in India. Includes a description of training programs, training needs and approaches, and curriculum and training procedures. Monitoring and evaluation of child welfare programs are discussed. Appendices include descriptions of and norms for training programs. (AS)

Helicopter Visual Aid System

NASA Technical Reports Server (NTRS)

Baisley, R. L.

1973-01-01

The results of an evaluation of police helicopter effectiveness revealed a need for improved visual capability. A JPL program developed a method that would enhance visual observation capability for both day and night usage and demonstrated the feasibility of the adopted approach. This approach made use of remote pointable optics, a display screen, a slaved covert searchlight, and a coupled camera. The approach was proved feasible through field testing and by judgement against evaluation criteria.

Small-grants programs: lessons from community-based approaches to changing nutrition environments.

PubMed

Johnson, Donna B; Smith, Lynne T; Bruemmer, Barbara

2007-02-01

Providing small grants to community organizations can be an effective way to encourage changes in the environment that support better nutrition. This is effective because these organizations can provide insights into their communities, ready-made relationships with community members, and the trust of the community. Small-grants programs are more likely to be successful when they are tailored to the needs of individual communities, led by organizations that have established reputations with the community, fully supported by the lead community organization, and engage local partners that complement the skills and resources of the lead organization. An evaluation of a small-grants program, Grants for Healthy Youth, found that grantees developed unique approaches to improving their community nutrition environments, gained experience and skills in program development, built partnerships, and received recognition for their project work. Grantees faced some common barriers, especially with program evaluation. Small-grants programs can be an effective way to improve community nutrition environments, but granting agencies need to provide effective technical assistance to communities throughout the process.

Dissemination research: the University of Wisconsin Population Health Institute.

PubMed

Remington, Patrick L; Moberg, D Paul; Booske, Bridget C; Ceraso, Marion; Friedsam, Donna; Kindig, David A

2009-08-01

Despite significant accomplishments in basic, clinical, and population health research, a wide gap persists between research discoveries (ie, what we know) and actual practice (ie, what we do). The University of Wisconsin Population Health Institute (Institute) researchers study the process and outcomes of disseminating evidence-based public health programs and policies into practice. This paper briefly describes the approach and experience of the Institute's programs in population health assessment, health policy, program evaluation, and education and training. An essential component of this dissemination research program is the active engagement of the practitioners and policymakers. Each of the Institute's programs conducts data collection, analysis, education, and dialogue with practitioners that is closely tied to the planning, implementation, and evaluation of programs and policies. Our approach involves a reciprocal exchange of knowledge with non-academic partners, such that research informs practice and practice informs research. Dissemination research serves an important role along the continuum of research and is increasingly recognized as an important way to improve population health by accelerating the translation of research into practice.

Long-Term Evaluation of a Life Skills Approach for Alcohol and Drug Abuse Prevention.

ERIC Educational Resources Information Center

Brochu, Serge; Souliere, Michelle

1988-01-01

Three-day life skills re-education program, embedded in 10-week new employee basic training had no long-term effects on alcohol and drug knowledge and attitudes. Findings suggest that primary prevention program targeting adults may be too late to affect alcohol and drug habits, life skills approach may work best in secondary prevention efforts,…

Including Students with Learning Disabilities in Mainstream Classes: A 2-Year Spanish Study Using a Collaborative Approach to Intervention.

ERIC Educational Resources Information Center

Cardona, Cristina

This study evaluated the effectiveness of an inclusive program for students (grades 2 through 4) with mild learning disabilities (LD) in four elementary schools in Spain. It compared two support strategies, a within-class integrated program versus resource room approach with students having either high or low learning potential. The study also…

Program Planners’ Perspectives of Promotora Roles, Recruitment, and Selection

PubMed Central

Koskan, Alexis; Hilfinger Messias, DeAnne K.; Friedman, Daniela B.; Brandt, Heather M.; Walsemann, Katrina M.

2013-01-01

Objective Program planners work with promotoras (the Spanish term for female community health workers) to reduce health disparities among underserved populations. Based on the Role-Outcomes Linkage Evaluation Model for Community Health Workers (ROLES) conceptual model, we explored how program planners conceptualized the promotora role and the approaches and strategies they used to recruit, select, and sustain promotoras. Design We conducted semi-structured, in-depth interviews with a purposive convenience sample of 24 program planners, program coordinators, promotora recruiters, research principal investigators, and other individuals who worked closely with promotoras on United States-based health programs for Hispanic women (ages 18 and older). Results Planners conceptualized the promotora role based on their personal experiences and their understanding of the underlying philosophical tenets of the promotora approach. Recruitment and selection methods reflected planners’ conceptualizations and experiences of promotoras as paid staff or volunteers. Participants described a variety of program planning and implementation methods. They focused on sustainability of the programs, the intended health behavior changes or activities, and the individual promotoras. Conclusion To strengthen health programs employing the promotora delivery model, job descriptions should delineate role expectations and boundaries and better guide promotora evaluations. We suggest including additional components such as information on funding sources, program type and delivery, and sustainability outcomes to enhance the ROLES conceptual model. The expanded model can be used to guide program planners in the planning, implementing, and evaluating of promotora health programs. PMID:23039847

An Empirical Examination of Validity in Evaluation

ERIC Educational Resources Information Center

Peck, Laura R.; Kim, Yushim; Lucio, Joanna

2012-01-01

This study addresses validity issues in evaluation that stem from Ernest R. House's book, "Evaluating With Validity". The authors examine "American Journal of Evaluation" articles from 1980 to 2010 that report the results of policy and program evaluations. The authors classify these evaluations according to House's "major approaches" typology…

Participatory Evaluation as Seen in a Vygotskian Framework

ERIC Educational Resources Information Center

Higa, Terry Ann F.; Brandon, Paul R.

2008-01-01

In participatory evaluations of K-12 programs, evaluators develop school faculty's and administrators' evaluation capacity by training them to conduct evaluation tasks and providing consultation while the tasks are conducted. A strong case can be made that the capacity building in these evaluations can be examined using a Vygotskian approach. We…

An Integrated Constraint Programming Approach to Scheduling Sports Leagues with Divisional and Round-robin Tournaments

DOE Office of Scientific and Technical Information (OSTI.GOV)

Carlsson, Mats; Johansson, Mikael; Larson, Jeffrey

Previous approaches for scheduling a league with round-robin and divisional tournaments involved decomposing the problem into easier subproblems. This approach, used to schedule the top Swedish handball league Elitserien, reduces the problem complexity but can result in suboptimal schedules. This paper presents an integrated constraint programming model that allows to perform the scheduling in a single step. Particular attention is given to identifying implied and symmetry-breaking constraints that reduce the computational complexity significantly. The experimental evaluation of the integrated approach takes considerably less computational effort than the previous approach.

Quasi-experimental evaluation without regression analysis.

PubMed

Rohrer, James E

2009-01-01

Evaluators of public health programs in field settings cannot always randomize subjects into experimental or control groups. By default, they may choose to employ the weakest study design available: the pretest, posttest approach without a comparison group. This essay argues that natural experiments involving comparison groups are within reach of public health program managers. Methods for analyzing natural experiments are discussed.

An Approach for Selecting a Theoretical Framework for the Evaluation of Training Programs

ERIC Educational Resources Information Center

Tasca, Jorge Eduardo; Ensslin, Leonardo; Ensslin, Sandra Rolim; Alves, Maria Bernardete Martins

2010-01-01

Purpose: This research paper proposes a method for selecting references related to a research topic, and seeks to exemplify it for the case of a study evaluating training programs. The method is designed to identify references with high academic relevance in databases accessed via the internet, using a bibliometric analysis to sift the selected…

A Team Approach to Management by Objectives with Special Emphasis on Managerial Self-Evaluation.

ERIC Educational Resources Information Center

Alvir, Howard P.

This kit contains everything needed to explain, criticize and plan, simulate, and evaluate a management by objectives (MBO) program. The kit has been field tested in state agencies, schools, businesses, and volunteer organizations. Rather than present only the strengths of MBO, this program defines MBO, presents its strong points in discussing the…

Comparison of Outcomes for Youth Apprenticeship Projects and Youth Career Development Projects. Supplementary Report.

ERIC Educational Resources Information Center

Richards, James M., Jr.; And Others

The New Youth Initiatives in Apprenticeship Program (YAP) was compared with the Youth Career Development Program (YCD). Data for 1979 and 1980 came from an evaluation of YAP projects by CSR, Incorporated, and an evaluation of the YCD projects by the Educational Testing Service. A multiple regression approach was used to compare student…

Extension's Efforts to Help Kids Be SAFE: Evaluation of a Statewide Bullying Prevention Program

ERIC Educational Resources Information Center

Duke, Adrienne; Norton, Jessica

2017-01-01

The evaluation reported in this article examined the effectiveness of a statewide bullying prevention program, Be SAFE. Be SAFE involves use of a positive youth development approach to influence peer groups rather than individual bullies or victims. Through the use of pre- and postprogram questionnaires, we found increases in youths' knowledge of…

Making Room for Foster Children: A Criterion-Referenced Approach to ESEA Title I Migrant Program Evaluation.

ERIC Educational Resources Information Center

Oxford, Rebecca L.; And Others

The Washington state Title I Migrant Program Evaluation project is a feasibility study designed to assess the suitability of existing normed criterion referenced tests to measuring mathematics achievement at grades four, five, and six. Objectives include judging the technical qualities and content of several normed criterion referenced tests;…

Research on teacher education programs: logic model approach.

PubMed

Newton, Xiaoxia A; Poon, Rebecca C; Nunes, Nicole L; Stone, Elisa M

2013-02-01

Teacher education programs in the United States face increasing pressure to demonstrate their effectiveness through pupils' learning gains in classrooms where program graduates teach. The link between teacher candidates' learning in teacher education programs and pupils' learning in K-12 classrooms implicit in the policy discourse suggests a one-to-one correspondence. However, the logical steps leading from what teacher candidates have learned in their programs to what they are doing in classrooms that may contribute to their pupils' learning are anything but straightforward. In this paper, we argue that the logic model approach from scholarship on evaluation can enhance research on teacher education by making explicit the logical links between program processes and intended outcomes. We demonstrate the usefulness of the logic model approach through our own work on designing a longitudinal study that focuses on examining the process and impact of an undergraduate mathematics and science teacher education program. Copyright © 2012 Elsevier Ltd. All rights reserved.

Development of a Peer Teaching-Assessment Program and a Peer Observation and Evaluation Tool

PubMed Central

Trujillo, Jennifer M.; Barr, Judith; Gonyeau, Michael; Van Amburgh, Jenny A.; Matthews, S. James; Qualters, Donna

2008-01-01

Objectives To develop a formalized, comprehensive, peer-driven teaching assessment program and a valid and reliable assessment tool. Methods A volunteer taskforce was formed and a peer-assessment program was developed using a multistep, sequential approach and the Peer Observation and Evaluation Tool (POET). A pilot study was conducted to evaluate the efficiency and practicality of the process and to establish interrater reliability of the tool. Intra-class correlation coefficients (ICC) were calculated. Results ICCs for 8 separate lectures evaluated by 2-3 observers ranged from 0.66 to 0.97, indicating good interrater reliability of the tool. Conclusion Our peer assessment program for large classroom teaching, which includes a valid and reliable evaluation tool, is comprehensive, feasible, and can be adopted by other schools of pharmacy. PMID:19325963

Evaluation of the FORETELL consortium operational test : weather information for surface transportation, evaluation strategy

DOT National Transportation Integrated Search

1998-07-01

The purpose of the independent evaluation is to assess the effectiveness of the FORETELL Program in achieving certain ARTS goals and objectives. Independent evaluations of ITS Operational Tests require a well documented structured approach to ensure ...

Evaluating Educational Programmes: The Need and the Response.

ERIC Educational Resources Information Center

Stake, Robert E.

This survey of recent developments in educational program evaluation is intended for persons who commission, implement, direct, or carry out evaluation studies. The attitudes of government officials, educators, and researchers toward assessment and their own evaluation needs are discussed. Various approaches to evaluation are briefly described;…

Evaluation Theory, Models, and Applications

ERIC Educational Resources Information Center

Stufflebeam, Daniel L.; Shinkfield, Anthony J.

2007-01-01

"Evaluation Theory, Models, and Applications" is designed for evaluators and students who need to develop a commanding knowledge of the evaluation field: its history, theory and standards, models and approaches, procedures, and inclusion of personnel as well as program evaluation. This important book shows how to choose from a growing…

[In search of results of technical cooperation].

PubMed

Jourdan Hidalgo, L; Manuel Sotelo, J

1999-06-01

For the Pan American Health Organization (PAHO), technical cooperation (TC) is the process by which the PAHO Member States work with the Organization, as equal partners, to identify and reach their own health goals and to promote self-sufficiency in health development, through programs that respond to those countries' needs and national priorities. Since 1978, PAHO has used the American Regional Planning, Programming, Monitoring, and Evaluation System (AMPES) to establish management procedures and to facilitate decision-making in health TC. As part of AMPES, PAHO uses a "logical approach to project management" to structure the work program of the Organization and to identify the expected results from TC activities and TC resource investments. This project management approach, which replaces the "functional approaches" system used to date, also helps establish a causative relationship between the programmed activities and the results that PAHO expects, and between the activities and the hoped-for outcomes in the countries. As part of an ongoing process of rethinking international health TC, several years ago PAHO began a four-phase study on the usefulness and validity of functional approaches and on the need to propose new ones or to modify existing ones. The results of the initial phase showed it was difficult to classify the activities because the functional-approaches categories were not mutually exclusive and the TC activities were complex. Further, the expected results did not specify the product for which the PAHO Secretariat was accountable within a certain time frame nor the Secretariat's level of responsibility. Thus, a new and more flexible classification of expected results was proposed, with the following categories: cooperation networks and alliances; surveillance and information systems; standards and guidelines; research and evaluation studies; plans, projects, and policies; methods, models, and technologies; training programs; promotional campaigns and advocacy; and direct support. In the second phase of the study, it was concluded that the proposed classification system made it possible to more precisely identify the products of PAHO technical cooperation projects, the Organization's degree of responsibility, and the bases for estimating needed resources. The new system could also facilitate monitoring and evaluation. In addition, the third phase of analyzing the functional approaches has begun. Its objective is to evaluate the effect of technical cooperation based on the changes incorporated in the programming of activities.

Assessing the effectiveness of a place-based conservation education program by applying utilization-focused evaluation

NASA Astrophysics Data System (ADS)

Flowers, Alice Blood

Lack of personal connection to the natural world by most American youth builds reason for assessing effectiveness of conservation education programs. Place-based learning is important in helping youth understand how their personal and societal well-being are linked and dependent upon their local habitats. Across Montana 2277 students in grades 3--10 participate in an interactive year long fishing education program with their teachers called Hooked on Fishing (HOF). The purpose of my study was to assess the effectiveness of HOF, a place-based conservation education program established in 1996, and modeled after the national Hooked on Fishing, Not on Drugs program. Using a quasi-experimental nonequivalent group study design, students received a pre-survey during the beginning of the program, a post-survey after the program, and an extended post-survey 12 to 14 weeks later. Teachers voluntarily participated in an Internet survey during May 2006, and program instructors voluntarily participated in a structured open-ended telephone interview in June 2006. A key component of my study was the decision to conduct the evaluation process using an approach which included stakeholders in the development of the instruments to measure student outcomes. This approach is called utilization-focused evaluation and was developed by Michael Q. Patton. The motive of this approach is to promote the usability of the evaluation results. The results are considered to have a better chance to be applied by the program stakeholders to not only gauge program effectiveness, but to be used to improve the program. Two research questions were: (1) does the frequency of outdoor experiences have significant affects on students' knowledge, skills, attitudes, and intended stewardship behaviors; and (2) does improved knowledge of local natural resources have significant affects on students' skills, attitudes and intended stewardship behavior. Nonparametric statistical analyses calculated statistical significant results for most knowledge and skill outcomes in a positive direction of change with 2--3 HOF outdoor experiences. Attitudinal and intended behavior outcomes did not show similar results. Internet teacher survey and instructor interviews provided qualitative depth and insight to student self-reported responses.

Methodological Innovations in Public Health Education: Transdisciplinary Problem Solving

PubMed Central

Lawlor, Edward F.; Sebert-Kuhlmann, Anne K.; McBride, Timothy D.

2015-01-01

In 2008, the faculty of the Brown School at Washington University in St. Louis designed a Master of Public Health program centered on transdisciplinary problem solving in public health. We have described the rationale for our approach, guiding principles and pedagogy for the program, and specific transdisciplinary competencies students acquire. We have explained how transdisciplinary content has been organized and delivered, how the program is being evaluated, and how we have demonstrated the feasibility of this approach for a Master of Public Health degree. PMID:25706031

Methodological innovations in public health education: transdisciplinary problem solving.

PubMed

Lawlor, Edward F; Kreuter, Matthew W; Sebert-Kuhlmann, Anne K; McBride, Timothy D

2015-03-01

In 2008, the faculty of the Brown School at Washington University in St. Louis designed a Master of Public Health program centered on transdisciplinary problem solving in public health. We have described the rationale for our approach, guiding principles and pedagogy for the program, and specific transdisciplinary competencies students acquire. We have explained how transdisciplinary content has been organized and delivered, how the program is being evaluated, and how we have demonstrated the feasibility of this approach for a Master of Public Health degree.

Assessing Conformity to Standards for Treatment Foster Care.

ERIC Educational Resources Information Center

Farmer, Elizabeth M. Z.; Burns, Barbara J.; Dubs, Melanie S.; Thompson, Shealy

2002-01-01

This study examined conformity to the Program Standards for Treatment Foster Care among 42 statewide programs. Findings suggest fair to good overall conformity, with considerable variation among programs. A discussion of methodological and substantive considerations for future research and evaluation using this approach is included. (Contains…

A Computer-aided Learning Exercise in Spectrophotometry.

ERIC Educational Resources Information Center

Pamula, Frederick

1994-01-01

Discusses the use of a computer simulation program in teaching the concepts of spectrophotometry. Introduces several parts of the program and program usage. Presents an assessment activity to evaluate students' mastery of material. Concludes with the advantages of this approach to the student and to the assessor. (ASK)

AN ORGANIZATION GUIDE TO POLLUTION PREVENTION

EPA Science Inventory

This Pollution Prevention (P2) Guide provides information to help organizations get P2 programs started on t re-evaluate existing P2 programs. It presents an alternative method for working on P2 projects and four approaches to implementing a P2 program in an organization. The int...

Earth Sciences Data and Information System (ESDIS) program planning and evaluation methodology development

NASA Technical Reports Server (NTRS)

Dickinson, William B.

1995-01-01

An Earth Sciences Data and Information System (ESDIS) Project Management Plan (PMP) is prepared. An ESDIS Project Systems Engineering Management Plan (SEMP) consistent with the developed PMP is also prepared. ESDIS and related EOS program requirements developments, management and analysis processes are evaluated. Opportunities to improve the effectiveness of these processes and program/project responsiveness to requirements are identified. Overall ESDIS cost estimation processes are evaluated, and recommendations to improve cost estimating and modeling techniques are developed. ESDIS schedules and scheduling tools are evaluated. Risk assessment, risk mitigation strategies and approaches, and use of risk information in management decision-making are addressed.

Satellite Power System (SPS) financial/management scenarios

NASA Technical Reports Server (NTRS)

Vajk, J. P.

1978-01-01

The possible benefits of a Satellite Power System (SPS) program, both domestically and internationally, justify detailed and imaginative investigation of the issues involved in financing and managing such a large-scale program. In this study, ten possible methods of financing a SPS program are identified ranging from pure government agency to private corporations. The following were analyzed and evaluated: (1) capital requirements for SPS; (2) ownership and control; (3) management principles; (4) organizational forms for SPS; (5) criteria for evaluation; (6) detailed description and preliminary evaluation of alternatives; (7) phased approaches; and (8) comparative evaluation. Key issues and observations and recommendations for further study are also presented.

Deductive Evaluation: Formal Code Analysis With Low User Burden

NASA Technical Reports Server (NTRS)

Di Vito, Ben. L

2016-01-01

We describe a framework for symbolically evaluating iterative C code using a deductive approach that automatically discovers and proves program properties. Although verification is not performed, the method can infer detailed program behavior. Software engineering work flows could be enhanced by this type of analysis. Floyd-Hoare verification principles are applied to synthesize loop invariants, using a library of iteration-specific deductive knowledge. When needed, theorem proving is interleaved with evaluation and performed on the fly. Evaluation results take the form of inferred expressions and type constraints for values of program variables. An implementation using PVS (Prototype Verification System) is presented along with results for sample C functions.

A mandala of faculty development: using theory-based evaluation to explore contexts, mechanisms and outcomes.

PubMed

Onyura, Betty; Ng, Stella L; Baker, Lindsay R; Lieff, Susan; Millar, Barbara-Ann; Mori, Brenda

2017-03-01

Demonstrating the impact of faculty development, is an increasingly mandated and ever elusive goal. Questions have been raised about the adequacy of current approaches. Here, we integrate realist and theory-driven evaluation approaches, to evaluate an intensive longitudinal program. Our aim is to elucidate how faculty development can work to support a range of outcomes among individuals and sub-systems in the academic health sciences. We conducted retrospective framework analysis of qualitative focus group data gathered from 79 program participants (5 cohorts) over a 10-year period. Additionally, we conducted follow-up interviews with 15 alumni. We represent the interactive relationships among contexts, mechanisms, and outcomes as a "mandala" of faculty development. The mandala illustrates the relationship between the immediate program context, and the broader institutional context of academic health sciences, and identifies relevant change mechanisms. Four primary mechanisms were collaborative-reflection, self-reflection and self-regulation, relationship building, and pedagogical knowledge acquisition. Individual outcomes, including changed teaching practices, are described. Perhaps most interestingly, secondary mechanisms-psychological and structural empowerment-contributed to institutional outcomes through participants' engagement in change leadership in their local contexts. Our theoretically informed evaluation approach models how faculty development, situated in appropriate institutional contexts, can trigger mechanisms that yield a range of benefits for faculty and their institutions. The adopted methods hold potential as a way to demonstrate the often difficult-to-measure outcomes of educational programs, and allow for critical examination as to how and whether faculty development programs can accomplish their espoused goals.

Integrating Intimate Partner Violence Screening and Counseling in a Family Planning Clinic: Evaluation of a Pilot Project in Conakry, Guinea.

PubMed

Samandari, Ghazaleh; Delamou, Alexandre; Traore, Pernamou; Diallo, Fatoumata Guilinty; Millimono, Sita; Camara, Bienvenu Salim; Laffe, Kira; Verani, Fabio; Tolliver, Maimouna

2016-06-01

Few programs exist to address Intimate Partner Violence (IPV) in Guinea. In 2014, Engender Health, in partnership with the local health authorities in Conakry, Guinea, piloted an integrated approach to IPV screening and counseling, within an existing family planning clinic. This article describes both the process of formulating and implementing this approach, as well as the results of an evaluation of the program. From January to June of 2014, Engender Health staff trained midwives at the Conakry International Planned Parenthood Federation family planning clinic staff in screening and counseling client for IPV. Program evaluators used project records, interview with program staff (n=3), midwives (n=3) and client exit interviews (n=53) to measure the outcomes of this pilot project. Regardless of their IPV status, clients appreciated having a venue in which to discuss IPV. Program staff also felt empowered by the additional training and support for IPV screening. The evaluation yielded valuable suggestions for improvement, including more time for staff training and mock client interview practice, additional skills in counseling, and stronger referral links for women who screen positive for IPV. Integrating IPV screening into family planning services is an important and feasible method for reaching vulnerable women with IPV services.

Evaluating a culturally tailored peer-mentoring and education pilot intervention among Chinese breast cancer survivors using a mixed-methods approach.

PubMed

Lu, Qian; You, Jin; Man, Jenny; Loh, Alice; Young, Lucy

2014-11-01

To evaluate a social support intervention that was culturally tailored for Chinese Americans who face many challenges because of cultural and linguistic barriers. Intervention with a one-group pre- or post-test design, mixed methods, and a community-based participatory research (CBPR) approach. Southern California. 14 Chinese American breast cancer survivors post-treatment and eight breast cancer peer mentors. The intervention was a 10-week program to provide emotional and informational support through peer mentoring and education. Health outcomes were assessed before and after the intervention. Eight weekly process evaluations and two focus group interviews also were conducted. Depressive and anxiety symptoms. The program was associated with a decrease in depressive symptoms. Participants valued the program highly. Inductive analysis suggested possible mechanisms for effectiveness, such as reducing stigma, empowerment, and increased sense of belonging. The peer-mentoring and education program has the potential to serve as a model intervention for ethnic minorities. Mixed methods and CBPR are valuable in evaluating pilot interventions with minorities. Focusing on relationships may be fruitful for designing novel interventions for cancer survivors from collectivistic cultures. Peer-mentoring and education programs can be integrated into communities and clinics to improve care for underserved minority cancer survivors and to reduce health disparities.

Evaluations of Sexual Assault Prevention Programs in Military Settings: A Synthesis of the Research Literature.

PubMed

Orchowski, Lindsay M; Berry-Cabán, Cristóbal S; Prisock, Kara; Borsari, Brian; Kazemi, Donna M

2018-03-01

The prevention of sexual assault (SA) in the U.S. military is a significant priority. This study applied the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to a literature search that identified research evaluating SA prevention programs conducted within military settings. Only six studies published between 2005 and 2016 met criteria for inclusion in the review. Studies demonstrated high heterogeneity in the: (1) conceptual framework of the prevention approach; (2) target population and timing of administration; (3) study recruitment methods; (4) methodological design; (5) method of delivery, program dosage and theory of change; and (6) outcome administration and efficacy. Scientific rigor according to the Oxford Center for Evidence-based Medicine was also variable. Several gaps in the research base were identified. Specifically, research evaluating SA prevention programs have only been conducted among U.S. Army and U.S. Navy samples. Most studies did not examine whether program participation was associated with reductions in rates of sexual violence. Studies also lacked utilization of a long-term follow-up period. Additionally, studies did not reflect the types of SA prevention programs currently being implemented in military settings. Taken together, further research is needed to enhance the evidence base for SA prevention in the military, and to evaluate the effectiveness of the approaches currently being conducted with service members.

Approaches and impact of non-academic research capacity strengthening training models in sub-Saharan Africa: a systematic review.

PubMed

Mugabo, Lambert; Rouleau, Dominique; Odhiambo, Jackline; Nisingizwe, Marie Paul; Amoroso, Cheryl; Barebwanuwe, Peter; Warugaba, Christine; Habumugisha, Lameck; Hedt-Gauthier, Bethany L

2015-06-09

Research is essential to identify and prioritize health needs and to develop appropriate strategies to improve health outcomes. In the last decade, non-academic research capacity strengthening trainings in sub-Saharan Africa, coupled with developing research infrastructure and the provision of individual mentorship support, has been used to build health worker skills. The objectives of this review are to describe different training approaches to research capacity strengthening in sub-Saharan Africa outside academic programs, assess methods used to evaluate research capacity strengthening activities, and learn about the challenges facing research capacity strengthening and the strategies/innovations required to overcome them. The PubMed database was searched using nine search terms and articles were included if 1) they explicitly described research capacity strengthening training activities, including information on program duration, target audience, immediate program outputs and outcomes; 2) all or part of the training program took place in sub-Saharan African countries; 3) the training activities were not a formal academic program; 4) papers were published between 2000 and 2013; and 5) both abstract and full paper were available in English. The search resulted in 495 articles, of which 450 were retained; 14 papers met all inclusion criteria and were included and analysed. In total, 4136 people were trained, of which 2939 were from Africa. Of the 14 included papers, six fell in the category of short-term evaluation period and eight in the long-term evaluation period. Conduct of evaluations and use of evaluation frameworks varied between short and long term models and some trainings were not evaluated. Evaluation methods included tests, surveys, interviews, and systems approach matrix. Research capacity strengthening activities in sub-Saharan Africa outside of academic settings provide important contributions to developing in-country capacity to participate in and lead research. Institutional support, increased funds, and dedicated time for research activities are critical factors that lead to the development of successful programs. Further, knowledge sharing through scientific articles with sufficient detail is needed to enable replication of successful models in other settings.

Twinning "Practices of Change" with "Theory of Change": Room for Emergence in Advocacy Evaluation

ERIC Educational Resources Information Center

Arensman, Bodille; van Waegeningh, Cornelie; van Wessel, Margit

2018-01-01

Theory of change (ToC) is currently "the" approach for the evaluation and planning of international development programs. This approach is considered especially suitable for complex interventions. We question this assumption and argue that ToC's focus on cause-effect logic and intended outcomes does not do justice to the recursive nature…

Evaluation Approaches to Intelligent Computer-Assisted Instruction. Testing Study Group: The Impact of Advances in Artificial Intelligence on Test Development.

ERIC Educational Resources Information Center

Baker, Eva L.

Some special problems associated with evaluating intelligent computer-assisted instruction (ICAI) programs are addressed. This paper intends to describe alternative approaches to the assessment and improvement of such applications and to provide examples of efforts undertaken and shortfalls. Issues discussed stem chiefly from the technical demands…

Connected Vehicle Pilot Deployment Program Independent Evaluation: Mobility, Environmental, and Public Agency Efficiency Refined Evaluation Plan - New York City

DOT National Transportation Integrated Search

2018-03-01

The purpose of this report is to provide a refined evaluation plan detailing the approach to be used by the Texas A&M Transportation Institute Connected Vehicle Pilot Deployment Evaluation Team for evaluating the mobility, environmental, and public a...

Training Evaluation: An Analysis of the Stakeholders' Evaluation Needs

ERIC Educational Resources Information Center

Guerci, Marco; Vinante, Marco

2011-01-01

Purpose: In recent years, the literature on program evaluation has examined multi-stakeholder evaluation, but training evaluation models and practices have not generally taken this problem into account. The aim of this paper is to fill this gap. Design/methodology/approach: This study identifies intersections between methodologies and approaches…

Estimating the Counterfactual Impact of Conservation Programs on Land Cover Outcomes: The Role of Matching and Panel Regression Techniques

PubMed Central

Jones, Kelly W.; Lewis, David J.

2015-01-01

Deforestation and conversion of native habitats continues to be the leading driver of biodiversity and ecosystem service loss. A number of conservation policies and programs are implemented—from protected areas to payments for ecosystem services (PES)—to deter these losses. Currently, empirical evidence on whether these approaches stop or slow land cover change is lacking, but there is increasing interest in conducting rigorous, counterfactual impact evaluations, especially for many new conservation approaches, such as PES and REDD, which emphasize additionality. In addition, several new, globally available and free high-resolution remote sensing datasets have increased the ease of carrying out an impact evaluation on land cover change outcomes. While the number of conservation evaluations utilizing ‘matching’ to construct a valid control group is increasing, the majority of these studies use simple differences in means or linear cross-sectional regression to estimate the impact of the conservation program using this matched sample, with relatively few utilizing fixed effects panel methods—an alternative estimation method that relies on temporal variation in the data. In this paper we compare the advantages and limitations of (1) matching to construct the control group combined with differences in means and cross-sectional regression, which control for observable forms of bias in program evaluation, to (2) fixed effects panel methods, which control for observable and time-invariant unobservable forms of bias, with and without matching to create the control group. We then use these four approaches to estimate forest cover outcomes for two conservation programs: a PES program in Northeastern Ecuador and strict protected areas in European Russia. In the Russia case we find statistically significant differences across estimators—due to the presence of unobservable bias—that lead to differences in conclusions about effectiveness. The Ecuador case illustrates that if time-invariant unobservables are not present, matching combined with differences in means or cross-sectional regression leads to similar estimates of program effectiveness as matching combined with fixed effects panel regression. These results highlight the importance of considering observable and unobservable forms of bias and the methodological assumptions across estimators when designing an impact evaluation of conservation programs. PMID:26501964

Estimating the Counterfactual Impact of Conservation Programs on Land Cover Outcomes: The Role of Matching and Panel Regression Techniques.

PubMed

Jones, Kelly W; Lewis, David J

2015-01-01

Deforestation and conversion of native habitats continues to be the leading driver of biodiversity and ecosystem service loss. A number of conservation policies and programs are implemented--from protected areas to payments for ecosystem services (PES)--to deter these losses. Currently, empirical evidence on whether these approaches stop or slow land cover change is lacking, but there is increasing interest in conducting rigorous, counterfactual impact evaluations, especially for many new conservation approaches, such as PES and REDD, which emphasize additionality. In addition, several new, globally available and free high-resolution remote sensing datasets have increased the ease of carrying out an impact evaluation on land cover change outcomes. While the number of conservation evaluations utilizing 'matching' to construct a valid control group is increasing, the majority of these studies use simple differences in means or linear cross-sectional regression to estimate the impact of the conservation program using this matched sample, with relatively few utilizing fixed effects panel methods--an alternative estimation method that relies on temporal variation in the data. In this paper we compare the advantages and limitations of (1) matching to construct the control group combined with differences in means and cross-sectional regression, which control for observable forms of bias in program evaluation, to (2) fixed effects panel methods, which control for observable and time-invariant unobservable forms of bias, with and without matching to create the control group. We then use these four approaches to estimate forest cover outcomes for two conservation programs: a PES program in Northeastern Ecuador and strict protected areas in European Russia. In the Russia case we find statistically significant differences across estimators--due to the presence of unobservable bias--that lead to differences in conclusions about effectiveness. The Ecuador case illustrates that if time-invariant unobservables are not present, matching combined with differences in means or cross-sectional regression leads to similar estimates of program effectiveness as matching combined with fixed effects panel regression. These results highlight the importance of considering observable and unobservable forms of bias and the methodological assumptions across estimators when designing an impact evaluation of conservation programs.

Comprehensive Family-Centered Training Programs: Five Comparative Case Studies. Evaluation of the Mountain-Plains Education & Economic Development Program, Inc.

ERIC Educational Resources Information Center

Bale, Richard L.; Sprague, C. Fremont

The Mountain-Plains Education & Economic Development Program, which exemplifies the comprehensive, residential family-centered approach to serving the economically disadvantaged, was compared to four similar programs in the United States: Arizona Job Colleges (AJC) in Arizona; Madera Employment Training Center (METC) in California; Manpower,…

From Knowledge to Action: Tips for Encouraging and Measuring Program-Related Behavior Change

ERIC Educational Resources Information Center

Chazdon, Scott; Horntvedt, Jody; Templin, Elizabeth

2016-01-01

It is challenging to document the behavior changes that result from Extension programming. This article describes an evaluation method we call the "action items method." Unlike other approaches for measuring behavior change, this method requires program participants to define their own action plans as part of a program and then asks them…

Combining Primary Prevention and Risk Reduction Approaches in Sexual Assault Protection Programming

ERIC Educational Resources Information Center

Menning, Chadwick; Holtzman, Mellisa

2015-01-01

Objective: The object of this study is to extend prior evaluations of Elemental, a sexual assault protection program that combines primary prevention and risk reduction strategies within a single program. Participants and Methods: During 2012 and 2013, program group and control group students completed pretest, posttest, and 6-week and 6-month…

Evaluating an Experimental Audio-Visual Module Programmed to Teach a Basic Anatomical and Physiological System.

ERIC Educational Resources Information Center

Federico, Pat-Anthony

The learning efficiency and effectiveness of teaching an anatomical and physiological system to Air Force enlisted trainees utilizing an experimental audiovisual programed module was compared to that of a commercial linear programed text. It was demonstrated that the audiovisual programed approach to training was more efficient than and equally as…

Digital electronic engine control F-15 overview

NASA Technical Reports Server (NTRS)

Kock, B.

1984-01-01

A flight test evaluation of the digital elctronic engine control (DEEC) system was conducted. An overview of the flight program is presented. The roles of the participating parties, the system, and the flight program objectives are described. The test program approach is discussed, and the engine performance benefits are summarized. A description of the follow-on programs is included.

Cognitive Academic Language Learning Approach (Project CALLA), Community School District 2 Special Alternative Instruction Program. Final Evaluation Report, 1992-93. OREA Report.

ERIC Educational Resources Information Center

Lynch, Joanne

Cognitive Academic Language Learning Approach (Project CALLA) was a federally funded program serving 960 limited-English-proficient students in 10 Manhattan (New York) elementary schools in 1992-93 its third year of operation. The project provided instruction in English as a Second Language (ESL), mathematics, science, and social studies in…

Development and evaluation of a leadership training program for public health emergency response: results from a Chinese study.

PubMed

Wang, Chongjian; Wei, Sheng; Xiang, Hao; Wu, Jing; Xu, Yihua; Liu, Li; Nie, Shaofa

2008-10-30

Since the 9/11 attack and severe acute respiratory syndrome (SARS), the development of qualified and able public health leaders has become a new urgency in building the infrastructure needed to address public health emergencies. Although previous studies have reported that the training of individual leaders is an important approach, the systemic and scientific training model need further improvement and development. The purpose of this study was to develop, deliver, and evaluate a participatory leadership training program for emergency response. Forty-one public health leaders (N = 41) from five provinces completed the entire emergency preparedness training program in China. The program was evaluated by anonymous questionnaires and semi-structured interviews held prior to training, immediately post-training and 12-month after training (Follow-up). The emergency preparedness training resulted in positive shifts in knowledge, self-assessment of skills for public health leaders. More than ninety-five percent of participants reported that the training model was scientific and feasible. Moreover, the response of participants in the program to the avian influenza outbreak, as well as the planned evaluations for this leadership training program, further demonstrated both the successful approaches and methods and the positive impact of this integrated leadership training initiative. The emergency preparedness training program met its aims and objectives satisfactorily, and improved the emergency capability of public health leaders. This suggests that the leadership training model was effective and feasible in improving the emergency preparedness capability.

Evaluating in political turmoil: nursing challenges in prevention programs.

PubMed

Laperrière, Hélène

2007-03-01

The concrete insertion of nurses into the context of an inquiry contributes to empirical evaluation research of health promotion programs. As interveners and concrete actors in social movements, nurses are in a privileged position to give realism to a local understanding of the political and cultural context of evaluative research. Drawing on the practice of empirical evaluation research, this paper seeks to generate new methodological approaches in a way that broadens nursing inquiries in community health nursing. It explores new ways of thinking about epistemology and knowledge production in nursing practice. For 5 months an evaluative research project adopting a participatory-action research approach was conducted with local community actors in an AIDS prevention project in Amazonas (Brazil) in a prostitution setting. An auto-ethnographic journal was used as a reflective approach for the critical analysis of nursing research activities. This paper calls for a closer relationship between scientific research settings and the sociopolitical and the sociocultural aspects of nursing practice. It increases an incorporation of popular, social and professional experiential learning and skill acquisition in embedded knowledge production.

Evaluating an In-School Drug Prevention Program for At-Risk Youth.

ERIC Educational Resources Information Center

DeWit, David J.; Steep, Barbara; Silverman, Gloria; Stevens-Lavigne, Andrea; Ellis, Kathy; Smythe, Cindy; Rye, Barbara J.; Braun, Kathy; Wood, Eileen

2000-01-01

A drug prevention program involving 167 at-risk students in grades 8-10 at 9 Ontario schools resulted in reduced use of and less supportive attitudes toward alcohol, cannabis, tobacco, and tranquilizers. Program success is attributed to high attendance and retention, community health professionals' participation, comprehensive approach, strong…

Approaches for Evaluating the NRC Resident Research Associateship Program at NIST

ERIC Educational Resources Information Center

Sislin, John, Ed.

2007-01-01

The NRC Resident Research Associateship Program at NIST provides two-year temporary appointments for outstanding scientists and engineers. This book describes program applicants and awardees and offers suggestions for an in-depth assessment of career outcomes. Preliminary investigation indicates that outreach efforts produce more qualified…

Academic Day Camp.

ERIC Educational Resources Information Center

Akron Public Schools, OH.

This report of an academic day camp program for disadvantaged inner-city children includes a description of the program as well as an evaluation based on staff recommendations and standardized test scores. The program provides an all-day experience with an individualized approach to improvement in reading and mathematics skills; in the afternoon,…

Benchmarking. Issues in the Design and Implementation of a Benchmarking System for Employment and Training Programs for Young People.

ERIC Educational Resources Information Center

Coughlin, David C.; Bielen, Rhonda P.

This paper has been prepared to assist the United States Department of Labor to explore new approaches to evaluating and measuring the performance of employment and training activities for youth. As one of several tools for evaluating success of local youth training programs, "benchmarking" provides a system for measuring the development…

A Cost-Effectiveness Evaluation Approach to Improving Resource Allocations for School Systems. Administering for Change Program. A Professional Paper.

ERIC Educational Resources Information Center

Temkin, Sanford

This dissertation begins with a description of some methods employed in making public-sector resource-allocation decisions, with conclusions on the appropriateness of each method for evaluating the ongoing programs of a school system. The second section has been rewritten and published as "A Comprehensive Theory of Cost-Effectiveness" (EA 002…

Evaluation of an Alcohol Risk Reduction Program (PRIME for Life) in Young Swedish Military Conscripts

ERIC Educational Resources Information Center

Hallgren, Mats A.; Kallmen, Hakan; Leifman, Hakan; Sjolund, Torbjorn; Andreasson, Sven

2009-01-01

Purpose: The purpose of this paper is to evaluate the effectiveness of the PRIME for Life risk reduction program in reducing alcohol consumption and improving knowledge and attitudes towards alcohol use in male Swedish military conscripts, aged 18 to 22 years. Design/methodology/approach: A quasi-experimental design was used in which 1,371…

How Service Works. Summary of the W. K. Kellogg Foundation's "Service Works": A Retrospective Evaluation of Higher Education Service Programs.

ERIC Educational Resources Information Center

Kellogg Foundation, Battle Creek, MI.

Service learning is an undergraduate education strategy found across four program areas funded by the W. K. Kellogg Foundation (WKKF). This cross-foundation, retrospective evaluation examines 35 projects of $100,000 or more funded between 1985-95. Study information was based on qualitative data and approaches, with the majority of project…

Evaluation Methodologies: A Study to Evaluate the Functioning and Impact of the President's Council on Physical Fitness and Sports. Report No. 3.

ERIC Educational Resources Information Center

Bruce, Robert; And Others

This report presents an overview of research objectives, sampling approaches, data collection procedures, and instruments and plans for analysis used in assessing the impact of the President's Council on Physical Fitness and Sports on different types of fitness programs. Surveys were conducted of: (1) community fitness programs; (2) employee…

Process Evaluation of the Boost-A™ Transition Planning Program for Adolescents on the Autism Spectrum: A Strengths-Based Approach

ERIC Educational Resources Information Center

Hatfield, Megan; Falkmer, Marita; Falkmer, Torbjörn; Ciccarelli, Marina

2018-01-01

A process evaluation was conducted to determine the effectiveness, usability, and barriers and facilitators related to the Better OutcOmes & Successful Transitions for Autism (BOOST-A™), an online transition planning program. Adolescents on the autism spectrum (n = 33) and their parents (n = 39) provided feedback via an online questionnaire.…

Assessing Change in the Teaching Practice of Faculty in a Faculty Development Program for Primary Care Physicians: Toward a Mixed Method Evaluation Approach.

ERIC Educational Resources Information Center

Pinheiro, Sandro O.; Rohrer, Jonathan D.; Heimann, C. F. Larry

This paper describes a mixed method evaluation study that was developed to assess faculty teaching behavior change in a faculty development fellowship program for community-based hospital faculty. Principles of adult learning were taught to faculty participants over the fellowship period. These included instruction in teaching methods, group…

Substance Abuse, Relapse, and Treatment Program Evaluation in Malaysia: Perspective of Rehab Patients and Staff Using the Mixed Method Approach

PubMed Central

Chie, Qiu Ting; Tam, Cai Lian; Bonn, Gregory; Dang, Hoang Minh; Khairuddin, Rozainee

2016-01-01

This study examined reasons for substance abuse and evaluated the effectiveness of substance treatment programs in Malaysia through interviews with rehab patients and staff. Substance rehab patients (aged 18–69 years; n = 30) and staff (ages 30–72 years; n = 10) participated in semi-structured interviews covering a range of topics, including family and peer relationships, substance use and treatment history, factors for substance use and relapse, motivation for entering treatment, work experience, job satisfaction, treatment evaluation, and patient satisfaction. Most patients did not demonstrate the substance progression trend and had normal family relationships. Most patients reported having peers from normal family backgrounds as well. Various environmental and personal factors was cited as contributing to substance abuse and relapse. There was no significant difference between patient and staff program evaluation scores although the mean score for patients was lower. A holistic treatment approach with a combination of cognitive–behavioral, medical, social, and spiritual components was favored by patients. Suggestions for improving existing programs include better tailoring treatment to individual needs, and providing more post-treatment group support. PMID:27303313

NASA Collaborative Approach Mitigates Environmentally-Driven Obsolescence

NASA Technical Reports Server (NTRS)

Greene, Brian; Leeney, Bob; Richards, Joni

2016-01-01

National Aeronautics and Space Administration (NASA) missions, like Department of Defense (DoD) organizations, require the rigorous testing and qualification of critical materials. Obsolescence supply risks created by environmental requirements can affect the cost, schedule and performance of NASA missions and the resilience of critical infrastructure. The NASA Technology Evaluation for Environmental Risk Mitigation (TEERM) Principal Center helps to identify obsolescence supply risks driven by environmental requirements and works proactively with NASA Centers and Programs, the DoD, the European Space Agency (ESA) and other agencies and partners to identify and evaluate environmentally friendly alternatives. TEERM tracks environmental regulations, identifies the potential loss of material availability and works with NASA programs and Centers to evaluate potential impacts through a risk assessment approach. TEERM collaborative projects identify, demonstrate and evaluate commercially viable alternative technologies and materials. A major focus during the Space Shuttle Program was the need to replace ozone depleting substances that were used in spray foam and cleaning applications. The potential obsolescence of coatings containing hexavalent chromium and the risks associated with lead free solder were also of concern for the Space Shuttle and present ongoing risks to new programs such as the Space Launch System. One current project teams NASA and ESA in the evaluation and testing of individual coatings and coating systems as replacements for hexavalent chromium coatings in aerospace applications. The proactive, collaborative approach used by TEERM helps reduce the cost burden on any one team partner, reduces duplication of effort, and enhances the technical quality and overall applicability of the testing and analysis.

Organizational Wellness Program Implementation and Evaluation: A Holistic Approach to Improve the Wellbeing of Middle Managers.

PubMed

Medina, Maria Del Consuelo; Calderon, Angelica; Blunk, Dan I; Mills, Brandy W; Leiner, Marie

2018-06-01

: Employee wellness programs can provide benefits to institutions as well as employees and their families. Despite the attempts of some organizations to implement programs that take a holistic approach to improve physical, mental, and social wellness, the most common programs are exclusively comprised of physical and nutritional components. In this study, we implemented a wellness program intervention, including training using a holistic approach to improve the wellbeing of middle managers in several multinational organizations. We included control and experimental groups to measure wellness and teamwork with two repeated measures. Our results indicated that employees receiving the intervention had improved measures of wellness and teamwork. A positive relationship was found between wellness and teamwork in the experimental group when compared with the control group. Taken together, the data suggest that implementation of these programs would provide valuable outcomes for both employees and organizations.

Psychosocial stress and cardiovascular disease. Part 3: Clinical and policy implications of research on the transcendental meditation program.

PubMed

Walton, Kenneth G; Schneider, Robert H; Salerno, John W; Nidich, Sanford I

2005-01-01

Cardiovascular disease (CVD) remains the leading cause of death in the United States today and a major contributor to total health care costs. Psychosocial stress has been implicated in CVD, and psychosocial approaches to primary and secondary prevention are gaining research support. This third article in the series on psychosocial stress and CVD continues the evaluation of one such approach, the Maharishi Transcendental Meditation program, a psychophysiological approach from the Vedic tradition that is systematically taught by qualified teachers throughout the world. Evidence suggests not only that this program can provide benefits in prevention but also that it may reduce CVD-related and other health care expenses. On the basis of data from the studies available to date, the Transcendental Meditation program may be responsible for reductions of 80% or greater in medical insurance claims and payments to physicians. This article evaluates the implications of research on the Transcendental Meditation program for health care policy and for large-scale clinical implementation of the program. The Transcendental Meditation program can be used by individuals of any ethnic or cultural background, and compliance with the practice regimen is generally high. The main steps necessary for wider adoption appear to be: (1) educating health care providers and patients about the nature and expected benefits of the program, and (2) adjustments in public policies at the state and national levels to allow this program to be included in private and public health insurance plans.

Evaluation of a Digital Game-Based Learning Program for Enhancing Youth Mental Health: A Structural Equation Modeling of the Program Effectiveness.

PubMed

Huen, Jenny My; Lai, Eliza Sy; Shum, Angie Ky; So, Sam Wk; Chan, Melissa Ky; Wong, Paul Wc; Law, Y W; Yip, Paul Sf

2016-10-07

Digital game-based learning (DGBL) makes use of the entertaining power of digital games for educational purposes. Effectiveness assessment of DGBL programs has been underexplored and no attempt has been made to simultaneously model both important components of DGBL: learning attainment (ie, educational purposes of DGBL) and engagement of users (ie, entertaining power of DGBL) in evaluating program effectiveness. This study aimed to describe and evaluate an Internet-based DGBL program, Professor Gooley and the Flame of Mind, which promotes mental health to adolescents in a positive youth development approach. In particular, we investigated whether user engagement in the DGBL program could enhance their attainment on each of the learning constructs per DGBL module and subsequently enhance their mental health as measured by psychological well-being. Users were assessed on their attainment on each learning construct, psychological well-being, and engagement in each of the modules. One structural equation model was constructed for each DGBL module to model the effect of users' engagement and attainment on the learning construct on their psychological well-being. Of the 498 secondary school students that registered and participated from the first module of the DGBL program, 192 completed all 8 modules of the program. Results from structural equation modeling suggested that a higher extent of engagement in the program activities facilitated users' attainment on the learning constructs on most of the modules and in turn enhanced their psychological well-being after controlling for users' initial psychological well-being and initial attainment on the constructs. This study provided evidence that Internet intervention for mental health, implemented with the technologies and digital innovations of DGBL, could enhance youth mental health. Structural equation modeling is a promising approach in evaluating the effectiveness of DGBL programs.

Individualised Qualitative Evaluation.

ERIC Educational Resources Information Center

O'Sullivan, Denis

1987-01-01

The author discusses student evaluation in relation to adult and continuing education programs offered by the Department of Adult Education, University College, Cork. He highlights the need for a more individualized and interactive approach to evaluation, allowing the student to benefit from qualitative feedback in the process of being evaluated.…

School-based programs to reduce sexual risk-taking behaviors.

PubMed

Kirby, D

1992-09-01

This article reviews the major approaches implemented during the last two decades to reduce sexual risk-taking behaviors, examines their evidence for success, and provides several recommendations for effective programs and program evaluations. This article does not discuss more broad-based sexuality education programs which address sexuality in a broader context. Instead, this article focuses primarily on programs that educators believed would reduce unprotected sexual intercourse.

Experiences with Recruitment of Marginalized Groups in a Danish Health Promotion Program: A Document Evaluation Study.

PubMed

Rasmussen, Marianne; Poulsen, Eva Kanstrup; Rytter, Anne Stoffersen; Kristiansen, Tine Mechlenborg; Bak, Carsten Kronborg

2016-01-01

Studies have found that marginalized groups living in deprived neighborhoods are less likely to participate in health programs compared to the majority of society. This study evaluates recruitment approaches conducted during a national government-funded project in 12 deprived neighborhoods across Denmark between 2010 and 2014. The aim of this study was to understand how recruitment approaches could promote participation in health programs within deprived neighborhoods to reach marginalized groups. Documents from all 12 of the included municipalities were collected to conduct a document evaluation. The collected documents consisted of 1,500 pages of written material with 12 project descriptions, three midterm and 10 final evaluations. The collected data were analyzed through a qualitative content analysis. The results are based on the fact that only 10 municipalities have developed evaluations related to recruitment, and only three evaluations provided a description of which marginalized groups were recruited. Challenges related to recruitment consist of difficulties involving the target group, including general distrust, language barriers and a lack of ability to cope with new situations and strangers. Additional geographical challenges emerged, especially in rural areas. Positive experiences with recruitment approaches were mainly related to relationship building and trust building, especially through face-to-face contact and the project employees' presence in the neighborhood. Additionally, adjusting some of the interventions and the recruitment strategy increased participation. This study found that relation and trust between the residents and the project employees is an important factor in the recruitment of marginalized groups in deprived neighborhoods as well as adjusting the health interventions or recruitment strategy to the target groups. In future research, it is necessary to examine which recruitment approaches are effective under which circumstances to increase participation among marginalized groups.

Building research and evaluation capacity in population health: the NSW Health approach.

PubMed

Edwards, Barry; Stickney, Beth; Milat, Andrew; Campbell, Danielle; Thackway, Sarah

2016-02-01

Issue addressed An organisational culture that values and uses research and evaluation (R&E) evidence to inform policy and practice is fundamental to improving health outcomes. The 2016 NSW Government Program Evaluation Guidelines recommend investment in training and development to improve evaluation capacity. The purpose of this paper is to outline the approaches taken by the NSW Ministry of Health to develop R&E capacity and assess these against existing models of practice. Method The Ministry of Health's Centre for Epidemiology and Evidence (CEE) takes an evidence-based approach to building R&E capacity in population health. Strategies are informed by: the NSW Population Health Research Strategy, R&E communities of practice across the Ministry and health Pillar agencies and a review of the published evidence on evaluation capacity building (ECB). An internal survey is conducted biennially to monitor research activity within the Ministry's Population and Public Health Division. One representative from each of the six centres that make up the Division coordinates completion of the survey by relevant staff members for their centre. Results The review identified several ECB success factors including: implementing a tailored multifaceted approach; an organisational commitment to R&E; and offering experiential training and ongoing technical support to the workforce. The survey of research activity found that the Division funded a mix of research assets, research funding schemes, research centres and commissioned R&E projects. CEE provides technical advice and support services for staff involved in R&E and in 2015, 22 program evaluations were supported. R&E capacity building also includes a series of guides to assist policy makers, practitioners and researchers to commission, undertake and use policy-relevant R&E. Staff training includes workshops on critical appraisal, program logic and evaluation methods. From January 2013 to June 2014 divisional staff published 84 peer-reviewed papers and one book chapter. Conclusion A strategic approach to R&E capacity building compares favourably with organisational dimensions of ECB and has facilitated the generation of high quality population health R&E in NSW. So what? An evidence-based multistrategy approach to population health R&E can result in substantial contributions to the population-health evidence base.

Assessment of the Sustainability Capacity of a Coordinated Approach to Chronic Disease Prevention.

PubMed

Moreland-Russell, Sarah; Combs, Todd; Polk, LaShaun; Dexter, Sarah

This article outlines some factors that influenced the sustainability capacity of a coordinated approach to chronic disease prevention in state and territory health departments. This study involved a cross-sectional design and mixed-methods approach. Quantitative data were collected using the Program Sustainability Assessment Tool (PSAT), a 40-item multiple-choice instrument that assesses 8 domains of sustainability capacity (environmental support, funding stability, partnerships, organizational capacity, program evaluation, program adaptation, communications, and strategic planning). Qualitative data were collected via phone interviews. The PSAT was administered to staff and stakeholders from public health departments in 50 US states, District of Columbia, and Puerto Rico, who were involved in the implementation of coordinated chronic disease programs. Phone interviews were conducted with program coordinators in each state. Sustainability score patterns and state-level categorical results, as well as strengths and opportunities for improvement across the 8 program sustainability domains, were explored. On average, programs reported the strongest sustainability capacity in the domains of program adaptation, environmental support, and organizational capacity, while funding stability, strategic planning, and communications yielded lowest scores, indicating weakest capacity. Scores varied the most by state in environmental support and strategic planning. The PSAT results highlight the process through which states approached the sustainability of coordinated chronic disease initiatives. This process included an initial focus on program evaluation and partnerships with transfer of priority to long-term strategic planning, communications, and funding stability to further establish coordinated chronic disease efforts. Qualitative interviews provided further context to PSAT results, indicating that leadership, communications, partnerships, funding stability, and policy change were perceived as keys to success of the transition. Integrating these findings into future efforts may help those in transition establish greater sustainability capacity. The PSAT results and interviews provide insight into the capacity for sustainability for programs transitioning from traditional siloed programs to coordinated chronic disease programs.

"New Choices" for women with addictions: perceptions of program participants

PubMed Central

Sword, Wendy; Niccols, Alison; Fan, Aimei

2004-01-01

Background Substance use in pregnancy is a major public health problem. It can have profound effects on pregnancy outcomes, and childhood health and development. Additionally, women who use substances have their own health-related issues. Although intervention is important, these women often have difficulty using traditional systems of care. The New Choices program is a centralized, multi-sector approach to service delivery that has attempted to overcome barriers to care by offering one-stop shopping in a supportive environment. As part of an evaluation of this program designed for women who are pregnant and/or parenting young children, interviews were conducted with participants to gain insight into their experiences in New Choices and perceptions of any changes attributed to program involvement. Methods A qualitative, exploratory design was used to guide data collection and analysis. Four women participated in a focus group interview and seven women agreed to individual interviews over the course of the program evaluation (N = 11). A semi-structured interview guide was used to explore women's experiences in New Choices and their perceptions of the program and its impact. The interview data were analyzed using NVivo software and an inductive approach to data analysis. Results The emergent themes captured women's motivations for attending New Choices, benefits of participation, and overall quality of the program. Children were the primary motivating factor for program enrollment. Perceived benefits included decreased substance use, improved maternal health, enhanced opportunity for employment, increased access to other resources, enhanced parenting skills, and improved child behaviour and development. Women highly valued the comprehensive and centralized approach to service delivery that provided a range of informal and formal supports. Conclusions Interview findings endorse the appropriateness and potential efficacy of a collaborative, centralized approach to service provision for women with substance use issues. Although the findings provide insight into an alternative model of service delivery for women with addictions, future research is needed to evaluate the effectiveness of the intervention. Research also is needed to determine which program components or constellation of components contribute to desired outcomes, and to learn more about processes that underlie changes in behaviour. PMID:15086957

Commitment to Practice Change: An Evaluator's Perspective

PubMed Central

Shershneva, Marianna B.; Wang, Min-fen; Lindeman, Gary C.; Savoy, Julia N.; Olson, Curtis A.

2010-01-01

A commitment to practice change (CTC) approach may be used in educational program evaluation to document practice changes, examine the educational impact relative to the instructional focus, and improve understanding of the learning-to-change continuum. The authors reviewed various components and procedures of this approach and discussed some practical aspects of its application using as an example of a study evaluating a presentation on menopausal care for primary care physicians. The CTC approach is a valuable evaluation tool, but it requires supplementation with other data to have a complete picture of the impact of education on practice. From the evaluation perspective, the self-reported nature of the CTC data is a major limitation of this method. PMID:20457715

Engaging pregnant and parenting teens: early challenges and lessons learned from the Evaluation of Adolescent Pregnancy Prevention Approaches.

PubMed

Asheer, Subuhi; Berger, Amanda; Meckstroth, Alicia; Kisker, Ellen; Keating, Betsy

2014-03-01

This article draws on data from the ongoing federal Evaluation of Adolescent Pregnancy Prevention Approaches to discuss the early implementation experiences of two new and innovative programs intended to delay rapid repeat pregnancy among teen mothers: (1) AIM 4 Teen Moms, in Los Angeles County, California; and (2) Teen Options to Prevent Pregnancy (T.O.P.P.), in Columbus, Ohio. Program staff report common challenges in working with teen mothers, particularly concerning recruitment and retention, staff capacity and training, barriers to participation, and participants' overarching service needs. Lessons learned in addressing these challenges provide useful guidance to program developers, providers, policy makers, and stakeholders working with similar populations. Copyright © 2014 Society for Adolescent Health and Medicine. All rights reserved.

Formative process evaluation for implementing a social marketing intervention to increase walking among African Americans in the Positive Action for Today's Health trial.

PubMed

Coulon, Sandra M; Wilson, Dawn K; Griffin, Sarah; St George, Sara M; Alia, Kassandra A; Trumpeter, Nevelyn N; Wandersman, Abraham K; Forthofer, Melinda; Robinson, Shamika; Gadson, Barney

2012-12-01

Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today's Health trial. We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities.

Evaluation of Natural Language Processors.

DTIC Science & Technology

1980-11-01

techniques described. Common practice in describing natural language processors is to describe the programs, then give about 20 examples of correctly...make a decision based on performance as to which approaches are most promising for further research and development. The lack of evaluation leaves...successively more difficult problems. This approach might be compared to children taking achievement tests in school. A 90% score on problems involving

High Tech Educators Network Evaluation.

ERIC Educational Resources Information Center

O'Shea, Dan

A process evaluation was conducted to assess the High Tech Educators Network's (HTEN's) activities. Four basic components to the evaluation approach were documentation review, program logic model, written survey, and participant interviews. The model mapped the basic goals and objectives, assumptions, activities, outcome expectations, and…

Two autowire versions for CDC-3200 and IBM-360

NASA Technical Reports Server (NTRS)

Billingsley, J. B.

1972-01-01

Microelectronics program was initiated to evaluate circuitry, packaging methods, and fabrication approaches necessary to produce completely procured logic system. Two autowire programs were developed for CDC-3200 and IBM-360 computers for use in designing logic systems.

Community Based Competitive Employment Preparation of Developmentally Disabled Persons: A Program Description and Evaluation.

ERIC Educational Resources Information Center

Stodden, Robert A.; Browder, Phyllis Meighen

1986-01-01

Success in competitive employment placement of a demonstration project serving 53 persons with developmental disabilities is associated with three factors: (1) training approach and methodology; (2) program management; and (3) trainee characteristics. (CL)

Evaluation of a community-based sex offender treatment program using a good lives model approach.

PubMed

Harkins, Leigh; Flak, Vanja E; Beech, Anthony R; Woodhams, Jessica

2012-12-01

In this study the authors assessed a Good Lives model (GLM) approach to sex offender treatment and compare it to a standard Relapse Prevention program. The comparisons examined (a) attrition rates, (b) treatment change in areas targeted in treatment and achievement of a posttreatment treated profile, and (c) views of offenders and facilitators. There were no differences in the attrition rates or the rates of treatment change between the two programs, indicating that they were equally effective at retaining participants and achieving change on areas targeted within treatment. Both facilitators and program participants reported the Good Lives approach module's impact in a positive, future-focused manner. In contrast, those who attended the Relapse Prevention module did not report their perceptions and motivations in a manner that was focused on the positives in their future as frequently as those who attended the module with the Good Lives model approach.

From Implementation to Outcomes to Impacts: Designing a Comprehensive Program Evaluation

NASA Astrophysics Data System (ADS)

Shebby, S.

2015-12-01

Funders are often interested in learning about the impact of program activities, yet before the impacts are determined, educational evaluations should first examine program implementation and outcomes. Implementation evaluation examines how and the extent to which program activities are delivered as intended, including the extent to which activities reached the targeted participants. Outcome evaluation is comprised of a systematic examination of the effects that a program has on program participants, such as changes in knowledge, attitudes, beliefs, values, and behaviors. In this presentation, presenters will share insights on evaluating the implementation, outcomes, and impacts associated with an online science curriculum for K-2 students. The science curriculum was designed to provide students with access to science concepts and skills in an interactive and innovative environment, and teachers with embedded, aligned, and on-demand professional development. One of the most important—and challenging—steps in this evaluation was to select outcomes that were well-defined, measurable, and aligned to program activities, as well as relevant to program stakeholders. An additional challenge was to measure implementation given limited access to the classroom environment. This presentation will include a discussion of the process evaluators used to select appropriate implementation indicators and outcomes (teacher and student), design an evaluation approach, and craft data collection instruments. Although examples provided are specific to the K-2 science intervention, the best practices discussed are pertinent to all program and event evaluations. Impact evaluation goes beyond implementation and outcome evaluation to inform whether a program is working or not. It requires a comparison group to inform what outcomes would have been in the absence of the intervention. As such, this presentation will also include a discussion of impacts, including how impacts are defined and measured, and some common challenges in evaluating program impact.

Evaluating the impact of implementation factors on family-based prevention programming: methods for strengthening causal inference.

PubMed

Crowley, D Max; Coffman, Donna L; Feinberg, Mark E; Greenberg, Mark T; Spoth, Richard L

2014-04-01

Despite growing recognition of the important role implementation plays in successful prevention efforts, relatively little work has sought to demonstrate a causal relationship between implementation factors and participant outcomes. In turn, failure to explore the implementation-to-outcome link limits our understanding of the mechanisms essential to successful programming. This gap is partially due to the inability of current methodological procedures within prevention science to account for the multitude of confounders responsible for variation in implementation factors (i.e., selection bias). The current paper illustrates how propensity and marginal structural models can be used to improve causal inferences involving implementation factors not easily randomized (e.g., participant attendance). We first present analytic steps for simultaneously evaluating the impact of multiple implementation factors on prevention program outcome. Then, we demonstrate this approach for evaluating the impact of enrollment and attendance in a family program, over and above the impact of a school-based program, within PROSPER, a large-scale real-world prevention trial. Findings illustrate the capacity of this approach to successfully account for confounders that influence enrollment and attendance, thereby more accurately representing true causal relations. For instance, after accounting for selection bias, we observed a 5% reduction in the prevalence of 11th grade underage drinking for those who chose to receive a family program and school program compared to those who received only the school program. Further, we detected a 7% reduction in underage drinking for those with high attendance in the family program.

An Analytical Approach to Salary Evaluation for Educational Personnel

ERIC Educational Resources Information Center

Bruno, James Edward

1969-01-01

"In this study a linear programming model for determining an 'optimal' salary schedule was derived then applied to an educational salary structure. The validity of the model and the effectiveness of the approach were established. (Author)

Crossing boundaries: the design of an interdisciplinary training program to improve care for the frail elderly.

PubMed

Kolomitro, Klodiana; Stockley, Denise; Egan, Rylan; MacDonald, Michelle L

2015-01-01

The Technology Evaluation in the Elderly Network (TVN) was funded in July 2012 under the Canadian Networks of Centres of Excellence program. This article highlights the development and preliminary evaluation of the TVN Interdisciplinary Training Program. This program is based on an experiential learning approach that crosses a multitude of disciplines including health sciences, law, social sciences, and ethical aspects of working with the frail elderly. Opportunities within the program include mentorship, interdisciplinary online collaborative projects, external placements, academic products, pre-grant submission, trainee-driven requirements, Network meetings, online modules/webinars, and most importantly active involvement with patients, families, and their support systems. The authors have 120 trainees from approximately 23 different disciplines including law, ethics, public policy, social work, and engineering engaged in the program. Based on our evaluation this program has been perceived as highly valuable by the participants and the community.

Evaluation of the Comprehensive School Reform Program Implementation and Outcomes. Third-Year Report

ERIC Educational Resources Information Center

Orland, Martin; Connolly, Brooke; Fong, Tony; Sosenko, Lauren Davis; Tushnet, Naida C.; Yin, Robert K.; Burt, Janeula M.; Warner, Emily

2008-01-01

The Comprehensive School Reform (CSR) Program was established as a demonstration program in 1998 and authorized as a full program in 2002 as part of the No Child Left Behind Act (NCLB). It is one approach to help low-performing K-12 public schools meet state performance standards. CSR emphasizes two major concepts: mandating that school reform…

Design and Implementation of a Pilot Obesity Prevention Program in a Low-Resource School: Lessons Learned and Research Recommendations

ERIC Educational Resources Information Center

Baskin, Monica L.; Zunker, Christie; Worley, Courtney B.; Dial, Brenda; Kimbrough, Linda

2009-01-01

Purpose: This paper seeks to describe the design, implementation, and lessons learned from an obesity prevention pilot program delivered in a low resource school in the USA. Design/methodology/approach: A planned program evaluation was conducted to: document explicitly the process of designing and implementing the program; and assess the…

A combined disease management and process modeling approach for assessing and improving care processes: a fall management case-study.

PubMed

Askari, Marjan; Westerhof, Richard; Eslami, Saied; Medlock, Stephanie; de Rooij, Sophia E; Abu-Hanna, Ameen

2013-10-01

To propose a combined disease management and process modeling approach for evaluating and improving care processes, and demonstrate its usability and usefulness in a real-world fall management case study. We identified essential disease management related concepts and mapped them into explicit questions meant to expose areas for improvement in the respective care processes. We applied the disease management oriented questions to a process model of a comprehensive real world fall prevention and treatment program covering primary and secondary care. We relied on interviews and observations to complete the process models, which were captured in UML activity diagrams. A preliminary evaluation of the usability of our approach by gauging the experience of the modeler and an external validator was conducted, and the usefulness of the method was evaluated by gathering feedback from stakeholders at an invitational conference of 75 attendees. The process model of the fall management program was organized around the clinical tasks of case finding, risk profiling, decision making, coordination and interventions. Applying the disease management questions to the process models exposed weaknesses in the process including: absence of program ownership, under-detection of falls in primary care, and lack of efficient communication among stakeholders due to missing awareness about other stakeholders' workflow. The modelers experienced the approach as usable and the attendees of the invitational conference found the analysis results to be valid. The proposed disease management view of process modeling was usable and useful for systematically identifying areas of improvement in a fall management program. Although specifically applied to fall management, we believe our case study is characteristic of various disease management settings, suggesting the wider applicability of the approach. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

The Politics of Speaking: An Approach to Evaluating Bilingual-Bicultural Schools. Bilingual Education Paper Series, Vol. 1, No. 6.

ERIC Educational Resources Information Center

Erickson, Frederick

A method of evaluating bilingual-bicultural education programs that has a sociolinguistic basis uses samples of the language spoken by a number of bilingual program students as they go through their school day. A random sample of the child's speech would be continuously recorded for an hour, with a bilingual observer taking running notes on where…

Understanding Acid-Base Concepts: Evaluating the Efficacy of a Senior High School Student-Centred Instructional Program in Indonesia

ERIC Educational Resources Information Center

Rahayu, Sri; Chandrasegaran, A. L.; Treagust, David F.; Kita, Masakazu; Ibnu, Suhadi

2011-01-01

This study was a mixed quantitative-qualitative research to evaluate the efficacy of a designed student-centred instructional (DSCI) program for teaching about acids and bases. The teaching innovation was designed based on constructivist, hands-on inquiry and context-based approaches and implemented in seven 45-min lessons with a class of 36 grade…

Workplace Literacy Programs: Variations of Approach and Limits of Impact.

ERIC Educational Resources Information Center

Mikulecky, Larry

Six workplace literacy programs were evaluated for impact upon learners, learners' families, and learners' productivity. Site 1 was an automotive plant where learners were involved in technical preparation, the General Educational Development program, and English as a Second Language (ESL). Site 2 was a wood-processing plant with a communication…

A Digital Program Informs Low-Income Caregivers of Preschool-Age Children about Family Meals

ERIC Educational Resources Information Center

Lohse, Barbara; Rifkin, Robin; Arnold, Kristen; Least, Christine

2012-01-01

Objective: To evaluate the digital program, "Mealtime is Family Time", as a means of educating caregivers of preschoolers on the importance of family meals within the division of feeding responsibility framework. Methods: Descriptive design using 2 approaches: focus group program review and discussion or self-report survey after independent…

Santa Fe School Precision Teaching Program, Evaluation Report 1974-75.

ERIC Educational Resources Information Center

Spencer, Mary L.; Henderson, Joan C.

The Santa Fe Precision Teaching for Effective Learning, (PTEL) an ESEA Title III program, was selected as a remedial instructional approach to the performance and motivational problems of Santa Fe students. It proposed the following six major program objectives: (1) planning and implementation of start-up activities; (2) staff training in the…

Evaluating Industry-Based Doctoral Research Programs: Perspectives and Outcomes of Australian Cooperative Research Centre Graduates

ERIC Educational Resources Information Center

Manathunga, Catherine; Pitt, Rachael; Cox, Laura; Boreham, Paul; Mellick, George; Lant, Paul

2012-01-01

Researchers of the future will need to be able to work across the increasingly porous boundaries between university, industry, government and community sectors. Concerns have been raised internationally for several decades about the content and approaches adopted in doctoral programs. Innovative doctoral programs that facilitate students'…

Career Exploration 10-12. Career Exploration Guidance Program.

ERIC Educational Resources Information Center

Young, John; And Others

Designed as a group guidance approach which can be used with study hall groups and subject matter area classes, this career guidance program provides experiences for high school students in the area of employability and work adjustment. Intended to span eighteen days, the program includes such topics as evaluation of self and skills, values…

Psychiatric Input as Part of a Comprehensive Evaluation Program for Socially and Emotionally Disturbed Children.

ERIC Educational Resources Information Center

Mendelsohn, Sylvia R.; And Others

1985-01-01

The paper describes a multidisciplinary team approach that enhances communication between the school and the team regarding services for socially and emotionaly disturbed children. A program integrating assessment and consultation services is provided. The program includes structured observations of the child at school. (Author/CL)

Putting Empirical Knowledge to Work: Linking Research and Programming on Marital Quality

ERIC Educational Resources Information Center

Adler-Baeder, Francesca; Higginbotham, Brian; Lamke, Leanne

2004-01-01

When selecting a marriage education curriculum, educators can turn to programs that have been evaluated for effectiveness; however, few curricula have undergone such study. An alternative approach, consistent with best practices, is to ensure a research base for program content. A translation process model is offered as an initial attempt to…

Implementing a Project-Based Learning Model in a Pre-Service Leadership Program

ERIC Educational Resources Information Center

Albritton, Shelly; Stacks, Jamie

2016-01-01

This paper describes two instructors' efforts to more authentically engage students in a preservice leadership program's course called Program Planning and Evaluation by using a project-based learning approach. Markham, Larmer, and Ravitz (2003) describe project-based learning (PjBL) as "a systematic teaching method that engages students in…

Empowering Organizations: Approaches to Tobacco Control through Youth Empowerment Programs

ERIC Educational Resources Information Center

LeRoy, Lisa; Benet, Dana Jones; Mason, Theresa; Austin, W. David; Mills, Sherry

2004-01-01

Whereas most evaluations of youth empowerment focus on individual outcomes (i.e., were individual youths empowered?), this article focuses on the program as the unit of analysis and seeks to explain how organizational structures, program design features, and processes lead to organizational empowerment (OE). OE is defined as organizational efforts…

Applying Coaching Strategies to Support Youth- and Family-Focused Extension Programming

ERIC Educational Resources Information Center

Olson, Jonathan R.; Hawkey, Kyle R.; Smith, Burgess; Perkins, Daniel F.; Borden, Lynne M.

2016-01-01

In this article, we describe how a peer-coaching model has been applied to support community-based Extension programming through the Children, Youth, and Families at Risk (CYFAR) initiative. We describe the general approaches to coaching that have been used to help with CYFAR program implementation, evaluation, and sustainability efforts; we…

Evaluation of the implementation of an integrated primary care network for prevention and management of cardiometabolic risk in Montréal.

PubMed

Provost, Sylvie; Pineault, Raynald; Tousignant, Pierre; Hamel, Marjolaine; Da Silva, Roxane Borgès

2011-11-10

The goal of this project is to evaluate the implementation of an integrated and interdisciplinary program for prevention and management of cardiometabolic risk (PCMR). The intervention is based on the Chronic Care Model. The study will evaluate the implementation of the PCMR in 6 of the 12 health and social services centres (CSSS) in Montréal, and the effects of the PCMR on patients and the practice of their primary care physicians up to 40 months following implementation, as well as the sustainability of the program. Objectives are: 1-to evaluate the effects of the PCMR and their persistence on patients registered in the program and the practice of their primary care physicians, by implementation site and degree of exposure to the program; 2-to assess the degree of implementation of PCMR in each CSSS territory and identify related contextual factors; 3-to establish the relationships between the effects observed, the degree of PCMR implementation and the related contextual factors; 4-to assess the impact of the PCMR on strengthening local services networks. The evaluation will use a mixed design that includes two complementary research strategies. The first strategy is similar to a quasi-experimental "before-after" design, based on a quantitative approach; it will look at the program's effects and their variations among the six territories. The effects analysis will use data from a clinical database and from questionnaires completed by participating patients and physicians. Over 3000 patients will be recruited. The second strategy corresponds to a multiple case study approach, where each of the six CSSS constitutes a case. With this strategy, qualitative methods will set out the context of implementation using data from semi-structured interviews with program managers. The quantitative data will be analyzed using linear or multilevel models complemented with an interpretive approach to qualitative data analysis. Our study will identify contextual factors associated with the effectiveness, successful implementation and sustainability of such a program. The contextual information will enable us to extrapolate our results to other contexts with similar conditions. ClinicalTrials.gov: NCT01326130.

Bleeding disorder education in obstetrics and gynecology residency training: a national survey.

PubMed

Dietrich, Jennifer E; Tran, Xuan G; Giardino, Angelo P

2011-04-01

The purpose of this study was to assess the educational approach to the bleeding disorder evaluation in Obstetrics and Gynecology residency training programs in the continental United States. Information was sought from chief residents regarding training experiences and fund of knowledge regarding the evaluation of menorrhagia and diagnosis of bleeding disorders during their residency. A 24-item questionnaire was sent to the chief residents at 241 non-military Obstetrics and Gynecology residency programs. The study was conducted at Texas Children's Health Plan in Houston, Texas. Chief residents at 241 non-military Obstetrics and Gynecology residency programs. Responses to questionnaires. The overall response rate was 30%. Residents reported training in the medical evaluation of menorrhagia during residency with a mean of 9.1 hours per year in the first year of residency and 11.1 hours/year in the 2(nd), 3(rd) and 4(th) years; 67.7% reported they viewed their training in the medical evaluation of menorrhagia and bleeding disorders as sufficient preparation for clinical practice; and over two thirds reported specific training in common bleeding disorders, such as von Willebrand disease. The current state of training in the evaluation of menorrhagia and bleeding disorders appeared to be mixed regarding the evaluation of dysfunctional uterine bleeding. An area for improvement was identified to better approach best clinical practice in the evaluation of women with menorrhagia and underlying bleeding disorders, which can be guided by the thoughtful approach taken in the recent NHLBI von Willebrand disease guidelines. Copyright © 2011 North American Society for Pediatric and Adolescent Gynecology. Published by Elsevier Inc. All rights reserved.

Space station common module power system network topology and hardware development

NASA Technical Reports Server (NTRS)

Landis, D. M.

1985-01-01

Candidate power system newtork topologies for the space station common module are defined and developed and the necessary hardware for test and evaluation is provided. Martin Marietta's approach to performing the proposed program is presented. Performance of the tasks described will assure systematic development and evaluation of program results, and will provide the necessary management tools, visibility, and control techniques for performance assessment. The plan is submitted in accordance with the data requirements given and includes a comprehensive task logic flow diagram, time phased manpower requirements, a program milestone schedule, and detailed descriptions of each program task.

Federal Highway Administration research and technology evaluation final report : Eco-Logical

DOT National Transportation Integrated Search

2018-03-01

This report documents an evaluation of Federal Highway Administrations (FHWA) Research and Technology Programs activities on the implementation of the Eco-Logical approach by State transportation departments and metropolitan planning organizati...

Analyzing Array Manipulating Programs by Program Transformation

NASA Technical Reports Server (NTRS)

Cornish, J. Robert M.; Gange, Graeme; Navas, Jorge A.; Schachte, Peter; Sondergaard, Harald; Stuckey, Peter J.

2014-01-01

We explore a transformational approach to the problem of verifying simple array-manipulating programs. Traditionally, verification of such programs requires intricate analysis machinery to reason with universally quantified statements about symbolic array segments, such as "every data item stored in the segment A[i] to A[j] is equal to the corresponding item stored in the segment B[i] to B[j]." We define a simple abstract machine which allows for set-valued variables and we show how to translate programs with array operations to array-free code for this machine. For the purpose of program analysis, the translated program remains faithful to the semantics of array manipulation. Based on our implementation in LLVM, we evaluate the approach with respect to its ability to extract useful invariants and the cost in terms of code size.

Cultural Competence Approaches to Evaluation in Tribal Communities.

ERIC Educational Resources Information Center

Running Wolf, Paulette; Soler, Robin; Manteuffel, Brigitte; Sondheimer, Diane; Santiago, Rolando L.; Erickson, Jill Shephard

Disability research and program evaluation have generally been viewed with suspicion in Indian country because research designs, procedures, instruments, and interpretation and dissemination of outcomes have often ignored potential cultural conflicts. This paper explores a national evaluation in eight tribal communities that established systems of…

Evaluating and giving feedback to mentors: new evidence-based approaches.

PubMed

Anderson, Lauren; Silet, Karin; Fleming, Michael

2012-02-01

A comprehensive mentoring program includes a variety of components. One of the most important is the ongoing assessment of and feedback to mentors. Scholars need strong active mentors who have the expertise, disposition, motivation, skills, and the ability to accept feedback and to adjust their mentoring style. Assessing the effectiveness of a given mentor is no easy task. Variability in learning needs and academic goals among scholars makes it difficult to develop a single evaluation instrument or a standardized procedure for evaluating mentors. Scholars, mentors, and program leaders are often reluctant to conduct formal evaluations, as there are no commonly accepted measures. The process of giving feedback is often difficult and there is limited empirical data on efficacy. This article presents a new and innovative six-component approach to mentor evaluation that includes the assessment of mentee training and empowerment, peer learning and mentor training, scholar advocacy, mentee-mentor expectations, mentor self-reflection, and mentee evaluation of their mentor. © 2012 Wiley Periodicals, Inc.

Behavior-based safety at Amtrak-Chicago associated with reduced injuries and costs.

DOT National Transportation Integrated Search

2007-02-01

The Federal Railroad Administration (FRA) Human Factors Research and Development (R&D) Program is sponsoring the Clear Signal for Action Program (CSA) to evaluate whether an approach that combines behavior-based safety (BBS) and continuous improvemen...

Development and Implementation of a Mental Health Work Rehabilitation Program: Results of a Developmental Evaluation.

PubMed

Sylvain, Chantal; Durand, Marie-José; Velasquez Sanchez, Astrid; Lessard, Nathalie; Maillette, Pascale

2018-05-23

Purpose Long-term work disability due to common mental disorders (CMDs) is a growing problem. Yet optimal interventions remain unclear and little is known about implementation challenges in everyday practice. This study aimed to support and evaluate, in real time, the development and implementation of a work rehabilitation program (WRP) designed to promote post-CMD return-to-work (RTW). Methods A 2-year developmental evaluation was performed using a participatory approach. At program outset, the researchers held five work meetings to revise the program's logic model and discuss its underlying change theory with clinicians. Data collection tools used throughout the study period were structured charts of activities conducted with workers (n = 41); in-depth interviews with program clinicians and managers (n = 9); and participant observation during work meetings. Quantitative data were analyzed using descriptive statistics. Qualitative data underwent thematic analysis using a processual approach. Results Three types of activity were developed and implemented: individual and group interventions targeting workers, and joint activities targeting partners (physicians, employers, others). While worker-targeted activities were generally implemented as planned, joint activities were sporadic. Analysis of the implementation process revealed five challenges faced by clinicians. Determinants included clinicians, host organization, sociopolitical context and resources provided by the evaluation. Conclusion The program studied is original in that it is based on the best available scientific knowledge, yet adapted to contextual particularities. The identified implementation challenges highlight the need for greater importance to be placed on the external, non-program context to ensure sustainable implementation in everyday practice.

Evaluation of a community-based, family focused healthy weights initiative using the RE-AIM framework.

PubMed

Jung, Mary E; Bourne, Jessica E; Gainforth, Heather L

2018-01-26

Childhood overweight and obesity is a major public health concern. Community-based interventions have the potential to reach caregivers and children. However, the overall health impact of these programs is rarely comprehensively assessed. This study evaluated a physical activity and healthy eating family program (Healthy Together; HT) using the RE-AIM framework. Ten sites implemented the 5-week program. Thirty-nine staff members and 277 program participants (126 caregivers [M age  = 35.6] and 151 children [M age  = 13]) participated in the evaluation. Each RE-AIM dimension was assessed independently using a mixed-methods approach. Sources of data included archival records, interviews and surveys. Effectiveness outcome variables were assessed at pre- and post-intervention and 6-month follow-up. Reach: HT participants were almost entirely recruited from existing programs within sites. Effectiveness: Caregivers' nutrition related efficacy beliefs increased following HT (ps < .03). Participation in HT was not associated with significant changes in physical activity or nutrition behaviour or perceived social support (ps > .05). Knowledge surrounding healthy diets and physical activity increased in children and caregivers (ps < .05). Adoption: Thirty-five percent of sites approached to implement HT expressed interest. The 10 sites selected recruited existing staff members to implement HT. Program objectives were met 72.8% of the time and 71 adaptations were made. HT was finance- and time-dependent. Maintenance: Two sites fully implemented HT in the follow-up year and 5 sites incorporated aspects of HT into other programs. Working alongside organizations that develop community programs to conduct comprehensive, arms-length evaluations can systematically highlight areas of success and challenges. Overall HT represents a feasible community-based intervention; however further support is required in order to ensure the program is effective at positively targeting the desired outcomes. As a result of this evaluation, modifications are currently being implemented to HT.

Monitoring and evaluating transition and sustainability of donor-funded programs: Reflections on the Avahan experience.

PubMed

Bennett, Sara; Ozawa, Sachiko; Rodriguez, Daniela; Paul, Amy; Singh, Kriti; Singh, Suneeta

2015-10-01

In low and middle-income countries, programs funded and implemented by international donors frequently transition to local funding and management, yet such processes are rarely evaluated. We reflect upon experience evaluating the transition of a large scale HIV/AIDS prevention program in India, known as Avahan, in order to draw lessons about transition evaluation approaches and implementation challenges. In terms of conceptualizing the transition theory, the evaluation team identified tensions between the idea of institutionalizing key features of the Avahan program, and ensuring program flexibility to promote sustainability. The transition was planned in three rounds allowing for adaptations to transition intervention and program design during the transition period. The assessment team found it important to track these changes in order to understand which strategies and contextual features supported transition. A mixed methods evaluation was employed, combining semi-structured surveys of transitioning entities (conducted pre and post transition), with longitudinal case studies. Qualitative data helped explain quantitative findings. Measures of transition readiness appeared robust, but we were uncertain of the robustness of institutionalization measures. Finally, challenges to the implementation of such an evaluation are discussed. Given the scarceness of transition evaluations, the lessons from this evaluation may have widespread relevance. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Evaluating International Research Ethics Capacity Development: An Empirical Approach

PubMed Central

Ali, Joseph; Kass, Nancy E.; Sewankambo, Nelson K.; White, Tara D.; Hyder, Adnan A.

2014-01-01

The US National Institutes of health, Fogarty International Center (NIH-FIC) has, for the past 13 years, been a leading funder of international research ethics education for resource-limited settings. Nearly half of the NIH-FIC funding in this area has gone to training programs that train individuals from sub-Saharan Africa. Identifying the impact of training investments, as well as the potential predictors of post-training success, can support curricular decision-making, help establish funding priorities, and recognize the ultimate outcomes of trainees and training programs. Comprehensive evaluation frameworks and targeted evaluation tools for bioethics training programs generally, and for international research ethics programs in particular, are largely absent from published literature. This paper shares an original conceptual framework, data collection tool, and detailed methods for evaluating the inputs, processes, outputs, and outcomes of research ethics training programs serving individuals in resource-limited settings. This paper is part of a collection of papers analyzing the Fogarty International Center’s International Research Ethics Education and Curriculum Development program. PMID:24782071

Evaluating a complex, multi-site, community-based program to improve healthcare quality: the summative research design for the Aligning Forces for Quality initiative.

PubMed

Scanlon, Dennis P; Wolf, Laura J; Alexander, Jeffrey A; Christianson, Jon B; Greene, Jessica; Jean-Jacques, Muriel; McHugh, Megan; Shi, Yunfeng; Leitzell, Brigitt; Vanderbrink, Jocelyn M

2016-08-01

The Aligning Forces for Quality (AF4Q) initiative was the Robert Wood Johnson Foundation's (RWJF's) signature effort to increase the overall quality of healthcare in targeted communities throughout the country. In addition to sponsoring this 16-site complex program, RWJF funded an independent scientific evaluation to support objective research on the initiative's effectiveness and contributions to basic knowledge in 5 core programmatic areas. The research design, data, and challenges faced during the summative evaluation phase of this near decade-long program are discussed. A descriptive overview of the summative research design and its development for a multi-site, community-based, healthcare quality improvement initiative is provided. The summative research design employed by the evaluation team is discussed. The evaluation team's summative research design involved a data-driven assessment of the effectiveness of the AF4Q program at large, assessments of the impact of AF4Q in the specific programmatic areas, and an assessment of how the AF4Q alliances were positioned for the future at the end of the program. The AF4Q initiative was the largest privately funded community-based healthcare improvement initiative in the United States to date and was implemented at a time of rapid change in national healthcare policy. The implementation of large-scale, multi-site initiatives is becoming an increasingly common approach for addressing problems in healthcare. The summative evaluation research design for the AF4Q initiative, and the lessons learned from its approach, may be valuable to others tasked with evaluating similarly complex community-based initiatives.

Using Ripple Effect Mapping to Evaluate Program Impact: Choosing or Combining the Methods That Work Best for You

ERIC Educational Resources Information Center

Emery, Mary; Higgins, Lorie; Chazdon, Scott; Hansen, Debra

2015-01-01

A mind mapping approach to evaluation called Ripple Effects Mapping (REM) has been developed and used by a number of Extension faculty across the country recently. This article describes three approaches to REM, as well as key differences and similarities. The authors, each from different land-grant institutions, believe REM is an effective way to…

Framework for a National Testing and Evaluation Program ...

EPA Pesticide Factsheets

Abstract:The National STEPP Program seeks to improve water quality by accelerating the effective implementation and adoption of innovative stormwater management technologies. Itwill attempt to accomplish this by establishing practices through highly reliable, and cost-effective Stormwater control measures (SCM) testing, evaluation, and verification services. The program will aim to remove barriers to innovation, minimize duplicative performance evaluation needs, increase confidence that regulatory requirements are met by creating consistency among testing and evaluation protocols, and establishing equity between public domain and proprietary SCM evaluation approaches.The Environmental Technology Verification Program, established by the U.S. Environmental Protection Agency (EPA) 18 years ago, was the only national program of its kindin the stormwater sector, but is now defunct, leaving a national leadership void. The STEPP initiative was triggered in part by regulatory demands in the government and private sectors to fill this vacuum. A concerted focus and study of this matter led to the release of a Water Environment Federation (WEF) white paper entitled “Investigation into the Feasibility of a National Testing and Evaluation Program for Stormwater Products and Practices” in February 2014. During this second phase of the STEPP initiative, and with EPA support, five analogous technology evaluation programs related to both stormwater and non-stormwater were an

EXPLORATORY STUDY AND PLANNING FOR A FOURTH-YEAR, PAID INTERNSHIP TEACHER EDUCATION PROGRAM FOR BOTH ELEMENTARY AND SECONDARY TEACHERS. FINAL REPORT.

ERIC Educational Resources Information Center

RYERSON, NORMAN E.

THROUGH THE DEVELOPMENT OF A COOPERATIVE COLLEGE-UNIVERSITY GRADUATE STUDY PROGRAM, THE AUGMENTATION OF A PAID FOURTH-YEAR INTERNSHIP APPROACH IN A TEACHER TRAINING PROGRAM WAS EXPLORED. STEPS INCLUDED--(1) INVOLVEMENT OF THE FACULTY IN AN EVALUATION OF THE TEACHER EDUCATION PROGRAM AND THE COLLEGE CURRICULUM, (2) CHANGING WEEKLY SCHEDULES AND THE…

The Educational Programs Audit Dress Rehearsal; Paradigm One: Practice Makes Perfect or How a New Approach to the Audit Helps Programs Succeed.

ERIC Educational Resources Information Center

Pfeffer, Eileen; Kester, Donald L.

Described is a procedure (Audit Dress Rehearsal) used in a special education program audit consultation service which included a practice audit designed to lower anxiety and raise awareness of concern for program success. The introduction includes sections dealing with evaluation and audit personnel, planning and implementing an audit, and stages…

The Center For Medicare And Medicaid Innovation's blueprint for rapid-cycle evaluation of new care and payment models.

PubMed

Shrank, William

2013-04-01

The Affordable Care Act established the Center for Medicare and Medicaid Innovation to test innovative payment and service delivery models. The goal is to reduce program expenditures while preserving or improving the quality of care provided to beneficiaries of Medicare, Medicaid, and the Children's Health Insurance Program. Central to the success of the Innovation Center is a new, rapid-cycle approach to evaluation. This article describes that approach--setting forth how the Rapid Cycle Evaluation Group aims to deliver frequent feedback to providers in support of continuous quality improvement, while rigorously evaluating the outcomes of each model tested. This article also describes the relationship between the group's work and that of the Office of the Actuary at the Centers for Medicare and Medicaid Services, which plays a central role in the assessment of new models.

Causal inferences on the effectiveness of complex social programs: Navigating assumptions, sources of complexity and evaluation design challenges.

PubMed

Chatterji, Madhabi

2016-12-01

This paper explores avenues for navigating evaluation design challenges posed by complex social programs (CSPs) and their environments when conducting studies that call for generalizable, causal inferences on the intervention's effectiveness. A definition is provided of a CSP drawing on examples from different fields, and an evaluation case is analyzed in depth to derive seven (7) major sources of complexity that typify CSPs, threatening assumptions of textbook-recommended experimental designs for performing impact evaluations. Theoretically-supported, alternative methodological strategies are discussed to navigate assumptions and counter the design challenges posed by the complex configurations and ecology of CSPs. Specific recommendations include: sequential refinement of the evaluation design through systems thinking, systems-informed logic modeling; and use of extended term, mixed methods (ETMM) approaches with exploratory and confirmatory phases of the evaluation. In the proposed approach, logic models are refined through direct induction and interactions with stakeholders. To better guide assumption evaluation, question-framing, and selection of appropriate methodological strategies, a multiphase evaluation design is recommended. Copyright © 2016 Elsevier Ltd. All rights reserved.

The Use of the Data-to-Action Framework in the Evaluation of CDC's DELTA FOCUS Program.

PubMed

Armstead, Theresa L; Kearns, Megan; Rambo, Kirsten; Estefan, Lianne Fuino; Dills, Jenny; Rivera, Moira S; El-Beshti, Rasha

The Centers for Disease Control and Prevention's (CDC's) Domestic Violence Prevention Enhancements and Leadership Through Alliances, Focusing on Outcomes for Communities United with States (DELTA FOCUS) program is a 5-year cooperative agreement (2013-2018) funding 10 state domestic violence coalitions and local coordinated community response teams to engage in primary prevention of intimate partner violence. Grantees' prevention strategies were often developmental and emergent; therefore, CDC's approach to program oversight, administration, and support to grantees required a flexible approach. CDC staff adopted a Data-to-Action Framework for the DELTA FOCUS program evaluation that supported a culture of learning to meet dynamic and unexpected information needs. Briefly, a Data-to-Action Framework involves the collection and use of information in real time for program improvement. Utilizing this framework, the DELTA FOCUS data-to-action process yielded important insights into CDC's ongoing technical assistance, improved program accountability by providing useful materials, and information for internal agency leadership, and helped build a learning community among grantees. CDC and other funders, as decision makers, can promote program improvements that are data-informed by incorporating internal processes supportive of ongoing data collection and review.

Steps towards incorporating heterogeneities into program theory: A case study of a data-driven approach.

PubMed

Sridharan, Sanjeev; Jones, Bobby; Caudill, Barry; Nakaima, April

2016-10-01

This paper describes a framework that can help refine program theory through data explorations and stakeholder dialogue. The framework incorporates the following steps: a recognition that program implementation might need to be multi-phased for a number of interventions, the need to take stock of program theory, the application of pattern recognition methods to help identify heterogeneous program mechanisms, and stakeholder dialogue to refine the program. As part of the data exploration, a method known as developmental trajectories is implemented to learn about heterogeneous trajectories of outcomes in longitudinal evaluations. This method identifies trajectory clusters and also can estimate different treatment impacts for the various groups. The framework is highlighted with data collected in an evaluation of an alcohol risk-reduction program delivered in a college fraternity setting. The framework discussed in the paper is informed by a realist focus on "what works for whom under what contexts." The utility of the framework in contributing to a dialogue on heterogeneous mechanism and subsequent implementation is described. The connection of the ideas in paper to a 'learning through principled discovery' approach is also described. Copyright © 2016. Published by Elsevier Ltd.

Michigan English Study of Structure for Curriculum Evaluation (MESSAGE).

ERIC Educational Resources Information Center

Taylor, Vi Marie

The Michigan English Study of Structure for Curriculum Evaluation (MESSAGE) training session involved 40 leaders of English curriculum study in a 7-day intensive program. They studied objective evaluation in terms of behavior, instruction, and institution; approaches to the English curriculum through instructional objectives that include…

76 FR 14399 - Agency Information Collection Request; 60-Day Public Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2011-03-16

.... Proposed Project: Evaluation of Adolescent Pregnancy Prevention Approaches and the Impact Evaluation of the Teen Pregnancy Prevention Program Grantees: Baseline Data Collection--OMB No. OS-0990--NEW-- Office of... overseeing and coordinating adolescent pregnancy prevention evaluation efforts as part of the Teen Pregnancy...

Program Evaluation Theory and Practice: A Comprehensive Guide

ERIC Educational Resources Information Center

Mertens, Donna M.; Wilson, Amy T.

2012-01-01

This engaging text takes an evenhanded approach to major theoretical paradigms in evaluation and builds a bridge from them to evaluation practice. Featuring helpful checklists, procedural steps, provocative questions that invite readers to explore their own theoretical assumptions, and practical exercises, the book provides concrete guidance for…

The Handbook of Leadership Development Evaluation

ERIC Educational Resources Information Center

Hannum, Kelly M., Ed.; Martineau, Jennifer W., Ed.; Reinelt, Claire, Ed.

2007-01-01

With the increase in the number of organizational leadership development programs, there is a pressing need for evaluation to answer important questions, improve practice, and inform decisions. The Handbook is a comprehensive resource filled with examples, tools, and the most innovative models and approaches designed to evaluate leadership…

Facilitators and barriers to students' learning in an obesity prevention graduate program.

PubMed

Do, Kieu Anh; Anderson-Knott, Mindy; de Guzman, Maria Rosario T; Boeckner, Linda; Koszewski, Wanda

2018-01-01

Childhood obesity is a major public health concern with underpinnings at the individual, family, community and societal levels. The Transdisciplinary Childhood Obesity Prevention Graduate Certificate Program (TOP) is an innovative graduate-level certificate program developed to train professionals to understand and address obesity from multiple perspectives using an interprofessional education (IPE) approach. Currently, there is limited knowledge on what promotes or hinders learning in IPE approaches dealing with obesity prevention. The goal of this report is to address this gap by describing facilitators and barriers to learning in a graduate-level training program. Using a qualitative research design, semi-structured interviews were collected from 23 professional students, as part of a larger program evaluation project for TOP. Thematic analysis revealed the challenges and strengths of the program that relate specifically to: its interprofessional approach, its structure, and its activities. Interprofessional exchanges were reported to expand students' learning, but adequate interprofessional representation must be maintained, and the complexity of interprofessional collaborations must also be well-coordinated. Standardising the program structure and courses for consistency across professions, and clear communication are critical to program success. Findings add to the existing literature on what promotes effective learning in a professional obesity prevention program using an IPE approach.

Rehabilitation of the elbow following sports injury.

PubMed

Ellenbecker, Todd S; Pieczynski, Tad E; Davies, George J

2010-01-01

Evaluation of the athlete with an elbow injury involves a complete upper extremity approach and a corresponding treatment approach that addresses the identified deficiencies to restore normal function. A significant focus should be placed on the proximal aspect of the upper extremity in addition to the obvious distal injury. A detailed review of the available treatment modalities fails to identify any clear definitive choice to address pain levels; however, a combination of modalities and appropriate exercise can be used in the early rehabilitation phases. The use of a total arm strengthening program along with evaluation of the athlete's sport mechanics is required to successfully return the patient back to their preinjury level of function. A supervised interval sport return program is also a necessary component of the complete rehabilitation program for the athlete with an elbow injury.

Using a Mixed-Methods RE-AIM Framework to Evaluate Community Health Programs for Older Latinas.

PubMed

Schwingel, Andiara; Gálvez, Patricia; Linares, Deborah; Sebastião, Emerson

2017-06-01

This study used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework to evaluate a promotora-led community health program designed for Latinas ages 50 and older that sought to improve physical activity, nutrition, and stress management. A mixed-methods evaluation approach was administered at participant and organizational levels with a focus on the efficacy, adoption, implementation, and maintenance components of the RE-AIM theoretical model. The program was shown to be effective at improving participants' eating behaviors, increasing their physical activity levels, and lowering their depressive symptoms. Promotoras felt motivated and sufficiently prepared to deliver the program. Some implementation challenges were reported. More child care opportunities and an increased focus on mental well-being were suggested. The promotora delivery model has promise for program sustainability with both promotoras and participants alike expressing interest in leading future programs.

IEP: Developing Criteria for the Evaluation of Individualized Education Program Provisions. Exploring Issues in the Implementation of P.L. 94-142.

ERIC Educational Resources Information Center

LINC Resources, Inc., Columbus, OH.

Four papers address approaches to evaluate the implementation of the individualized education program (IEP) provisions of P.L. 94-142, The Education for All Handicapped Children Act. The papers are "Auditing the IEP System: A Self-Audit System for Use by Local Education Agencies" by Beth Stephens and Daniel Macy, which examines a 10-step…

Return on investment in healthcare leadership development programs.

PubMed

Jeyaraman, Maya M; Qadar, Sheikh Muhammad Zeeshan; Wierzbowski, Aleksandra; Farshidfar, Farnaz; Lys, Justin; Dickson, Graham; Grimes, Kelly; Phillips, Leah A; Mitchell, Jonathan I; Van Aerde, John; Johnson, Dave; Krupka, Frank; Zarychanski, Ryan; Abou-Setta, Ahmed M

2018-02-05

Purpose Strong leadership has been shown to foster change, including loyalty, improved performance and decreased error rates, but there is a dearth of evidence on effectiveness of leadership development programs. To ensure a return on the huge investments made, evidence-based approaches are needed to assess the impact of leadership on health-care establishments. As a part of a pan-Canadian initiative to design an effective evaluative instrument, the purpose of this paper was to identify and summarize evidence on health-care outcomes/return on investment (ROI) indicators and metrics associated with leadership quality, leadership development programs and existing evaluative instruments. Design/methodology/approach The authors performed a scoping review using the Arksey and O'Malley framework, searching eight databases from 2006 through June 2016. Findings Of 11,868 citations screened, the authors included 223 studies reporting on health-care outcomes/ROI indicators and metrics associated with leadership quality (73 studies), leadership development programs (138 studies) and existing evaluative instruments (12 studies). The extracted ROI indicators and metrics have been summarized in detail. Originality/value This review provides a snapshot in time of the current evidence on ROI indicators and metrics associated with leadership. Summarized ROI indicators and metrics can be used to design an effective evaluative instrument to assess the impact of leadership on health-care organizations.

Evaluation of a supervisor training program for ODOT's EcoDrive program.

DOT National Transportation Integrated Search

2016-12-01

Eco-driving consists of using energy-efficient approaches to driving aimed at reducing : fuel consumption and, ultimately, CO2 emissions. A previous study found that an EcoDrive : informational campaign was effective at increasing the use of eco-driv...

A Description of the Hawthorn Center Early Intervention Program.

ERIC Educational Resources Information Center

Glovinsky, Ira; Keller, Jackie

An early intervention project for preschool handicapped children is described. Specific program goals (including improved functioning in social, emotional, communication, and preacademic areas) are listed, and roles of the interdisciplinary staff members are considered. Among evaluation approaches discussed are psychological measures, family…

GHG MITIGATION TECHNOLOGY PERFORMANCE EVALUATIONS UNDERWAY AT THE GHG TECHNOLOGY VERIFICATION CENTER

EPA Science Inventory

The paper outlines the verification approach and activities of the Greenhouse Gas (GHG) Technology Verification Center, one of 12 independent verification entities operating under the U.S. EPA-sponsored Environmental Technology Verification (ETV) program. (NOTE: The ETV program...

A software technology evaluation program

NASA Technical Reports Server (NTRS)

Novaes-Card, David N.

1985-01-01

A set of quantitative approaches is presented for evaluating software development methods and tools. The basic idea is to generate a set of goals which are refined into quantifiable questions which specify metrics to be collected on the software development and maintenance process and product. These metrics can be used to characterize, evaluate, predict, and motivate. They can be used in an active as well as passive way by learning form analyzing the data and improving the methods and tools based upon what is learned from that analysis. Several examples were given representing each of the different approaches to evaluation. The cost of the approaches varied inversely with the level of confidence in the interpretation of the results.

Evaluation and Use of Water Monitoring Data in Pesticide Aquatic Exposure Assessments

EPA Pesticide Factsheets

The EPA Office of Pesticide Programs uses a tiered approach to risk assessment. The tiered approach screens out low-risk pesticides and focuses refined assessments and resources on pesticides most likely to pose a risk of concern.

A Preliminary Evaluation of Federal Dental Manpower Subsidy Program. Health Manpower Policy Discussion Paper Series, No. A5.

ERIC Educational Resources Information Center

Feldstein, Paul J.

This paper suggests an approach and an analysis of the impact that federal manpower programs have had on dentistry. The main beneficiaries under health manpower subsidy programs are indicated and some policy alternatives suggested. Emphasis is placed on federal subsidies for increasing the supply of dentists including students' financial…

Visualization of Concurrent Program Executions

NASA Technical Reports Server (NTRS)

Artho, Cyrille; Havelund, Klaus; Honiden, Shinichi

2007-01-01

Various program analysis techniques are efficient at discovering failures and properties. However, it is often difficult to evaluate results, such as program traces. This calls for abstraction and visualization tools. We propose an approach based on UML sequence diagrams, addressing shortcomings of such diagrams for concurrency. The resulting visualization is expressive and provides all the necessary information at a glance.

Learning Through Survival: An Approach to Foreign Language Teaching. A Program Description and Evaluation.

ERIC Educational Resources Information Center

Travers, Claudia S.; Reilly, Joseph T. M.

A foreign travel program for eighth grade French language students from Shoreham, New York was developed to provide language speaking practice in Quebec, Canada. The program was designed to allow students to spend as much time as possible speaking the language and interacting with the people. The following "survival through speaking"…

Coming Home: A Group-Based Approach for Assisting Military Veterans in Transition

ERIC Educational Resources Information Center

Westwood, Marvin J.; McLean, Holly; Cave, Douglas; Borgen, William; Slakov, Paul

2010-01-01

This study is an evaluation of the Veterans Transition Program, a residential, group-based program designed to assist the transition of military personnel back into Canadian society by aiding with their personal and career readjustment. Participants in the program included 18 male soldiers who experienced varying degrees of combat-related trauma.…

Helping Low-Achieving First-Grade Readers: A Program Combining Reading Recovery Tutoring and Small-Group Instruction.

ERIC Educational Resources Information Center

Dorn, Linda; Allen, Anne

1995-01-01

Evaluates an approach that supplemented existing Reading Recovery programs with small-group, early literacy instruction in 28 Arkansas public schools. The program enabled many children to receive timely support. When space became available in Reading Recovery, these children made accelerated progress and were discontinued earlier than children who…

Supporting research and technology

NASA Technical Reports Server (NTRS)

1971-01-01

The development of definition of the modular space station is discussed. The modular approach was evaluated, the requirements were defined, and program definition and design were accomplished. The features of the program which significantly affect the initial development and early operating costs were identified and their impacts on the program were assessed. Specifications of various systems and components are included.

Lessons learned using a values-engaged approach to attend to culture, diversity, and equity in a STEM program evaluation.

PubMed

Boyce, Ayesha S

2017-10-01

Evaluation must attend meaningfully and respectfully to issues of culture, race, diversity, power, and equity. This attention is especially critical within the evaluation of science, technology, engineering, and mathematics (STEM) educational programming, which has an explicit agenda of broadening participation. The purpose of this article is to report lessons learned from the implementation of a values-engaged, educative (Greene et al., 2006) evaluation within a multi-year STEM education program setting. This meta-evaluation employed a case study design using data from evaluator weekly systematic reflections, review of evaluation and program artifacts, stakeholder interviews, and peer review and assessment. The main findings from this study are (a) explicit attention to culture, diversity, and equity was initially challenged by organizational culture and under-developed evaluator-stakeholder professional relationship and (b) evidence of successful engagement of culture, diversity, and equity emerged in formal evaluation criteria and documents, and informal dialogue and discussion with stakeholders. The paper concludes with lessons learned and implications for practice. Copyright © 2017 Elsevier Ltd. All rights reserved.

A combined impact-process evaluation of a program promoting active transport to school: understanding the factors that shaped program effectiveness.

PubMed

Crawford, S; Garrard, J

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts.

A Combined Impact-Process Evaluation of a Program Promoting Active Transport to School: Understanding the Factors That Shaped Program Effectiveness

PubMed Central

Crawford, S.; Garrard, J.

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts. PMID:23606865

Development and implementation of a competency-based clinical evaluation tool for midwifery education.

PubMed

Woeber, Kate

2018-07-01

The learning goals and evaluation strategies of competency-based midwifery programs must be explicit and well-defined. In the US, didactic learning is evaluated through a standardized certification examination, but standardized clinical competence evaluation is lacking. The Midwifery Competency Assessment Tool (MCAT) has been adapted from the International Confederation of Midwives' (ICM) "Essential Competencies" and from the American College of Nurse-Midwives' (ACNM) "Core Competencies", with student self-evaluation based on Benner's Novice-to-Expert theory. The MCAT allows for the measurement and monitoring of competence development in all domains of full-scope practice over the course of the midwifery program. Strengths of the MCAT are that it provides clear learning goals and performance evaluations for students, ensures and communicates content mapping across a curriculum, and highlights strengths and gaps in clinical opportunities at individual clinical sites and for entire programs. Challenges of the MCAT lie in balancing the number of competency items to be measured with the tedium of form completion, in ensuring the accuracy of student self-evaluation, and in determining "adequate" competence achievement when particular clinical opportunities are limited. Use of the MCAT with competency-based clinical education may facilitate a more standardized approach to clinical evaluation, as well as a more strategic approach to clinical site development and use. Copyright © 2018 Elsevier Ltd. All rights reserved.

Improving the Impact and Implementation of Disaster Education: Programs for Children Through Theory-Based Evaluation.

PubMed

Johnson, Victoria A; Ronan, Kevin R; Johnston, David M; Peace, Robin

2016-11-01

A main weakness in the evaluation of disaster education programs for children is evaluators' propensity to judge program effectiveness based on changes in children's knowledge. Few studies have articulated an explicit program theory of how children's education would achieve desired outcomes and impacts related to disaster risk reduction in households and communities. This article describes the advantages of constructing program theory models for the purpose of evaluating disaster education programs for children. Following a review of some potential frameworks for program theory development, including the logic model, the program theory matrix, and the stage step model, the article provides working examples of these frameworks. The first example is the development of a program theory matrix used in an evaluation of ShakeOut, an earthquake drill practiced in two Washington State school districts. The model illustrates a theory of action; specifically, the effectiveness of school earthquake drills in preventing injuries and deaths during disasters. The second example is the development of a stage step model used for a process evaluation of What's the Plan Stan?, a voluntary teaching resource distributed to all New Zealand primary schools for curricular integration of disaster education. The model illustrates a theory of use; specifically, expanding the reach of disaster education for children through increased promotion of the resource. The process of developing the program theory models for the purpose of evaluation planning is discussed, as well as the advantages and shortcomings of the theory-based approaches. © 2015 Society for Risk Analysis.

Verification approach for the Shuttle/Payload Contamination Evaluation computer program - Spacelab induced environment

NASA Technical Reports Server (NTRS)

Bareiss, L. E.

1978-01-01

The paper presents a compilation of the results of a systems level Shuttle/payload contamination analysis and related computer modeling activities. The current technical assessment of the contamination problems anticipated during the Spacelab program are discussed and recommendations are presented on contamination abatement designs and operational procedures based on experience gained in the field of contamination analysis and assessment, dating back to the pre-Skylab era. The ultimate test of the Shuttle/Payload Contamination Evaluation program will be through comparison of predictions with measured levels of contamination during actual flight.

PubMed Central

Baril, J.; Théoret, J.

1997-01-01

PROBLEM BEING ADDRESSED: Family medicine residents have difficulty developing the complex skills needed to work with families. OBJECTIVE OF PROGRAM: To develop an educational program to teach family medicine residents at Laval University a new type of family intervention technique, based on the systems approach, using a practical and interactive method. MAIN COMPONENTS OF PROGRAM: Using Brien's systematic planning model, the authors developed and tested a series of six interactive workshops on three themes. The first theme aims to motivate residents by showing them why it is important to use a systems approach with their patients. The second theme explores a family situation in an individual interview. The third enables residents to conduct a family interview. The whole program has been tested and evaluated. All of the materials needed to teach these skills are contained in a trainer's guide and a video cassette. CONCLUSION: This educational program's originality lies in its combination of the systems approach and interactive training for residents. The program could easily be presented as a continuing medical education activity. PMID:9333406

Formative Process Evaluation for Implementing a Social Marketing Intervention to Increase Walking Among African Americans in the Positive Action for Today’s Health Trial

PubMed Central

Wilson, Dawn K.; Griffin, Sarah; St. George, Sara M.; Alia, Kassandra A.; Trumpeter, Nevelyn N.; Wandersman, Abraham K.; Forthofer, Melinda; Robinson, Shamika; Gadson, Barney

2012-01-01

Objectives. Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today’s Health trial. Methods: We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. Results: Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. Conclusions: Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities. PMID:23078486

Evidence from the field: Findings on issues related to planning, implementing and evaluating gender-based programs.

PubMed

Kowalczyk, Shelly; Randolph, Suzanne; Stokes, Shereitte; Winston, Stefanie

2015-08-01

An Initiative of the United States Department of Health and Human Services' Office on Women's Health (OWH), Coalition for a Healthier Community (CHC), supports ten grantees across the U.S. in the implementation of gender-based health interventions targeting women and girls. A national evaluation is assessing whether gender-focused public health systems approaches are sustainable and cost effective in addressing health disparities in women and girls. To inform the evaluation, a systematic examination was conducted of literature in both the public and private sector designed to track, assess, understand, and improve women's health, public health systems approaches, and the cost-effectiveness and sustainability of gender-based programs. A two-person team assured the quality of the results following the review of abstracts and full-text articles. Of 123 articles meeting eligibility criteria (See inclusion criteria described in Section 2.2 below), only 18 met inclusion criteria specific to a focus on a systems approach, cost-effectiveness and/or sustainability. Studies assessing systems approaches suggested their effectiveness in changing perceptions and increasing knowledge within a community; increasing involvement of local decision-makers and other community leaders in women's health issues; and increasing community capacity to address women and girls' health. Further evaluation of the cost-effectiveness and sustainability of gender-based approaches is needed. Copyright © 2014 Elsevier Ltd. All rights reserved.

Strategies for more effective monitoring and evaluation systems in HIV programmatic scale-up in resource-limited settings: Implications for health systems strengthening.

PubMed

Nash, Denis; Elul, Batya; Rabkin, Miriam; Tun, May; Saito, Suzue; Becker, Mark; Nuwagaba-Biribonwoha, Harriet

2009-11-01

Program monitoring and evaluation (M&E) has the potential to be a cornerstone of health systems strengthening and of evidence-informed implementation and scale-up of HIV-related services in resource-limited settings. We discuss common challenges to M&E systems used in the rapid scale-up of HIV services as well as innovations that may have relevance to systems used to monitor, evaluate, and inform health systems strengthening. These include (1) Web-based applications with decentralized data entry and real-time access to summary reporting; (2) timely feedback of information to site and district staff; (3) site-level integration of traditionally siloed program area indicators; (4) longitudinal tracking of program and site characteristics; (5) geographic information systems; and (6) use of routinely collected aggregate data for epidemiologic analysis and operations research. Although conventionally used in the context of vertical programs, these approaches can form a foundation on which data relevant to other health services and systems can be layered, including prevention services, primary care, maternal-child health, and chronic disease management. Guiding principles for sustainable national M&E systems include country-led development and ownership, support for national programs and policies, interoperability, and employment of an open-source approach to software development.

Oversight mechanisms in public managed care programs: from little oversight to negotiations to shared decision-making.

PubMed

Beinecke, R H

1999-01-01

An expanded range of oversight mechanisms is being adopted to hold public human service programs more accountable to funding sources as well as consumers, family members, and providers. Most of these approaches are hierarchical in nature. Some involve negotiated agreements and each is designed to meet certain goals and functions. Each utilizes different forms of decision-making. Stakeholders prefer to be part of a shared decision-making process. Understanding these underlying premises can help to assess the strengths and weaknesses of each method and can suggest how to most effectively utilize combinations of approaches to improve program performance. Whether we will move toward a new paradigm emphasizing participation and collaboration rather than more formal structural approaches is yet undetermined but will greatly affect how programs are monitored and evaluated in the future.

Evaluating the Impact of Implementation Factors on Family-Based Prevention Programming: Methods for Strengthening Causal Inference

PubMed Central

Crowley, D. Max; Coffman, Donna L.; Feinberg, Mark; Greenberg, Mark; Spoth, Richard

2013-01-01

Despite growing recognition of the important role implementation plays in successful prevention efforts, relatively little work has sought to demonstrate a causal relationship between implementation factors and participant outcomes. In turn, failure to explore the implementation-to-outcome link limits our understanding of the mechanisms essential to successful programming. This gap is partially due to the inability of current methodological procedures within prevention science to account for the multitude of confounders responsible for variation in implementation factors (i.e., selection bias). The current paper illustrates how propensity and marginal structural models can be used to improve causal inferences involving implementation factors not easily randomized (e.g., participant attendance). We first present analytic steps for simultaneously evaluating the impact of multiple implementation factors on prevention program outcome. Then we demonstrate this approach for evaluating the impact of enrollment and attendance in a family program, over and above the impact of a school-based program, within PROSPER, a large scale real-world prevention trial. Findings illustrate the capacity of this approach to successfully account for confounders that influence enrollment and attendance, thereby more accurately representing true causal relations. For instance, after accounting for selection bias, we observed a 5% reduction in the prevalence of 11th grade underage drinking for those who chose to receive a family program and school program compared to those who received only the school program. Further, we detected a 7% reduction in underage drinking for those with high attendance in the family program. PMID:23430578

SCRL-Model for Human Space Flight Operations Enterprise Supply Chain

NASA Technical Reports Server (NTRS)

Tucker, Brian

2010-01-01

Standard approach to evaluate and configure adaptable and sustainable program and mission supply chains at an enterprise level. End-to-end view. Total Lifecycle. Evaluate the readiness of the supply chain during the supply chain development phase.

Quality Control in Title I Evaluation: Problems and Approaches.

ERIC Educational Resources Information Center

Hansen, Joe B.

Educational program evalaution is subject to many difficulties. Problems with evaluation include such conceptual problems as: (1) understanding of decisions to be made; (2) adequate definition; (3) consensus on values and criteria; (4) coordination among administrative levels; and (5) use of an appropriate evaluation model. Technical problems…

Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use

ERIC Educational Resources Information Center

Patton, Michael Quinn

2010-01-01

Developmental evaluation (DE) offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority shows how to conduct evaluations within a DE framework. Patton draws on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He…

A web-based program for informal caregivers of persons with Alzheimer's disease: an iterative user-centered design.

PubMed

Cristancho-Lacroix, Victoria; Moulin, Florence; Wrobel, Jérémy; Batrancourt, Bénédicte; Plichart, Matthieu; De Rotrou, Jocelyne; Cantegreil-Kallen, Inge; Rigaud, Anne-Sophie

2014-09-15

Web-based programs have been developed for informal caregivers of people with Alzheimer's disease (PWAD). However, these programs can prove difficult to adopt, especially for older people, who are less familiar with the Internet than other populations. Despite the fundamental role of usability testing in promoting caregivers' correct use and adoption of these programs, to our knowledge, this is the first study describing this process before evaluating a program for caregivers of PWAD in a randomized clinical trial. The objective of the study was to describe the development process of a fully automated Web-based program for caregivers of PWAD, aiming to reduce caregivers' stress, and based on the user-centered design approach. There were 49 participants (12 health care professionals, 6 caregivers, and 31 healthy older adults) that were involved in a double iterative design allowing for the adaptation of program content and for the enhancement of website usability. This process included three component parts: (1) project team workshops, (2) a proof of concept, and (3) two usability tests. The usability tests were based on a mixed methodology using behavioral analysis, semistructured interviews, and a usability questionnaire. The user-centered design approach provided valuable guidelines to adapt the content and design of the program, and to improve website usability. The professionals, caregivers (mainly spouses), and older adults considered that our project met the needs of isolated caregivers. Participants underlined that contact between caregivers would be desirable. During usability observations, the mistakes of users were also due to ergonomics issues from Internet browsers and computer interfaces. Moreover, negative self-stereotyping was evidenced, when comparing interviews and results of behavioral analysis. Face-to-face psycho-educational programs may be used as a basis for Web-based programs. Nevertheless, a user-centered design approach involving targeted users (or their representatives) remains crucial for their correct use and adoption. For future user-centered design studies, we recommend to involve end-users from preconception stages, using a mixed research method in usability evaluations, and implementing pilot studies to evaluate acceptability and feasibility of programs.

Evaluation of a Speed Mentoring Program: Achievement of Short-Term Mentee Goals and Potential for Longer-Term Relationships.

PubMed

Cellini, Melissa M; Serwint, Janet R; D'Alessandro, Donna M; Schulte, Elaine E; Osman, Cynthia

2017-07-01

Speed mentoring provides brief mentoring and networking opportunities. We evaluated 1) a national speed mentoring program's ability to encourage in-person networking and advice-sharing, and 2) 2 potential outcomes: helping mentees achieve 3-month goals, and fostering mentoring relationships after the program. An outcome approach logic model guided our program evaluation. Sixty mentees and 60 mentors participated. Each mentee met with 6 mentors for 10 minutes per pairing. At the program, mentees created goals. At 3 months, mentors sent mentees a reminder e-mail. At 4 months, participants received a Web-based survey. Forty-two (70%) mentees and 46 (77%) mentors completed the survey. Participants reported the program allowed them to share/receive advice, to network, to provide/gain different perspectives, and to learn from each other. Mentors as well as mentees identified shared interests, mentor-mentee chemistry, mentee initiative, and mentor approachability as key qualities contributing to ongoing relationships. Many mentor-mentee dyads had additional contact (approximately 60%) after the program and approximately one-third thought they were likely to continue the relationship. Goal-setting encouraged subsequent mentor-mentee contact and motivated mentees to work toward attaining their 3-month goals. The mentors aided mentees goal attainment by providing advice, offering support, and holding mentees accountable. A national speed mentoring program was an effective and efficient way to establish national connections, obtain different perspectives, and receive advice. Goal-setting helped mentees in achieving 3-month goals and fostering mentoring relationships outside of the program. These elements continue to be a part of this program and might be valuable for similar programs. Copyright © 2017 Academic Pediatric Association. Published by Elsevier Inc. All rights reserved.

Survey of university programs in remote sensing funded under grants from the NASA University-Space Applications program

NASA Technical Reports Server (NTRS)

Madigan, J. A.; Earhart, R. W.

1978-01-01

NASA's Office of Space and Terrestrial Applications (OSTA) is currently assessing approaches to transferring NASA technology to both the public and private sectors. As part of this assessment, NASA is evaluating the effectiveness of an ongoing program in remote sensing technology transfer conducted by 20 university contractors/grantees, each supported totally or partially by NASA funds. The University-Space Applications program has as its objective the demonstration of practical benefits from the use of remote sensing technology to a broad spectrum of new users, principally in state and local governments. To evaluate the University-Space Applications program, NASA has a near-term requirement for data on each university effort including total funding, funding sources, length of program, program description, and effectiveness measures.

Process evaluation of two home-based bimanual training programs in children with unilateral cerebral palsy (the COAD-study): protocol for a mixed methods study.

PubMed

Beckers, Laura; van der Burg, Jan; Janssen-Potten, Yvonne; Rameckers, Eugène; Aarts, Pauline; Smeets, Rob

2018-04-24

As part of the COAD-study two home-based bimanual training programs for young children with unilateral Cerebral Palsy (uCP) have been developed, both consisting of a preparation phase and a home-based training phase. Parents are coached to use either an explicit or implicit motor learning approach while teaching bimanual activities to their child. A process evaluation of these complex interventions is crucial in order to draw accurate conclusions and provide recommendations for implementation in clinical practice and further research. The aim of the process evaluation is to systematically assess fidelity of the home-based training programs, to examine the mechanisms that contribute to their effects on child-related and parent-related outcomes, and to explore the influence of contextual factors. A mixed methods embedded design is used that emerges from a pragmatism paradigm. The qualitative strand involves a generic qualitative approach. The process evaluation components fidelity (quality), dose delivered (completeness), dose received (exposure and satisfaction), recruitment and context will be investigated. Data collection includes registration of attendance of therapists and remedial educationalists to a course regarding the home-based training programs; a questionnaire to evaluate this course by the instructor; a report form concerning the preparation phase to be completed by the therapist; registration and video analyses of the home-based training; interviews with parents and questionnaires to be filled out by the therapist and remedial educationalist regarding the process of training; and focus groups with therapists and remedial educationalists as well as registration of drop-out rates and reasons, to evaluate the overall home-based training programs. Inductive thematic analysis will be used to analyse qualitative data. Qualitative and quantitative findings are merged through meta-inference. So far, effects of home-based training programs in paediatric rehabilitation have been studied without an extensive process evaluation. The findings of this process evaluation will have implications for clinical practice and further research regarding development and application of home-based bimanual training programs, executed by parents and aimed at improving activity performance and participation of children with uCP.

The Role of Curricular Approach, Rural-Urban Background, and Socioeconomic Status in Second Language Learning: The Cornwall Area Study

ERIC Educational Resources Information Center

Swain, M.; Barik, H. C.

1978-01-01

Presenting evaluation results of a kindergarten bilingual education program and followup program, this article indicates French immersion can be effective among rural and urban students of both middle-upper and low socioeconomic status. (JC)

A place to call home: study protocol for a longitudinal, mixed methods evaluation of two housing first adaptations in Sydney, Australia.

PubMed

Whittaker, Elizabeth; Swift, Wendy; Flatau, Paul; Dobbins, Timothy; Schollar-Root, Olivia; Burns, Lucinda

2015-04-09

This protocol describes a study evaluating two 'Housing First' programs, Platform 70 and Common Ground, presently being implemented in the inner-city region of Sydney, Australia. The Housing First approach prioritises housing individuals who are homeless in standard lease agreement tenancies as rapidly as possible to lock in the benefits from long-term accommodation, even where the person may not be seen as 'housing ready'. The longitudinal, mixed methods evaluation utilises both quantitative and qualitative data collected at baseline and 12-month follow-up time points. For the quantitative component, clients of each program were invited to complete client surveys that reported on several factors associated with chronic homelessness and were hypothesised to improve under stable housing, including physical and mental health status and treatment rates, quality of life, substance use patterns, and contact with the health and criminal justice systems. Semi-structured interviews with clients and stakeholders comprised the qualitative component and focused on individual experiences with, and perceptions of, the two programs. In addition, program data on housing stability, rental subsidies and support levels provided to clients by agencies was collected and will be used in conjunction with the client survey data to undertake an economic evaluation of the two programs. This study will systematically evaluate the efficacy of a scatter site model (Platform 70) and a congregated model (Common Ground) of the Housing First approach; an examination that has not yet been made either in Australia or internationally. A clear strength of the study is its timing. It was designed and implemented as the programs in question themselves were introduced. Moreover, the programs were introduced when the Australian Government, with State and Territory support, began a more focused, coordinated response to homelessness and funded rapid expansion of innovative homelessness programs across the country, including Common Ground supportive housing developments.

PREVENT: a program of the National Training Initiative on Injury and Violence Prevention.

PubMed

Runyan, Carol W; Gunther-Mohr, Carol; Orton, Stephen; Umble, Karl; Martin, Sandra L; Coyne-Beasley, Tamera

2005-12-01

Training practitioners to use evidence-based approaches to the primary prevention of violence is challenging as a result of the dearth of well-evaluated intervention programs and the lack of familiarity of some practitioners in drawing critically on existing literature. An element of the National Training Initiative in Injury and Violence Prevention, the PREVENT (Preventing Violence Through Education, Networking, and Technical Assistance) program began in late 2003 to train practitioners to address multiple types of violence by encouraging more widespread use of evidence-based approaches to primary prevention. It is intended to reach practitioners involved in addressing violence against women, sexual violence, child maltreatment, youth violence, and suicide in varied community settings. The program uses a combination of varied types of face-to-face training and distance learning coupled with opportunities for networking and technical assistance. Ultimately the program intends to stimulate and facilitate changes in individual, organizational, and cultural awareness and practices fostering primary prevention of violence. The project employs formative, process, and impact evaluation techniques aimed at improving delivery of the training as well as tracking changes in individual and organizations.

Systems Engineering Approach and Metrics for Evaluating Network-Centric Operations for U.S. Army Battle Command

DTIC Science & Technology

2013-07-01

Systems Engineering Approach and Metrics for Evaluating Network-Centric Operations for U.S. Army Battle Command by Jock O. Grynovicki and...Battle Command Jock O. Grynovicki and Teresa A. Branscome Human Research and Engineering Directorate, ARL...NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) Jock O. Grynovicki and Teresa A. Branscome 5d. PROJECT NUMBER 622716H70 5e. TASK NUMBER

Cost-inclusive evaluation: a banquet of approaches for including costs, benefits, and cost-effectiveness and cost-benefit analyses in your next evaluation.

PubMed

Yates, Brian T

2009-02-01

An introduction to the special issue on cost-inclusive evaluation, providing a brief history of the use of costs, benefits, cost-effectiveness, and cost-benefit analyses in the evaluation of human services. Two tables present brief glossaries of terms and analyses common in cost-inclusive program evaluation.

Evaluation of the Monongalia County Schools, Even Start Family Literacy Program. Monongalia County Even Start Program. Parents Are Their Children's First and Most Important Teachers.

ERIC Educational Resources Information Center

Meehan, Merrill L.

The Monongalia County (West Virgina) Even Start Program begun in 1993 to address family literacy and to prepare children for entrance into school, consists of three interrelated approaches: adult education, parenting education, and early childhood education. In June 1996, the program was serving 62 families (mostly low-income) with an average of…

Becoming FASD Informed: Strengthening Practice and Programs Working with Women with FASD

PubMed Central

Rutman, Deborah

2016-01-01

There is growing appreciation among health and social care providers, especially those working in community-based programs with women or young people with substance use problems and/or who have experienced violence, maltreatment, or trauma, that a high number of their program participants may have been prenatally exposed to alcohol or have fetal alcohol spectrum disorder (FASD). This article provides a conceptualization of the key components of an FASD-informed approach. Drawing on the emerging literature and the author’s research identifying the support needs and promising approaches in working with women, young adults, and adults with FASD, as well as evaluations of FASD-related programs, the article discusses what an FASD-informed approach is, why it is centrally important in working with women, adults, and young people who may have FASD, underlying principles of an FASD-informed approach, and examples of FASD-informed adaptations to practice, programming, and the physical environment. In this discussion, the benefits of using an FASD-informed approach for service providers and women living with FASD and their families, as well as conceptualization of FASD-informed policy and systems are highlighted. PMID:27199561

Defining, illustrating and reflecting on logic analysis with an example from a professional development program.

PubMed

Tremblay, Marie-Claude; Brousselle, Astrid; Richard, Lucie; Beaudet, Nicole

2013-10-01

Program designers and evaluators should make a point of testing the validity of a program's intervention theory before investing either in implementation or in any type of evaluation. In this context, logic analysis can be a particularly useful option, since it can be used to test the plausibility of a program's intervention theory using scientific knowledge. Professional development in public health is one field among several that would truly benefit from logic analysis, as it appears to be generally lacking in theorization and evaluation. This article presents the application of this analysis method to an innovative public health professional development program, the Health Promotion Laboratory. More specifically, this paper aims to (1) define the logic analysis approach and differentiate it from similar evaluative methods; (2) illustrate the application of this method by a concrete example (logic analysis of a professional development program); and (3) reflect on the requirements of each phase of logic analysis, as well as on the advantages and disadvantages of such an evaluation method. Using logic analysis to evaluate the Health Promotion Laboratory showed that, generally speaking, the program's intervention theory appeared to have been well designed. By testing and critically discussing logic analysis, this article also contributes to further improving and clarifying the method. Copyright © 2013 Elsevier Ltd. All rights reserved.

Guidelines for Creating, Implementing, and Evaluating Mind-Body Programs in a Military Healthcare Setting.

PubMed

Smith, Katherine; Firth, Kimberly; Smeeding, Sandra; Wolever, Ruth; Kaufman, Joanna; Delgado, Roxana; Bellanti, Dawn; Xenakis, Lea

2016-01-01

Research suggests that the development of mind-body skills can improve individual and family resilience, particularly related to the stresses of illness, trauma, and caregiving. To operationalize the research evidence that mind-body skills help with health and recovery, Samueli Institute, in partnership with experts in mind-body programming, created a set of guidelines for developing and evaluating mind-body programs for service members, veterans, and their families. The Guidelines for Creating, Implementing, and Evaluating Mind-Body Programs in a Military Healthcare Setting outline key strategies and issues to consider when developing, implementing, and evaluating a mind-body focused family empowerment approach in a military healthcare setting. Although these guidelines were developed specifically for a military setting, most of the same principles can be applied to the development of programs in the civilian setting as well. The guidelines particularly address issues unique to mind-body programs, such as choosing evidence-based modalities, licensure and credentialing, safety and contraindications, and choosing evaluation measures that capture the holistic nature of these types of programs. The guidelines are practical, practice-based guidelines, developed by experts in the fields of program development and evaluation, mind-body therapies, patient- and family-centered care, as well as, experts in military and veteran's health systems. They provide a flexible framework to create mind-body family empowerment programs and describe important issues that program developers and evaluators are encouraged to address to ensure the development of the most impactful, successful, evidence-supported programs possible. Copyright © 2016 Elsevier Inc. All rights reserved.

Evaluating the Impact of Training: A Collection of Federal Agency Evaluation Practices.

ERIC Educational Resources Information Center

Salinger, Ruth; Bartlett, Joan

The purpose of this document is to share various approaches used by federal agencies to assess needs and measure training effectiveness. The emphasis in the descriptions is on the evaluation process rather than on the results. One program was evaluated by employing return-on-investment (ROI) data and using volunteer line personnel who conducted…

Automated Writing Evaluation for Non-Native Speaker English Academic Writing: The Case of IADE and Its Formative Feedback

ERIC Educational Resources Information Center

Cotos, Elena

2010-01-01

This dissertation presents an innovative approach to the development and empirical evaluation of Automated Writing Evaluation (AWE) technology used for teaching and learning. It introduces IADE (Intelligent Academic Discourse Evaluator), a new web-based AWE program that analyzes research article Introduction sections and generates immediate,…

Forging a Strategic and Comprehensive Approach to Evaluation within Public and Nonprofit Organizations: Integrating Measurement and Analytics within Evaluation

ERIC Educational Resources Information Center

Newcomer, Kathryn; Brass, Clinton T.

2016-01-01

The "performance movement" has been a subject of enthusiasm and frustration for evaluators. Performance measurement, data analytics, and program evaluation have been treated as different tasks, and those addressing them speak their own languages in their own circles. We suggest that situating performance measurement and data analytics…

Participatory Approaches to Evaluating Outcomes and Designing Curriculum in Workplace Education Programs. The Report of the 1991 Evaluation of the Massachusetts Workplace Education Initiative.

ERIC Educational Resources Information Center

Sperazi, Laura; And Others

The Massachusetts Workplace Education Initiative (MWEI) was begun in 1985 as an inter-agency effort to bring adult basic education and English-as-a-Second-Language (ESL) instruction directly into workplaces throughout the state. The impetus for the program was a concern for large numbers of workers who did not have the skills necessary to compete…

Evaluation of a Digital Game-Based Learning Program for Enhancing Youth Mental Health: A Structural Equation Modeling of the Program Effectiveness

PubMed Central

Huen, Jenny MY; Lai, Eliza SY; Shum, Angie KY; So, Sam WK; Chan, Melissa KY; Wong, Paul WC; Law, YW

2016-01-01

Background Digital game-based learning (DGBL) makes use of the entertaining power of digital games for educational purposes. Effectiveness assessment of DGBL programs has been underexplored and no attempt has been made to simultaneously model both important components of DGBL: learning attainment (ie, educational purposes of DGBL) and engagement of users (ie, entertaining power of DGBL) in evaluating program effectiveness. Objective This study aimed to describe and evaluate an Internet-based DGBL program, Professor Gooley and the Flame of Mind, which promotes mental health to adolescents in a positive youth development approach. In particular, we investigated whether user engagement in the DGBL program could enhance their attainment on each of the learning constructs per DGBL module and subsequently enhance their mental health as measured by psychological well-being. Methods Users were assessed on their attainment on each learning construct, psychological well-being, and engagement in each of the modules. One structural equation model was constructed for each DGBL module to model the effect of users' engagement and attainment on the learning construct on their psychological well-being. Results Of the 498 secondary school students that registered and participated from the first module of the DGBL program, 192 completed all 8 modules of the program. Results from structural equation modeling suggested that a higher extent of engagement in the program activities facilitated users’ attainment on the learning constructs on most of the modules and in turn enhanced their psychological well-being after controlling for users’ initial psychological well-being and initial attainment on the constructs. Conclusions This study provided evidence that Internet intervention for mental health, implemented with the technologies and digital innovations of DGBL, could enhance youth mental health. Structural equation modeling is a promising approach in evaluating the effectiveness of DGBL programs. PMID:27717921

Test and Verification Approach for the NASA Constellation Program

NASA Technical Reports Server (NTRS)

Strong, Edward

2008-01-01

This viewgraph presentation is a test and verification approach for the NASA Constellation Program. The contents include: 1) The Vision for Space Exploration: Foundations for Exploration; 2) Constellation Program Fleet of Vehicles; 3) Exploration Roadmap; 4) Constellation Vehicle Approximate Size Comparison; 5) Ares I Elements; 6) Orion Elements; 7) Ares V Elements; 8) Lunar Lander; 9) Map of Constellation content across NASA; 10) CxP T&V Implementation; 11) Challenges in CxP T&V Program; 12) T&V Strategic Emphasis and Key Tenets; 13) CxP T&V Mission & Vision; 14) Constellation Program Organization; 15) Test and Evaluation Organization; 16) CxP Requirements Flowdown; 17) CxP Model Based Systems Engineering Approach; 18) CxP Verification Planning Documents; 19) Environmental Testing; 20) Scope of CxP Verification; 21) CxP Verification - General Process Flow; 22) Avionics and Software Integrated Testing Approach; 23) A-3 Test Stand; 24) Space Power Facility; 25) MEIT and FEIT; 26) Flight Element Integrated Test (FEIT); 27) Multi-Element Integrated Testing (MEIT); 28) Flight Test Driving Principles; and 29) Constellation s Integrated Flight Test Strategy Low Earth Orbit Servicing Capability.

The Top 30 Rising Stars Program: an inter-organizational approach to leadership succession planning.

PubMed

Dilworth, Katie; Lankshear, Sara; Cava, Maureen; Aldred, Jacqueline; Hawkes, Nancy; Lefebre, Nancy; Price, Jennifer; Lawler, Valerie

2011-01-01

An effective leadership development program is an organizational investment that advances individual performance while strengthening organizational capabilities. The Top 30 Rising Stars Program is a leadership succession program designed to enable leadership capacity building within and across organizations. Key components of the program include formal learning, stretch opportunities, and mentorship. Evaluation results reveal high participant satisfaction and an increase in reported self-confidence in their ability to assume a formal leadership position.

Curriculum evaluation and revision in a nascent field: the utility of the retrospective pretest--posttest model in a homeland security program of study.

PubMed

Pelfrey, William V; Pelfrey, William V

2009-02-01

Although most academic disciplines evolve at a measured pace, the emerging field of homeland security must, for reasons of safety and security, evolve rapidly. The Department of Homeland Security sponsored the establishment of a graduate educational program for key officials holding homeland security roles. Because homeland security is a nascent field, the establishment of a program curriculum was forced to draw from a variety of disciplines. Curriculum evaluation was complicated by the rapid changes occurring in the emerging discipline, producing response shift bias, and interfering with the pre-post assessments. To compensate for the validity threat associated with response shift bias, a retrospective pretest-posttest evaluative methodology was used. Data indicate the program has evolved in a significant and orderly fashion and these data support the use of this innovative evaluation approach in the development of any discipline.

Evaluation of the Dornier Gmbh interactive grid generation system

NASA Technical Reports Server (NTRS)

Brown, Robert L.

1989-01-01

An interactive grid generation program, INGRID, is investigated and evaluated. A description of the task and work performed, a description and evaluation of INGRID, and a discussion of the possibilities for bringing INGRID into the NASA and Numerical Aerodynamic Simulator (NAS) computing environments is included. The interactive grid generation program was found to be a viable approach for grid generation and determined that it could be converted to work in the NAS environment but that INGRID does not solve the fundamentally hard problems associated with grid generation, specifically, domain decomposition.

Avoiding Problems in Evaluation.

ERIC Educational Resources Information Center

Tallmadge, G. Kasten

1979-01-01

Discusses the need for awareness of real and potential problems in educational evaluation approaches, kinds of problems there are with true experimental designs and with quasi-experimental designs, and the criteria applied by the Joint Dissemination Review Panel to its review of educational programs. (MF)

The surgical residency interview: a candidate-centered, working approach.

PubMed

Seabott, Heather; Smith, Ryan K; Alseidi, Adnan; Thirlby, Richard C

2012-01-01

The interview process is a pivotal, differentiating component of the residency match. Our bias is toward a working interview, producing better fulfillment of the needs of both parties, and a more informed match selection for the candidates and program. We describe a "candidate-centered" approach for integrating applicant interviews into our daily work schedule. Applicants are informed upon accepting the interview of the working interview model. Our program offers 33 interview days over a 12-week period. A maximum of 5 applicants are hosted per day. Applicants are assigned to 1 of our general, thoracic, vascular, or plastic surgery teams. The interview day begins with the applicant changing into scrubs, attending a morning conference, and taking part in a program overview by a Chief Resident. Applicants join their host team where 4-8 hours are spent observing the operative team, on rounds and sharing lunch. The faculty and senior residents are responsible for interviewing and evaluating applicants though the Electronic Residency Application Service. A total of 13 surgeons are involved in the interview process resulting in broad-based evaluations. Each surgeon interviewed between 3 and 12 applicants. Faculty rate this interview approach highly because it allows them to maintain a rigorous operative schedule while interacting with applicants. Current residents are engaged in welcoming applicants to view the program. Faculty and residents believe cooperating in a real world manner aids their assessment of the applicant. Applicants routinely provide positive feedback, relaying this approach is informative, transparent, and should be the "standard." Applicants believe they are presented a realistic view of the program. Ultimately, this candidate-centered process may be attributable to our resident cohort who exhibit high satisfaction, excellent resident morale, and very low dropout rate. We present a candidate-centered, working interview approach used in the selection of general surgery residents. While it may require more resources than the traditional approach, it harbors advantages for the applicant and the program. Copyright © 2012 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Guidelines for the Effective Implementation and Administration of Law-Related Education: Including an Addendum for Guidelines for LRE Instructor Training and Program Evaluation.

ERIC Educational Resources Information Center

Colorado State Dept. of Education, Denver.

This booklet defines law-related education as an educational program for teaching non-lawyers about law, the legal system, and the fundamental principles and values on which constitutional democracy is based. The program's approach is characterized by relevant curriculum materials, interactive teaching strategies, and extensive use of the…

The Capabilities Approach: Rethinking Agency, Freedom, and Capital in Early Education

ERIC Educational Resources Information Center

Buzzelli, Cary A.

2015-01-01

This article presents an overview of the human capital approach to human development and how it has been used for evaluating early education programs. It critiques the human capital approach by considering how its focus on measurable returns on economic investments limits an understanding of the full and complex contribution of early education to…

A School Finance Computer Simulation Model

ERIC Educational Resources Information Center

Boardman, Gerald R.

1974-01-01

Presents a description of the computer simulation model developed by the National Educational Finance Project for use by States in planning and evaluating alternative approaches for State support programs. Provides a general introduction to the model, a program operation overview, a sample run, and some conclusions. (Author/WM)

Implementing and Evaluating a Peer-Led Team Learning Approach in Undergraduate Anatomy and Physiology

ERIC Educational Resources Information Center

Finn, Kevin; Campisi, Jay

2015-01-01

This article describes how a Peer-Led Team Learning (PLTL) program was implemented in a first-year, undergraduate Anatomy and Physiology course sequence to examine the student perceptions of the program and determine the effects of PLTL on student performance.

Illuminating the Black Box of Entrepreneurship Education Programs

ERIC Educational Resources Information Center

Maritz, Alex; Brown, Christopher R.

2013-01-01

Purpose: The aim of this paper is to explore the components of entrepreneurship education programs (EEPs) and their interrelationships to develop a conceptual framework through which entrepreneurship education may be contextually evaluated and developed. Design/methodology/approach: The paper presents an extensive literature review of the…

EVALUATION OF THE IMPLEMENTATION OF ASBESTOS OPERATIONS AND MAINTENANCE PROGRAMS IN NEW JERSEY SCHOOLS

EPA Science Inventory

Removal of asbestos-containing materials (ACM) was encouraged over alternative approaches of asbestos management in the past. However, in-place management and operstions nd maintenance programs are currently viewed by the EPA as the most appropriate overall strategy for managemen...

Formulating New Directions with Strategic Marketing Planning.

ERIC Educational Resources Information Center

Crompton, John L.

1983-01-01

This article describes both a short- and long-term strategic marketing planning approach for the parks and recreation manager. Both plans involve a needs assessment, objective development, marketing plans, and evaluations. Also discussed is a continuum of stragetic program options ranging from developing new programs to terminating existing…

Incoming College Students' Bystander Behaviors to Prevent Sexual Violence

ERIC Educational Resources Information Center

McMahon, Sarah; Banyard, Victoria L.; McMahon, Sheila M.

2015-01-01

Evaluations of bystander intervention education programs demonstrate that this approach results in students' increased willingness to intervene in prosocial ways to prevent sexual violence (e.g., Moynihan, Banyard, Arnold, Eckstein, & Stapleton, 2010). These programs often focus on first-year college students, though theories and research on…

Practice guidelines for program evaluation in community-based rehabilitation.

PubMed

Grandisson, Marie; Hébert, Michèle; Thibeault, Rachel

2017-06-01

This paper proposes practice guidelines to evaluate community-based rehabilitation (CBR) programs. These were developed through a rigorous three-phase research process including a literature review on good practices in CBR program evaluation, a field study during which a South Africa CBR program was evaluated, and a Delphi study to generate consensus among a highly credible panel of CBR experts from a wide range of backgrounds and geographical areas. The 10 guidelines developed are summarized into a practice model highlighting key features of sound CBR program evaluation. They strongly indicate that sound CBR evaluations are those that give a voice and as much control as possible to the most affected groups, embrace the challenge of diversity, and foster use of evaluation processes and findings through a rigorous, collaborative and empowering approach. The practice guidelines should facilitate CBR evaluation decisions in respect to facilitating an evaluation process, using frameworks and designing methods. Implications for rehabilitation Ten practice guidelines provide guidance to facilitate sound community-based rehabilitation (CBR) program evaluation decisions. Key indications of good practice include: • being as participatory and empowering as possible; • ensuring that all, including the most affected, have a real opportunity to share their thoughts; • highly considering mixed methods and participatory tools; • adapting to fit evaluation context, local culture and language(s); • defining evaluation questions and reporting findings using shared CBR language when possible, which the framework offered may facilitate.

Using health promotion guidelines for persons with disabilities to develop and evaluate a physical activity program for individuals with multiple sclerosis: A feasibility study.

PubMed

Dixon-Ibarra, Alicia; Nery-Hurwit, Mara; Driver, Simon; MacDonald, Megan

2017-04-01

The Health Education for Persons with Multiple Sclerosis (HEMS) program was developed in response to the need for interventions aimed at increasing physical activity for individuals with Multiple Sclerosis (MS). It was developed and evaluated using Drum and colleagues (2009) guidelines for implementing health promotion programs for individuals with disabilities. The purpose of this feasibility study is to describe the development, implementation, and evaluation of the HEMS program. Thirteen individuals with MS completed the 8-week health education program. A mixed method approach for evaluation was implemented (i.e., survey and focus groups). Process and resource feasibility demonstrated that over half of the participants attended at least 80% of the weekly sessions. Focus group data provided valuable feedback for future iterations of the program including critiques on the delivery, content, and group support provided. Outcome evaluation showed increases in self-efficacy (survey), improvements in theoretical constructs (focus groups), and increased physical activity (focus groups). Results show that health promotion programs for persons with MS can improve physical activity and related constructs. Next steps will be to revise, implement, and reevaluate the HEMS program in a larger randomized control trial. Published by Elsevier Ltd.

Early adolescent peer leader development in HIV prevention using youth-adult partnership with schools approach.

PubMed

Fongkaew, Warunee; Fongkaew, Kangwan; Suchaxaya, Prakin

2007-01-01

This article describes the development and evaluation of an HIV prevention program developed for early adolescents through participatory action research. The HIV prevention program included a curriculum that was delivered by trained younger youth leaders through a youth and adult partnership with 10 schools in Chiang Mai Province, Thailand. The curriculum used participatory learning experiences, "edutainment" approaches, and skills-building strategies for enhancing youth leaders' capacities. Results of the evaluation showed that the senior-junior peer education program was effective in leadership role preparation, in improving youth leaders' ability to share sexual and reproductive health knowledge, and in promoting positive attitudes toward themselves. Success also rested on the fact that adults took a critical role in providing the opportunities, assistance, and guidance so that young people could develop their leadership capacity in an atmosphere of trust and respect.

DERMAL EXPOSURE ASSESSMENT: A SUMMARY OF EPA APPROACHES

EPA Science Inventory

This final report presents a concise description and evaluation of the approaches used in the Agency for dermal exposure assessment including a discussion about harmonization and research needs in this area. The report is intended to be used by EPA program offices in their effort...

Frequency assignments for HFDF receivers in a search and rescue network

NASA Astrophysics Data System (ADS)

Johnson, Krista E.

1990-03-01

This thesis applies a multiobjective linear programming approach to the problem of assigning frequencies to high frequency direction finding (HFDF) receivers in a search-and-rescue network in order to maximize the expected number of geolocations of vessels in distress. The problem is formulated as a multiobjective integer linear programming problem. The integrality of the solutions is guaranteed by the totally unimodularity of the A-matrix. Two approaches are taken to solve the multiobjective linear programming problem: (1) the multiobjective simplex method as implemented in ADBASE; and (2) an iterative approach. In this approach, the individual objective functions are weighted and combined in a single additive objective function. The resulting single objective problem is expressed as a network programming problem and solved using SAS NETFLOW. The process is then repeated with different weightings for the objective functions. The solutions obtained from the multiobjective linear programs are evaluated using a FORTRAN program to determine which solution provides the greatest expected number of geolocations. This solution is then compared to the sample mean and standard deviation for the expected number of geolocations resulting from 10,000 random frequency assignments for the network.

Characteristics of effective summer learning programs in practice.

PubMed

Bell, Susanne R; Carrillo, Natalie

2007-01-01

The Center for Summer Learning examined various summer program models and found that there are nine characteristics that provide a framework for effective summer programs. In this chapter, the authors demonstrate how effective practices lead to positive results for young people. The nine characteristics of effective summer learning programs are (1) accelerating learning, (2) youth development, (3) proactive approach to summer learning, (4) leadership, (5) advanced planning, (6) staff development, (7) strategic partnerships, (8) evaluation and commitment to program improvement, and (9) sustainability and cost-effectiveness. These characteristics are divided into two sections. The first three characteristics address a program's approach to learning. Summer instructional techniques are most effective when academic learning is woven into enrichment activities and youth development. The second section covers program infrastructure to ensure the organization achieves and maintains quality programming. The nine characteristics complement each other to ensure a strong program that works to prevent summer learning loss and narrow the achievement gap. To demonstrate the variety of high-quality programs that include the nine characteristics, thirteen program profiles at the conclusion of the chapter each highlight one of the characteristics. These profiles show the various approaches that different summer programs have developed to accelerate academic achievement and promote positive development for young people in their communities.

A novel framework for analyzing conservation impacts: evaluation, theory, and marine protected areas.

PubMed

Mascia, Michael B; Fox, Helen E; Glew, Louise; Ahmadia, Gabby N; Agrawal, Arun; Barnes, Megan; Basurto, Xavier; Craigie, Ian; Darling, Emily; Geldmann, Jonas; Gill, David; Holst Rice, Susie; Jensen, Olaf P; Lester, Sarah E; McConney, Patrick; Mumby, Peter J; Nenadovic, Mateja; Parks, John E; Pomeroy, Robert S; White, Alan T

2017-07-01

Environmental conservation initiatives, including marine protected areas (MPAs), have proliferated in recent decades. Designed to conserve marine biodiversity, many MPAs also seek to foster sustainable development. As is the case for many other environmental policies and programs, the impacts of MPAs are poorly understood. Social-ecological systems, impact evaluation, and common-pool resource governance are three complementary scientific frameworks for documenting and explaining the ecological and social impacts of conservation interventions. We review key components of these three frameworks and their implications for the study of conservation policy, program, and project outcomes. Using MPAs as an illustrative example, we then draw upon these three frameworks to describe an integrated approach for rigorous empirical documentation and causal explanation of conservation impacts. This integrated three-framework approach for impact evaluation of governance in social-ecological systems (3FIGS) accounts for alternative explanations, builds upon and advances social theory, and provides novel policy insights in ways that no single approach affords. Despite the inherent complexity of social-ecological systems and the difficulty of causal inference, the 3FIGS approach can dramatically advance our understanding of, and the evidentiary basis for, effective MPAs and other conservation initiatives. © 2017 New York Academy of Sciences.

Improving Hospital-Wide Early Resource Allocation through Machine Learning.

PubMed

Gartner, Daniel; Padman, Rema

2015-01-01

The objective of this paper is to evaluate the extent to which early determination of diagnosis-related groups (DRGs) can be used for better allocation of scarce hospital resources. When elective patients seek admission, the true DRG, currently determined only at discharge, is unknown. We approach the problem of early DRG determination in three stages: (1) test how much a Naïve Bayes classifier can improve classification accuracy as compared to a hospital's current approach; (2) develop a statistical program that makes admission and scheduling decisions based on the patients' clincial pathways and scarce hospital resources; and (3) feed the DRG as classified by the Naïve Bayes classifier and the hospitals' baseline approach into the model (which we evaluate in simulation). Our results reveal that the DRG grouper performs poorly in classifying the DRG correctly before admission while the Naïve Bayes approach substantially improves the classification task. The results from the connection of the classification method with the mathematical program also reveal that resource allocation decisions can be more effective and efficient with the hybrid approach.

SEASAT economic assessment. Volume 10: The SATIL 2 program (a program for the evaluation of the costs of an operational SEASAT system as a function of operational requirements and reliability. [computer programs for economic analysis and systems analysis of SEASAT satellite systems

NASA Technical Reports Server (NTRS)

1975-01-01

The SATIL 2 computer program was developed to assist with the programmatic evaluation of alternative approaches to establishing and maintaining a specified mix of operational sensors on spacecraft in an operational SEASAT system. The program computes the probability distributions of events (i.e., number of launch attempts, number of spacecraft purchased, etc.), annual recurring cost, and present value of recurring cost. This is accomplished for the specific task of placing a desired mix of sensors in orbit in an optimal fashion in order to satisfy a specified sensor demand function. Flow charts are shown, and printouts of the programs are given.

A school-based interdisciplinary approach to promote health and academic achievement among children in a deprived neighborhood: study protocol for a mixed-methods evaluation.

PubMed

Abrahamse, Mariëlle E; Jonkman, Caroline S; Harting, Janneke

2018-04-10

The large number of children that grow up in poverty is concerning, especially given the negative developmental outcomes that can persist into adulthood. Poverty has been found as a risk factor to negatively affect academic achievement and health outcomes in children. Interdisciplinary interventions can be an effective way to promote health and academic achievement. The present study aims to evaluate a school-based interdisciplinary approach on child health, poverty, and academic achievement using a mixed-method design. Generally taken, outcomes of this study increase the knowledge about effective ways to give disadvantaged children equal chances early in their lives. An observational study with a mixed-methods design including both quantitative and qualitative data collection methods will be used to evaluate the interdisciplinary approach. The overall research project exists of three study parts including a longitudinal study, a cross-sectional study, and a process evaluation. Using a multi-source approach we will assess child health as the primary outcome. Child poverty and child academic achievement will be assessed as secondary outcomes. The process evaluation will observe the program's effects on the school environment and the program's implementation in order to obtain more knowledge on how to disseminate the interdisciplinary approach to other schools and neighborhoods. The implementation of a school-based interdisciplinary approach via primary schools combining the cross-sectoral domains health, poverty, and academic achievement is innovative and a step forward to reach an ethnic minority population. However, the large variety of the interventions and activities within the approach can limit the validity of the study. Including a process evaluation will therefore help to improve the interpretation of our findings. In order to contribute to policy and practice focusing on decreasing the unequal chances of children growing up in deprived neighborhoods, it is important to study whether the intervention leads to positive developmental outcomes in children. ( NTR 6571 ) (retrospectively registered on August 4, 2017).

Evaluating the Implementation of Performance Improvement Training: The E[superscript 3] Process for Success

ERIC Educational Resources Information Center

Larbi-Apau, Josephine A.; Moseley, James L.

2008-01-01

This article provides a comprehensive approach to careful review and evaluation of the implementation of performance training intervention. It discusses the E[superscript 3] process for success, a basic framework for evaluating the implementation phase of a training program implemented as a broad-based performance improvement strategy. The intent…

Analyzing Empirical Evaluations of Non-Experimental Methods in Field Settings

ERIC Educational Resources Information Center

Steiner, Peter M.; Wong, Vivian

2016-01-01

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

Alternative Methods in the Evaluation of School District Cash Management Programs.

ERIC Educational Resources Information Center

Dembowski, Frederick L.

1980-01-01

Empirically evaluates three measures of effectiveness of school district cash management: the rate of return method in common use and two new measures--efficiency rating and Net Present Value (NPV). The NPV approach allows examination of efficiency and provides a framework for evaluating other areas of educational policy. (Author/IRT)

The Evaluation Exchange. Volume XV Number 1. Spring 2010

ERIC Educational Resources Information Center

Coffman, Julia, Ed.; Harris, Erin, Ed.

2010-01-01

This issue of The Evaluation Exchange explores the promising practices and challenges associated with taking an enterprise to scale, along with the role that evaluation can and should play in that process. Surprisingly few examples exist of nonprofit efforts that have scaled up and achieved lasting success. A program or approach may be strong…

Employee Perceptions of Workplace Health Promotion Programs: Comparison of a Tailored, Semi-Tailored, and Standardized Approach.

PubMed

Street, Tamara D; Lacey, Sarah J

2018-04-28

In the design of workplace health promotion programs (WHPPs), employee perceptions represent an integral variable which is predicted to translate into rate of user engagement (i.e., participation) and program loyalty. This study evaluated employee perceptions of three workplace health programs promoting nutritional consumption and physical activity. Programs included: (1) an individually tailored consultation with an exercise physiologist and dietitian; (2) a semi-tailored 12-week SMS health message program; and (3) a standardized group workshop delivered by an expert. Participating employees from a transport company completed program evaluation surveys rating the overall program, affect, and utility of: consultations ( n = 19); SMS program ( n = 234); and workshops ( n = 86). Overall, participants’ affect and utility evaluations were positive for all programs, with the greatest satisfaction being reported in the tailored individual consultation and standardized group workshop conditions. Furthermore, mode of delivery and the physical presence of an expert health practitioner was more influential than the degree to which the information was tailored to the individual. Thus, the synergy in ratings between individually tailored consultations and standardized group workshops indicates that low-cost delivery health programs may be as appealing to employees as tailored, and comparatively high-cost, program options.

Lessons Learned From Community-Based Approaches to Sodium Reduction

PubMed Central

Kane, Heather; Strazza, Karen; Losby PhD, Jan L.; Lane, Rashon; Mugavero, Kristy; Anater, Andrea S.; Frost, Corey; Margolis, Marjorie; Hersey, James

2017-01-01

Purpose This article describes lessons from a Centers for Disease Control and Prevention initiative encompassing sodium reduction interventions in six communities. Design A multiple case study design was used. Setting This evaluation examined data from programs implemented in six communities located in New York (Broome County, Schenectady County, and New York City); California (Los Angeles County and Shasta County); and Kansas (Shawnee County). Subjects Participants (n = 80) included program staff, program directors, state-level staff, and partners. Measures Measures for this evaluation included challenges, facilitators, and lessons learned from implementing sodium reduction strategies. Analysis The project team conducted a document review of program materials and semi structured interviews 12 to 14 months after implementation. The team coded and analyzed data deductively and inductively. Results Five lessons for implementing community-based sodium reduction approaches emerged: (1) build relationships with partners to understand their concerns, (2) involve individuals knowledgeable about specific venues early, (3) incorporate sodium reduction efforts and messaging into broader nutrition efforts, (4) design the program to reduce sodium gradually to take into account consumer preferences and taste transitions, and (5) identify ways to address the cost of lower-sodium products. Conclusion The experiences of the six communities may assist practitioners in planning community-based sodium reduction interventions. Addressing sodium reduction using a community-based approach can foster meaningful change in dietary sodium consumption. PMID:24575726

Crossing 138: two approaches to churn under the Affordable Care Act.

PubMed

Ravel, Gabriel; DeSantis, J Angelo

2014-01-01

A predicted side effect of the Medicaid expansion and state-based Exchanges under the Affordable Care Act is churn. Churn is the shifting into and out of eligibility for insurance affordability programs due to income changes. Because the line between Medicaid and Exchange eligibility is fine -138% of the federal poverty level -millions of Americans are expected to gain and lose eligibility. Frequently, this churning undermines continuity of care, raises costs, and frustrates those affected. This article explores two proposed programs to mitigate the effects of churn: the Basic Health Program and the Bridge Program. This article evaluates both programs' ability to mitigate the effects of churn, the likely side effects to states' implementing them, and legal and practical obstacles to their implementation. It concludes that the Bridge Program is the better approach.

Development, implementation and evaluation of a clinical research engagement and leadership capacity building program in a large Australian health care service.

PubMed

Misso, Marie L; Ilic, Dragan; Haines, Terry P; Hutchinson, Alison M; East, Christine E; Teede, Helena J

2016-01-14

Health professionals need to be integrated more effectively in clinical research to ensure that research addresses clinical needs and provides practical solutions at the coal face of care. In light of limited evidence on how best to achieve this, evaluation of strategies to introduce, adapt and sustain evidence-based practices across different populations and settings is required. This project aims to address this gap through the co-design, development, implementation, evaluation, refinement and ultimately scale-up of a clinical research engagement and leadership capacity building program in a clinical setting with little to no co-ordinated approach to clinical research engagement and education. The protocol is based on principles of research capacity building and on a six-step framework, which have previously led to successful implementation and long-term sustainability. A mixed methods study design will be used. Methods will include: (1) a review of the literature about strategies that engage health professionals in research through capacity building and/or education in research methods; (2) a review of existing local research education and support elements; (3) a needs assessment in the local clinical setting, including an online cross-sectional survey and semi-structured interviews; (4) co-design and development of an educational and support program; (5) implementation of the program in the clinical environment; and (6) pre- and post-implementation evaluation and ultimately program scale-up. The evaluation focuses on research activity and knowledge, attitudes and preferences about clinical research, evidence-based practice and leadership and post implementation, about their satisfaction with the program. The investigators will evaluate the feasibility and effect of the program according to capacity building measures and will revise where appropriate prior to scale-up. It is anticipated that this clinical research engagement and leadership capacity building program will enable and enhance clinically relevant research to be led and conducted by health professionals in the health setting. This approach will also encourage identification of areas of clinical uncertainty and need that can be addressed through clinical research within the health setting.

Commentary on Foubert, Godin, & Tatum (2010): the evolution of sexual violence prevention and the urgency for effectiveness.

PubMed

Tharp, Andra Teten; DeGue, Sarah; Lang, Karen; Valle, Linda Anne; Massetti, Greta; Holt, Melissa; Matjasko, Jennifer

2011-11-01

Foubert, Godin, and Tatum describe qualitative effects among college men of The Men's Program, a one-session sexual violence prevention program. This article and the program it describes are representative of many sexual violence prevention programs that are in practice and provide an opportunity for a brief discussion of the development and evaluation of sexual violence prevention approaches. In this commentary, we will focus on two considerations for an evolving field: the adherence to the principles of prevention and the use of rigorous evaluation methods to demonstrate effectiveness. We argue that the problem of sexual violence has created urgency for effective prevention programs and that scientific and prevention standards provide the best foundation to meet this need.

Road usage charge pilot project final evaluation report for Washington State participants.

DOT National Transportation Integrated Search

2013-05-01

This report provides a summary of evaluation results of Washingtons participation in : the Road Usage Charge Pilot Program (RUCPP). The RUCPP was a trial of various : approaches and technologies for motorists in the States of Washington, Oregon, a...

Towards program theory validation: Crowdsourcing the qualitative analysis of participant experiences.

PubMed

Harman, Elena; Azzam, Tarek

2018-02-01

This exploratory study examines a novel tool for validating program theory through crowdsourced qualitative analysis. It combines a quantitative pattern matching framework traditionally used in theory-driven evaluation with crowdsourcing to analyze qualitative interview data. A sample of crowdsourced participants are asked to read an interview transcript and identify whether program theory components (Activities and Outcomes) are discussed and to highlight the most relevant passage about that component. The findings indicate that using crowdsourcing to analyze qualitative data can differentiate between program theory components that are supported by a participant's experience and those that are not. This approach expands the range of tools available to validate program theory using qualitative data, thus strengthening the theory-driven approach. Copyright © 2017 Elsevier Ltd. All rights reserved.

Finding Your Way™: a collaborative approach to increase awareness of missing person events among persons with dementia.

PubMed

Hillier, Loretta M; Harvey, David; Conway, Cathy; Hunt, Jocelyn; Hoffman, Ron

2016-04-01

To describe an innovative community-wide program aimed at increasing awareness of risks of missing person events among persons with dementia targeting various cultural groups and to present preliminary evaluation findings. Review of program records to describe program implementation and a community partner survey. Over 23 months, 386 partnerships were established to implement the program; 941 awareness-raising sessions were conducted reaching 23,495 individuals. There is an upward trend in number of sessions conducted in various languages and attendance. Community partners' (>85%) responded positively to the program and consultation process. This partnership experience demonstrates the effectiveness of a collaborative approach to the development and widespread dissemination of information and resource materials aimed at ensuring the safety of a vulnerable population.

Evaluation of Student Outcomes after Participating in a Medicare Outreach Program

ERIC Educational Resources Information Center

Hollingsworth, Joshua C.; Teeter, Benjamin S.; Westrick, Salisa C.

2015-01-01

This article describes the development of a service-learning project and analysis of student pharmacists' participation therein. Using a mixed-methods approach, this study analyzed student pharmacists' knowledge and attitudes after volunteering in the inaugural Medicare Outreach Program, a collaboration between the School of Pharmacy and State…

15 CFR 290.8 - Reviews of centers.

Code of Federal Regulations, 2011 CFR

2011-01-01

... the purpose of evaluation and program shaping. Centers are encouraged to try new approaches, must... at least annually, and at the end of its third year of operation according to the procedures and... ability to obtain suitable match, as well as on the budgetary requirements of its proposed program...

15 CFR 290.8 - Reviews of centers.

Code of Federal Regulations, 2010 CFR

2010-01-01

... the purpose of evaluation and program shaping. Centers are encouraged to try new approaches, must... at least annually, and at the end of its third year of operation according to the procedures and... ability to obtain suitable match, as well as on the budgetary requirements of its proposed program...

Producing Better Writers in Sociology: A Programmatic Approach

ERIC Educational Resources Information Center

Migliaccio, Todd; Carrigan, Jacqueline

2017-01-01

High-quality undergraduate student writing is a common and important objective for sociology programs while at the same time a continuous challenge. Programs often struggle to address writing adequately because of the difficulty of fully evaluating student writing and responding to any identified limitations, largely because of the impact on…

Evaluating Urban Resilience to Climate Change: A Multi-Sector Approach (Final Report)

EPA Science Inventory

EPA is announcing the availability of this final report prepared by the Air, Climate, and Energy (ACE) Research Program, located within the Office of Research and Development, with support from Cadmus. One of the goals of the ACE research program is to provide scientific informat...

Enhancing the Quality of Tutorials through Peer-Connected Tutor Training

ERIC Educational Resources Information Center

Calma, Angelito; Eggins, Mark

2012-01-01

This paper investigates how a peer-connected tutor training program can lead to quality enhancement by helping tutors to develop more effective teaching strategies and promoting better learning approaches among business students. It uses 2007-2010 evaluation data from 343 program participants from accounting, economics, finance and management and…

Intervention and Evaluation: A Proactive Team Approach to OD

ERIC Educational Resources Information Center

Gavin, James F.; McPhail, S. Morton

1978-01-01

A team of change agents initiated an intensive organizational development (OD) program in a nonacademic, service department of a midwestern university. Concluding comments include exploration of the feasibility of substantive OD change in short-term programs, and the possibility of nonlinear relations between OD outcomes and time investments. For…

A Whole Language Flight Plan: An Interview with Three Teachers.

ERIC Educational Resources Information Center

Griffith, Priscilla L.; Klesius, Janell

1990-01-01

Provides suggestions to teachers planning to implement a whole language program, based on interviews with three whole language teachers. Focuses on support for the whole language program; decisions about curriculum and evaluation; development of vocabulary and comprehension; strengths and weaknesses of the whole language approach; and preparation…

Learn by Doing - Phase I of the ToxCast Research Program

EPA Science Inventory

In 2007, the USEPA embarked on a multi-year, multi-million dollar research program to develop and evaluate a new approach to prioritizing the toxicity testing of environmental chemicals. ToxCast was divided into three main phases of effort – a proof of concept, an expansion and ...

The development of a parenting program for incarcerated mothers in Australia: a review of prison-based parenting programs.

PubMed

Newman, Claire; Fowler, Cathrine; Cashin, Andrew

2011-08-01

The increasing population of children with an incarcerated parent is a significant public health issue. A literature search highlighted that children of incarcerated parents experience psychological stressors that may potentially impact on health and behavioural outcomes. Parenting programs for prisoners may be of benefit as early parenting experiences during childhood have a significant impact on a child's future experiences as an adolescent and adult. A review of identified evaluation-based studies of parenting programs for prisoners (N = 11), although varied in program delivery approaches and evaluation methods, suggest that such programs have the potential to improve the parenting skills, knowledge and confidence of incarcerated parents. Finally, this paper provides an outline of the development of an Australian based parenting program for incarcerated mothers and their young children.

Programs and promotions: approaches by 25 Active Living by Design partnerships.

PubMed

Claus, Julie M; Dessauer, Mark; Brennan, Laura K

2012-11-01

From 2003 to 2008, a total of 25 community partnerships funded through Active Living by Design (ALbD) implemented physical activity programs and promotions as part of integrated approaches complementing policy and environment changes. This paper reviews the partnerships' efforts with respect to promotions and programs, the breadth and depth of these types of approaches, challenges, successes, and lessons learned. Through a mixed-methods approach, including interviews, focus groups, and web-based tracking, multiple sources of data were collected and analyzed from 2006 to 2010. Evaluators summarized quantitative data by counts or means and qualitative results using systematic coding procedures to identify themes, ideas, and concepts. All 25 community partnerships were engaged in programs and promotions of varying degrees throughout the initiative. Programs were categorized as community walking and biking programs, school programs, afterschool programs, and worksite programs, among others. Promotional strategies were categorized as social marketing campaigns, media, events, and communications. The most common programs included Safe Routes to School, walking clubs, and Bike/Walk to School Day. Media efforts were undertaken by all 25 partnerships, totaling 2659 TV, newspaper, and radio hits. Programs and promotions can be resource-intensive and have limited population impact when offered in isolation; however, these strategies help connect people to their environments (e.g., increase awareness of facilities, provide social support for use of facilities) in order to improve physical activity behaviors. Copyright © 2012 American Journal of Preventive Medicine. Published by Elsevier Inc. All rights reserved.

Applying the RE-AIM Framework to Evaluate Integrative Medicine Group Visits Among Diverse Women with Chronic Pelvic Pain.

PubMed

Chao, Maria T; Abercrombie, Priscilla D; Santana, Trilce; Duncan, Larissa G

2015-12-01

The purpose of this study was to evaluate group medical visits using an integrative health approach for underserved women with chronic pelvic pain (CPP). We implemented an integrative medicine program to improve quality of life among women with CPP using Centering, a group-based model that combines healthcare assessment, education, and social support. Patients were from university-affiliated and public hospital-affiliated clinics. We evaluated the program with qualitative and quantitative data to address components of the RE-AIM framework: Reach, Effectiveness, Adoption, Implementation, and Maintenance. Participants of the Centering CPP Program participants (n = 26) were demographically similar to a sample of women with CPP who sought care at Bay Area hospitals (n = 701). Participants were on average 40 years of age, a majority of whom were racial/ethnic minorities with low household income (76%). Women who attended four or more sessions (n = 16) had improved health-related quality of life, including decreases in average number of unhealthy days in the past month (from 24 to 18, p < .05), depressive symptoms (from 11.7 to 9.0, p < .05), and symptom severity (from 4.2 to 3.1, p < .01). Sexual health outcomes also improved (30.5 to 50.3, p = .02). No improvements were observed for pain catastrophizing. Our pilot program provides preliminary data that an integrative health approach using a group-based model can be adapted and implemented to reach diverse women with CPP to improve physical and psychological well-being. Given these promising findings, rigorous evaluation of implementation and effectiveness of this approach compared with usual care is warranted. Copyright © 2015 American Society for Pain Management Nursing. Published by Elsevier Inc. All rights reserved.

Improved numerical methods for turbulent viscous flows aerothermal modeling program, phase 2

NASA Technical Reports Server (NTRS)

Karki, K. C.; Patankar, S. V.; Runchal, A. K.; Mongia, H. C.

1988-01-01

The details of a study to develop accurate and efficient numerical schemes to predict complex flows are described. In this program, several discretization schemes were evaluated using simple test cases. This assessment led to the selection of three schemes for an in-depth evaluation based on two-dimensional flows. The scheme with the superior overall performance was incorporated in a computer program for three-dimensional flows. To improve the computational efficiency, the selected discretization scheme was combined with a direct solution approach in which the fluid flow equations are solved simultaneously rather than sequentially.

A review of programs that targeted environmental determinants of Aboriginal and Torres Strait Islander health.

PubMed

Johnston, Leah; Doyle, Joyce; Morgan, Bec; Atkinson-Briggs, Sharon; Firebrace, Bradley; Marika, Mayatili; Reilly, Rachel; Cargo, Margaret; Riley, Therese; Rowley, Kevin

2013-08-09

Effective interventions to improve population and individual health require environmental change as well as strategies that target individual behaviours and clinical factors. This is the basis of implementing an ecological approach to health programs and health promotion. For Aboriginal People and Torres Strait Islanders, colonisation has made the physical and social environment particularly detrimental for health. We conducted a literature review to identify Aboriginal health interventions that targeted environmental determinants of health, identifying 21 different health programs. Program activities that targeted environmental determinants of health included: Caring for Country; changes to food supply and/or policy; infrastructure for physical activity; housing construction and maintenance; anti-smoking policies; increased workforce capacity; continuous quality improvement of clinical systems; petrol substitution; and income management. Targets were categorised according to Miller's Living Systems Theory. Researchers using an Indigenous community based perspective more often identified interpersonal and community-level targets than were identified using a Western academic perspective. Although there are relatively few papers describing interventions that target environmental determinants of health, many of these addressed such determinants at multiple levels, consistent to some degree with an ecological approach. Interpretation of program targets sometimes differed between academic and community-based perspectives, and was limited by the type of data reported in the journal articles, highlighting the need for local Indigenous knowledge for accurate program evaluation. While an ecological approach to Indigenous health is increasingly evident in the health research literature, the design and evaluation of such programs requires a wide breadth of expertise, including local Indigenous knowledge.

A Review of Programs That Targeted Environmental Determinants of Aboriginal and Torres Strait Islander Health

PubMed Central

Johnston, Leah; Doyle, Joyce; Morgan, Bec; Atkinson-Briggs, Sharon; Firebrace, Bradley; Marika, Mayatili; Reilly, Rachel; Cargo, Margaret; Riley, Therese; Rowley, Kevin

2013-01-01

Objective: Effective interventions to improve population and individual health require environmental change as well as strategies that target individual behaviours and clinical factors. This is the basis of implementing an ecological approach to health programs and health promotion. For Aboriginal People and Torres Strait Islanders, colonisation has made the physical and social environment particularly detrimental for health. Methods and Results: We conducted a literature review to identify Aboriginal health interventions that targeted environmental determinants of health, identifying 21 different health programs. Program activities that targeted environmental determinants of health included: Caring for Country; changes to food supply and/or policy; infrastructure for physical activity; housing construction and maintenance; anti-smoking policies; increased workforce capacity; continuous quality improvement of clinical systems; petrol substitution; and income management. Targets were categorised according to Miller’s Living Systems Theory. Researchers using an Indigenous community based perspective more often identified interpersonal and community-level targets than were identified using a Western academic perspective. Conclusions: Although there are relatively few papers describing interventions that target environmental determinants of health, many of these addressed such determinants at multiple levels, consistent to some degree with an ecological approach. Interpretation of program targets sometimes differed between academic and community-based perspectives, and was limited by the type of data reported in the journal articles, highlighting the need for local Indigenous knowledge for accurate program evaluation. Implications: While an ecological approach to Indigenous health is increasingly evident in the health research literature, the design and evaluation of such programs requires a wide breadth of expertise, including local Indigenous knowledge. PMID:23939388

Marginal ambulatory teaching cost under varying levels of service utilization.

PubMed

Panton, D M; Mushlin, A I; Gavett, J W

1980-06-01

The ambulatory component of residency training jointly produces two products, namely, training and patient services. In costing educational programs of this type, two approaches are frequently taken. The first considers the total costs of the educational program, including training and patient services. These costs are usually constructed from historical accounting records. The second approach attempts to cost the joint products separately, based upon estimates of future changes in program costs, if the product in question is added to or removed from the program. The second approach relates to typical decisions facing the managers of medical centers and practices used for teaching purposes. This article reports such a study of costs in a primary-care residency training program in a hospital outpatient setting. The costs of the product, i.e., on-the-job training, are evaluated using a replacement-cost concept under different levels of patient services. The results show that the cost of the product, training, is small at full clinical utilization and is sensitive to changes in the volume of services provided.

Development and Evaluation of a Web-Based Training Program for Oral Health Care Providers on Secondary Prevention of Eating Disorders

PubMed Central

DeBate, Rita D.; Severson, Herbert; Zwald, Marissa L.; Shaw, Tracy; Christiansen, Steve; Koerber, Anne; Tomar, Scott; Brown, Kelli McCormack; Tedesco, Lisa A.

2009-01-01

Although oral health care providers (OHP) are key in the secondary prevention of eating disorders (ED), the majority are not engaged in assessment, referral, and case management. This innovative pilot project developed and evaluated a web-based training program for dental and dental hygiene students and providers on the secondary prevention of ED. The intervention combined didactic and skill-based objectives to train OHP on ED and its oral health effects, OHP roles, skills in identifying the oral signs of ED, communication, treatment, and referral. Using a convenience sample of OHP (n=66), a pre-/post-test evaluated short-term outcomes and user satisfaction. Results revealed statistically significant improvements in self-efficacy (p<.001); knowledge of oral manifestations from restrictive behaviors (p<.001) and purging behaviors (p<.001); knowledge of oral treatment options (p<.001); and attitudes towards the secondary prevention of ED (p<.001). Most participants strongly agreed or agreed that the program provided more information (89 percent) and resources (89 percent) about the secondary prevention of ED than were currently available; 91 percent strongly agreed or agreed that they would access this program for information regarding the secondary prevention of ED. This pilot project provides unique training in the clinical evaluation, patient approach, referral, and oral treatment that takes a multidisciplinary approach to address ED. PMID:19491349

Web-based oral health promotion program for older adults: Development and preliminary evaluation.

PubMed

Mariño, Rodrigo J; Marwaha, Parul; Barrow, Su-Yan

2016-07-01

This study reports on the impact evaluation of a Web-based oral health promotion programme aimed at improving the oral health knowledge, attitudes, practices and self-efficacy of independent-living older adults from Melbourne, Australia. With ethics approval from the University of Melbourne, a convenience sample of volunteers 55 years or older was invited to participate in a study to test a web-based oral health promotion program. Consenting volunteers were asked to undergo a structured interview as part of the pre-intervention data collection. The intervention was based on the ORHIS (Oral Health Information Seminars/Sheets) Model and involved computer interaction with six oral health presentations, with no direct oral health professional input. A one group pre-test-post-test quasi-experimental design was chosen to evaluate the intervention. A series of paired t-tests were used to compare pre-test with post-test results. Forty-seven active, independent-living older adults participated in this evaluation. After the intervention participants responded with higher levels of achievement than before participating in this Web-based oral health program. Participants showed significant improvements in oral health attitudes (4.10 vs. 4.94; p<0.01), knowledge (18.37 vs. 23.83; p<0.0001), and self-efficacy (84.37 vs.89.23; p<0.01), as well as, self-reported oral hygiene practices (i.e., frequency of use of dental floss) (p<0.05). The e-ORHIS approach was successful in improving oral health knowledge, attitudes and self-efficacy. As such, it represents a helpful approach for the design of (oral) health interventions in older adults. Further evaluation with a larger sample is required to test the long-term impact including the economic evaluation of the e-ORHIS approach. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Review of the Agard S&M panel evaluation program of the NASA-Lewis 'SRP' approach to high-temperature LCF life prediction. [Strainrange Partitioning for Low Cycle Fatigue

NASA Technical Reports Server (NTRS)

Hirschberg, M. H.

1978-01-01

Twenty laboratories in six countries participated in this program, each testing its own materials of interest under its own laboratory conditions. In this way the results obtained provided validation of the Strainrange Partitioning (SRP) method for a wide range of materials and insured maximum usefulness to each of the participating laboratories. The first, very necessary step in the evaluation of any life prediction approach - assessing the ability of the method to predict life of simple laboratory specimens subjected to complex loading, was thereby taken. The culmination of this program was the Specialists Meeting that was held in Aalborg, Denmark in April of 1978. At that meeting the various investigators shared their findings, thus providing the basis for an in-depth evaluation of the SRP method. While the results were variable from laboratory to laboratory, most investigators agreed that the SRP method was a significant step toward life prediction in the presence of high temperature and cyclic stresses.

Evaluation philosophy for shuttle launched payloads

NASA Technical Reports Server (NTRS)

Heuser, R. E.

1975-01-01

Some approaches to space-shuttle payload evaluation are examined. Issues considered include subsystem replacement in low-cost modular spacecraft (LCMS), validation of spacelab payloads, the use of standard components in shuttle-era spacecraft, effects of shuttle-induced environments on payloads, and crew safety. The LCMS is described, and goals are discussed for its evaluation program. Concepts regarding how the evaluation should proceed are considered.

Barbershop Talk With Brothers: using community-based participatory research to develop and pilot test a program to reduce HIV risk among Black heterosexual men.

PubMed

Wilson, Tracey E; Fraser-White, Marilyn; Williams, Kim M; Pinto, Angelo; Agbetor, Francis; Camilien, Brignel; Henny, Kirk; Browne, Ruth C; Gousse, Yolene; Taylor, Tonya; Brown, Humberto; Taylor, Raekiela; Joseph, Michael A

2014-10-01

There is a need for feasible, evidence-based interventions that support HIV risk reduction among heterosexual Black men. In this article, we describe the process for development of the Barbershop Talk With Brothers (BTWB) program and evaluation. The BTWB program is a theoretically grounded and community-based HIV prevention program that seeks to improve individual skills and motivation to decrease sexual risk, and that builds men's interest in and capacity for improving their community's health. Formative data collection included barbershop observations and barber focus groups, brief behavioral risk assessments of men in barbershops, and focus groups and individual interviews. Based on this information and in consultation with our steering committee, we developed the BTWB program and accompanying program evaluation. From April through November 2011, 80 men were recruited and completed a baseline assessment of a pilot test of the program; 78 men completed the program and 71 completed a 3-month assessment. The pilot evaluation procedures were feasible to implement, and assessments of pre- and post-test measures indicate that key behavioral outcomes and proposed mediators of those outcomes changed in hypothesized directions. Specifically, attitudes and self-efficacy toward consistent condom use improved, and respondents reported lower levels of sexual risk behavior from baseline to follow-up (all p < 0.05). Perceptions of community empowerment also increased (p = 0.06). While HIV stigma decreased, this difference did not reach statistical significance. Our approach to community-engaged program development resulted in an acceptable, feasible approach to reaching and educating heterosexual Black men about HIV prevention in community settings.

A framework for improving the cost-effectiveness of DSM program evaluations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sonnenblick, R.; Eto, J.

The prudence of utility demand-side management (DSM) investments hinges on their performance, yet evaluating performance is complicated because the energy saved by DSM programs can never be observed directly but only inferred. This study frames and begins to answer the following questions: (1) how well do current evaluation methods perform in improving confidence in the measurement of energy savings produced by DSM programs; (2) in view of this performance, how can limited evaluation resources be best allocated to maximize the value of the information they provide? The authors review three major classes of methods for estimating annual energy savings: trackingmore » database (sometimes called engineering estimates), end-use metering, and billing analysis and examine them in light of the uncertainties in current estimates of DSM program measure lifetimes. The authors assess the accuracy and precision of each method and construct trade-off curves to examine the costs of increases in accuracy or precision. Several approaches for improving evaluations for the purpose of assessing program cost effectiveness are demonstrated. The methods can be easily generalized to other evaluation objectives, such as shared savings incentive payments.« less

Doctor coach: a deliberate practice approach to teaching and learning clinical skills.

PubMed

Gifford, Kimberly A; Fall, Leslie H

2014-02-01

The rapidly evolving medical education landscape requires restructuring the approach to teaching and learning across the continuum of medical education. The deliberate practice strategies used to coach learners in disciplines beyond medicine can also be used to train medical learners. However, these deliberate practice strategies are not explicitly taught in most medical schools or residencies. The authors designed the Doctor Coach framework and competencies in 2007-2008 to serve as the foundation for new faculty development and resident-as-teacher programs. In addition to teaching deliberate practice strategies, the programs model a deliberate practice approach that promotes the continuous integration of newly developed coaching competencies by participants into their daily teaching practice. Early evaluation demonstrated the feasibility and efficacy of implementing the Doctor Coach framework across the continuum of medical education. Additionally, the Doctor Coach framework has been disseminated through national workshops, which have resulted in additional institutions applying the framework and competencies to develop their own coaching programs. Design of a multisource evaluation tool based on the coaching competencies will enable more rigorous study of the Doctor Coach framework and training programs and provide a richer feedback mechanism for participants. The framework will also facilitate the faculty development needed to implement the milestones and entrustable professional activities in medical education.

Consumer engagement and the development, evaluation, and dissemination of evidence-based parenting programs.

PubMed

Sanders, Matthew R; Kirby, James N

2012-06-01

A consumer perspective can contribute much to enhancing the "ecological fit" of population-level parenting interventions so they meet the needs of parents. This approach involves building relationships with consumer groups and soliciting consumer input into the relevance and acceptability of interventions, clarifying the enablers and barriers to engagement and involvement of parents, and clarifying variables that influence a parent's program completion. The adoption of a more collaborative approach to working with consumers is important if meaningful population-level change in the prevalence of serious social, emotional, and behavioral problems in children and young people is to be achieved. Parents seeking assistance for their children's behavior come from a diverse range of socioeconomic backgrounds, educational levels, cultures, and languages. This paper examines consumer engagement strategies that can be employed throughout the process of program development, evaluation, training, and dissemination, and in "scaling up" the intervention. We argue that a multilevel public health approach to parenting intervention requires a strong consumer perspective to enable interventions to be more responsive to the preferences and needs of families and to ensure improved population reach of interventions. Examples from large-scale dissemination trials are used to illustrate how consumer input can result in an increasingly differentiated suite of evidence-based parenting programs. Copyright © 2011. Published by Elsevier Ltd.