The Evaluation and Research of Multi-Project Programs: Program Component Analysis.

ERIC Educational Resources Information Center

Baker, Eva L.

1977-01-01

It is difficult to base evaluations on concepts irrelevant to state policy making. Evaluation of a multiproject program requires both time and differentiation of method. Data from the California Early Childhood Program illustrate process variables for program component analysis, and research questions for intraprogram comparison. (CP)

Improving Mandatory Tutoring: A Mixed-Methods Program Evaluation

ERIC Educational Resources Information Center

Baggett, Brooks

2009-01-01

In recent years, the local school leadership in a suburban southern U.S. high school adopted innovative academic intervention programs to assist underperforming students but did not develop formal methods to evaluate program effectiveness. This gap in the professional practice continued with the inception of mandatory tutoring (MT), an…

Simulation for Nurse Anesthesia Program Selection: Redesigned

ERIC Educational Resources Information Center

Roebuck, John Arthur

2017-01-01

Purpose: This project is meant to answer the research question: What applicant character traits do Nurse Anesthesia Program Directors and Faculty identify as favorable predictors for successful completion of a nurse anesthesia program, and what evaluation methods are best to evaluate these traits in prospective students? Methods: A prospective…

Program Evaluation of Community College Learning Assistance Centers: What Do LAC Directors Think?

ERIC Educational Resources Information Center

Franklin, Doug; Blankenberger, Bob

2016-01-01

Objective: This study seeks to determine the nature of current program evaluation practices for learning assistance centers (LACs), the practices being used for program evaluation, and whether LAC directors believe their practices are appropriate for evaluating program effectiveness. Method: We conducted a survey (n = 61) of community college LAC…

Evaluating a physician leadership development program - a mixed methods approach.

PubMed

Throgmorton, Cheryl; Mitchell, Trey; Morley, Tom; Snyder, Marijo

2016-05-16

Purpose - With the extent of change in healthcare today, organizations need strong physician leaders. To compensate for the lack of physician leadership education, many organizations are sending physicians to external leadership programs or developing in-house leadership programs targeted specifically to physicians. The purpose of this paper is to outline the evaluation strategy and outcomes of the inaugural year of a Physician Leadership Academy (PLA) developed and implemented at a Michigan-based regional healthcare system. Design/methodology/approach - The authors applied the theoretical framework of Kirkpatrick's four levels of evaluation and used surveys, observations, activity tracking, and interviews to evaluate the program outcomes. The authors applied grounded theory techniques to the interview data. Findings - The program met targeted outcomes across all four levels of evaluation. Interview themes focused on the significance of increasing self-awareness, building relationships, applying new skills, and building confidence. Research limitations/implications - While only one example, this study illustrates the importance of developing the evaluation strategy as part of the program design. Qualitative research methods, often lacking from learning evaluation design, uncover rich themes of impact. The study supports how a PLA program can enhance physician learning, engagement, and relationship building throughout and after the program. Physician leaders' partnership with organization development and learning professionals yield results with impact to individuals, groups, and the organization. Originality/value - Few studies provide an in-depth review of evaluation methods and outcomes of physician leadership development programs. Healthcare organizations seeking to develop similar in-house programs may benefit applying the evaluation strategy outlined in this study.

Using Program Theory-Driven Evaluation Science to Crack the Da Vinci Code

ERIC Educational Resources Information Center

Donaldson, Stewart I.

2005-01-01

Program theory-driven evaluation science uses substantive knowledge, as opposed to method proclivities, to guide program evaluations. It aspires to update, clarify, simplify, and make more accessible the evolving theory of evaluation practice commonly referred to as theory-driven or theory-based evaluation. The evaluator in this chapter provides a…

Using Mixed Methods and Collaboration to Evaluate an Education and Public Outreach Program (Invited)

NASA Astrophysics Data System (ADS)

Shebby, S.; Shipp, S. S.

2013-12-01

Traditional indicators (such as the number of participants or Likert-type ratings of participant perceptions) are often used to provide stakeholders with basic information about program outputs and to justify funding decisions. However, use of qualitative methods can strengthen the reliability of these data and provide stakeholders with more meaningful information about program challenges, successes, and ultimate impacts (Stern, Stame, Mayne, Forss, David & Befani, 2012). In this session, presenters will discuss how they used a mixed methods evaluation to determine the impact of an education and public outreach (EPO) program. EPO efforts were intended to foster more effective, sustainable, and efficient utilization of science discoveries and learning experiences through three main goals 1) increase engagement and support by leveraging of resources, expertise, and best practices; 2) organize a portfolio of resources for accessibility, connectivity, and strategic growth; and 3) develop an infrastructure to support coordination. The evaluation team used a mixed methods design to conduct the evaluation. Presenters will first discuss five potential benefits of mixed methods designs: triangulation of findings, development, complementarity, initiation, and value diversity (Greene, Caracelli & Graham, 2005). They will next demonstrate how a 'mix' of methods, including artifact collection, surveys, interviews, focus groups, and vignettes, was included in the EPO project's evaluation design, providing specific examples of how alignment between the program theory and the evaluation plan was best achieved with a mixed methods approach. The presentation will also include an overview of different mixed methods approaches and information about important considerations when using a mixed methods design, such as selection of data collection methods and sources, and the timing and weighting of quantitative and qualitative methods (Creswell, 2003). Ultimately, this presentation will provide insight into how a mixed methods approach was used to provide stakeholders with important information about progress toward program goals. Creswell, J.W. (2003). Research design: Qualitative, quantitative, and mixed approaches. Thousand Oaks, CA: Sage. Greene, J. C., Caracelli, V. J., & Graham, W. D. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. Stern, E; Stame, N; Mayne, J; Forss, K; Davis, R & Befani, B (2012) Broadening the range of designs and methods for impact evaluation. Department for International Development.

Chapter 13: Assessing Persistence and Other Evaluation Issues Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Violette, Daniel M.

Addressing other evaluation issues that have been raised in the context of energy efficiency programs, this chapter focuses on methods used to address the persistence of energy savings, which is an important input to the benefit/cost analysis of energy efficiency programs and portfolios. In addition to discussing 'persistence' (which refers to the stream of benefits over time from an energy efficiency measure or program), this chapter provides a summary treatment of these issues -Synergies across programs -Rebound -Dual baselines -Errors in variables (the measurement and/or accuracy of input variables to the evaluation).

An Evaluation Model Applied to a Mathematics-Methods Program Involving Three Characteristics of Teaching Style and Their Relationship to Pupil Achievement. Teacher Education Forum; Volume 3, Number 4.

ERIC Educational Resources Information Center

Dodd, Carol Ann

This study explores a technique for evaluating teacher education programs in terms of teaching competencies, as applied to the Indiana University Mathematics Methods Program (MMP). The evaluation procedures formulated for the study include a process product design in combination with a modification of Pophan's performance test paradigm and Gage's…

A Mixed-Methods Exploration of an Environment for Learning Computer Programming

ERIC Educational Resources Information Center

Mather, Richard

2015-01-01

A mixed-methods approach is evaluated for exploring collaborative behaviour, acceptance and progress surrounding an interactive technology for learning computer programming. A review of literature reveals a compelling case for using mixed-methods approaches when evaluating technology-enhanced-learning environments. Here, ethnographic approaches…

Adapting Job Analysis Methodology to Improve Evaluation Practice

ERIC Educational Resources Information Center

Jenkins, Susan M.; Curtin, Patrick

2006-01-01

This article describes how job analysis, a method commonly used in personnel research and organizational psychology, provides a systematic method for documenting program staffing and service delivery that can improve evaluators' knowledge about program operations. Job analysis data can be used to increase evaluators' insight into how staffs…

ACORN's Accelerated Income Redistribution Project: A Program Evaluation

ERIC Educational Resources Information Center

Brooks, Fred; Russell, Daniel; Fisher, Robert

2006-01-01

Objective: This study evaluated the Association of Community Organizations for Reform Now's (ACORN) efforts to increase the uptake of families claiming the earned income tax credit through door-to-door canvassing and managing free tax preparation clinics in three pilot cities. Method: The mixed-method program evaluation included administrative…

Evaluation of Training Programs for Rural Development

ERIC Educational Resources Information Center

Indira, A.

2008-01-01

An Evaluation of the "Impact Assessment of the Training Programs" of a National Level Training Institution in India was conducted using the Kirkpatrick Method (KP Method). The studied Institution takes up research, provides training, offers consultancy and initiates action in the rural sector of India. The evaluation study used a…

A framework for improving the cost-effectiveness of DSM program evaluations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sonnenblick, R.; Eto, J.

The prudence of utility demand-side management (DSM) investments hinges on their performance, yet evaluating performance is complicated because the energy saved by DSM programs can never be observed directly but only inferred. This study frames and begins to answer the following questions: (1) how well do current evaluation methods perform in improving confidence in the measurement of energy savings produced by DSM programs; (2) in view of this performance, how can limited evaluation resources be best allocated to maximize the value of the information they provide? The authors review three major classes of methods for estimating annual energy savings: trackingmore » database (sometimes called engineering estimates), end-use metering, and billing analysis and examine them in light of the uncertainties in current estimates of DSM program measure lifetimes. The authors assess the accuracy and precision of each method and construct trade-off curves to examine the costs of increases in accuracy or precision. Several approaches for improving evaluations for the purpose of assessing program cost effectiveness are demonstrated. The methods can be easily generalized to other evaluation objectives, such as shared savings incentive payments.« less

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed

Colantonio, A

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed Central

Colantonio, A.

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations. PMID:2728498

Formative Process Evaluation for Implementing a Social Marketing Intervention to Increase Walking Among African Americans in the Positive Action for Today’s Health Trial

PubMed Central

Wilson, Dawn K.; Griffin, Sarah; St. George, Sara M.; Alia, Kassandra A.; Trumpeter, Nevelyn N.; Wandersman, Abraham K.; Forthofer, Melinda; Robinson, Shamika; Gadson, Barney

2012-01-01

Objectives. Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today’s Health trial. Methods: We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. Results: Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. Conclusions: Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities. PMID:23078486

Using Corporate-Based Methods To Assess Technical Communication Programs.

ERIC Educational Resources Information Center

Faber, Brenton; Bekins, Linn; Karis, Bill

2002-01-01

Investigates methods of program assessment used by corporate learning sites and profiles value added methods as a way to both construct and evaluate academic programs in technical communication. Examines and critiques assessment methods from corporate training environments including methods employed by corporate universities and value added…

Evaluating the Sharing Stories youth theatre program: an interactive theatre and drama-based strategy for sexual health promotion among multicultural youth.

PubMed

Roberts, Meagan; Lobo, Roanna; Sorenson, Anne

2017-03-01

Issue addressed Rates of sexually transmissible infections among young people are high, and there is a need for innovative, youth-focused sexual health promotion programs. This study evaluated the effectiveness of the Sharing Stories youth theatre program, which uses interactive theatre and drama-based strategies to engage and educate multicultural youth on sexual health issues. The effectiveness of using drama-based evaluation methods is also discussed. Methods The youth theatre program participants were 18 multicultural youth from South East Asian, African and Middle Eastern backgrounds aged between 14 and 21 years. Four sexual health drama scenarios and a sexual health questionnaire were used to measure changes in knowledge and attitudes. Results Participants reported being confident talking to and supporting their friends with regards to safe sex messages, improved their sexual health knowledge and demonstrated a positive shift in their attitudes towards sexual health. Drama-based evaluation methods were effective in engaging multicultural youth and worked well across the cultures and age groups. Conclusions Theatre and drama-based sexual health promotion strategies are an effective method for up-skilling young people from multicultural backgrounds to be peer educators and good communicators of sexual health information. Drama-based evaluation methods are engaging for young people and an effective way of collecting data from culturally diverse youth. So what? This study recommends incorporating interactive and arts-based strategies into sexual health promotion programs for multicultural youth. It also provides guidance for health promotion practitioners evaluating an arts-based health promotion program using arts-based data collection methods.

Graduate Program Evaluation in the Area Leading Educational, Outlying and Backward

ERIC Educational Resources Information Center

Prasetyono, Hendro

2016-01-01

This study aimed to evaluate the Bachelor program Educating in the Regions Frontier, Outermost and Disadvantaged (SM-3T). The research location is at the Jakarta State University. The evaluation method used is Cippo consisting of components Context, Input, Process, Product and Outcomes. The method used is qualitative descriptive data analysis…

A Controlled Evaluation of a High School Biomedical Pipeline Program: Design and Methods

ERIC Educational Resources Information Center

Winkleby, Marilyn A.; Ned, Judith; Ahn, David; Koehler, Alana; Fagliano, Kathleen; Crump, Casey

2014-01-01

Given limited funding for school-based science education, non-school-based programs have been developed at colleges and universities to increase the number of students entering science- and health-related careers and address critical workforce needs. However, few evaluations of such programs have been conducted. We report the design and methods of…

Interim reliability evaluation program, Browns Ferry fault trees

DOE Office of Scientific and Technical Information (OSTI.GOV)

Stewart, M.E.

1981-01-01

An abbreviated fault tree method is used to evaluate and model Browns Ferry systems in the Interim Reliability Evaluation programs, simplifying the recording and displaying of events, yet maintaining the system of identifying faults. The level of investigation is not changed. The analytical thought process inherent in the conventional method is not compromised. But the abbreviated method takes less time, and the fault modes are much more visible.

EVALUATION OF OXYGEN-ENRICHED MSW/SEWAGE SLUDGE CO-INCINERATION DEMONSTRATION PROGRAM

EPA Science Inventory

This report provides an evaluation of a two-phased demonstration program conducted for the U.S. Environmental Protection Agency's Municipal Solid Waste Innovative Technology Evaluation Program, and the results thereof, of a recently developed method of sewage sludge managemen...

Evaluating the economic benefits of nonmotorized transportation : case studies and methods for the nonmotorized transportation pilot program communities.

DOT National Transportation Integrated Search

2015-03-01

This report examines potential methods for evaluating the economic benefits from nonmotorized transportation investments. The variety of potential economic benefits of bicycle and pedestrian infrastructure and programming investments discussed includ...

Using evaluation methods to guide the development of a tobacco-use prevention curriculum for youth: a case study.

PubMed

Bridge, P D; Gallagher, R E; Berry-Bobovski, L C

2000-01-01

Fundamental to the development of educational programs and curricula is the evaluation of processes and outcomes. Unfortunately, many otherwise well-designed programs do not incorporate stringent evaluation methods and are limited in measuring program development and effectiveness. Using an advertising lesson in a school-based tobacco-use prevention curriculum as a case study, the authors examine the role of evaluation in the development, implementation, and enhancement of the curricular lesson. A four-phase formative and summative evaluation design was developed to divide the program-evaluation continuum into a structured process that would aid in the management of the evaluation, as well as assess curricular components. Formative and summative evaluation can provide important guidance in the development, implementation, and enhancement of educational curricula. Evaluation strategies identified unexpected barriers and allowed the project team to make necessary "time-relevant" curricular adjustments during each stage of the process.

Applying Program Theory-Driven Approach to Design and Evaluate a Teacher Professional Development Program

ERIC Educational Resources Information Center

Lin, Su-ching; Wu, Ming-sui

2016-01-01

This study was the first year of a two-year project which applied a program theory-driven approach to evaluating the impact of teachers' professional development interventions on students' learning by using a mix of methods, qualitative inquiry, and quasi-experimental design. The current study was to show the results of using the method of…

Legislative Evaluation.

ERIC Educational Resources Information Center

Fox, Harrison

The speaker discusses Congressional program evaluation. From the Congressional perspective, good evaluators understand the political, social, and economic processes; are familiar with various evaluation methods; and know how to use authority and power within their roles. Program evaluation serves three major purposes: to anticipate social impact…

Multimethod Strategy for Assessing Program Fidelity: The National Evaluation of the Revised G.R.E.A.T. Program

ERIC Educational Resources Information Center

Esbensen, Finn-Aage; Matsuda, Kristy N.; Taylor, Terrance J.; Peterson, Dana

2011-01-01

This study reports the results of the process evaluation component of the Process and Outcome Evaluation of the Gang Resistance Education and Training (G.R.E.A.T.) program. The process evaluation consisted of multiple methods to assess program fidelity: (a) observations of G.R.E.A.T. Officer Trainings (G.O.T); (b) surveys and interviews of…

The feasibility of a telephone coaching program on heart failure home management for family caregivers

PubMed Central

Piamjariyakul, Ubolrat; Smith, Carol E.; Russell, Christy; Werkowitch, Marilyn; Elyachar, Andrea

2012-01-01

Objectives To test the feasibility of delivery and evaluate the helpfulness of a coaching heart failure (HF) home management program for family caregivers. Background The few available studies on providing instruction for family caregivers are limited in content for managing HF home care and guidance for program implementation. Method This pilot study employed a mixed methods design. The measures of caregiver burden, confidence, and preparedness were compared at baseline and 3 months post-intervention. Descriptive statistics were used to summarize program costs and demographic data. Content analysis research methods were used to evaluate program feasibility and helpfulness. Results Caregiver (n=10) burden scores were significantly reduced and raw scores of confidence and preparedness for HF home management improved 3 months after the intervention. Content analyses of nurse and caregiver post-intervention data found caregivers rated the program as helpful and described how they initiated HF management skills based on the program. Conclusion The program was feasible to implement. These results suggest the coaching program should be further tested with a larger sample size to evaluate its efficacy. PMID:23116654

INTEGRATION OF SPATIAL DATA: METHODS EVALUATION WITH REGARD TO DATA ISSUES AND ASSESSMENT QUESTIONS

EPA Science Inventory

EPA's Regional Vulnerability Assessment (REVA) Program is developing and demonstrating approaches to assess current and future environmental vulnerabilities at a regional scale. An initial effort within this research program has been to develop and evaluate methods to synthesize ...

The Development and Implementation of a Model for Evaluating Clinical Specialty Education Programs.

ERIC Educational Resources Information Center

McLean, James E.; And Others

A new method for evaluating cancer education programs, using an external/internal evaluation team is outlined. The internal program staff are required to collect the data, arrange for a site visit, provide access to personnel, and make available other information requested by the evaluators. The external team consists of a dentist with oncological…

The Logic of Evaluation.

ERIC Educational Resources Information Center

Welty, Gordon A.

The logic of the evaluation of educational and other action programs is discussed from a methodological viewpoint. However, no attempt is made to develop methods of evaluating programs. In Part I, the structure of an educational program is viewed as a system with three components--inputs, transformation of inputs into outputs, and outputs. Part II…

A Program Evaluation of a Leadership Academy for School Principals

ERIC Educational Resources Information Center

Wagner, Kristi E.

2014-01-01

This program evaluation focused on mid-range outcomes of a leadership academy for school principals. The mixed-methods evaluation included interviews, principals' instructional observation database, and teacher surveys. The Principal Academy program was designed to build principals' knowledge of high-yield instructional strategies (Hattie, 2009),…

Using multiple methods to assess learning and outcomes in an online degree-granting dental hygiene program.

PubMed

Springfield, Emily; Gwozdek, Anne E; Peet, Melissa; Kerschbaum, Wendy E

2012-04-01

Program evaluation is a necessary component of curricular change and innovation. It ascertains whether an innovation has met benchmarks and contributes to the body of knowledge about educational methodologies and supports the use of evidence-based practice in teaching. Education researchers argue that rigorous program evaluation should utilize a mixed-method approach, triangulating both qualitative and quantitative methods to understand program effectiveness. This approach was used to evaluate the University of Michigan Dental Hygiene Degree Completion E-Learning (online) Program. Quantitative data included time spent on coursework, grades, publications, course evaluation results, and survey responses. Qualitative data included student and faculty responses in focus groups and on surveys as well as students' portfolio reflections. The results showed the program was academically rigorous, fostering students' ability to connect theory with practice and apply evidence-based practice principles. These results also demonstrated that the students had learned to critically reflect on their practice and develop expanded professional identities; going beyond the role of clinician, they began to see themselves as educators, advocates, and researchers. This evaluation model is easily adaptable and is applicable to any health science or other professional degree program. This study also raised important questions regarding the effect of meta-reflection on student confidence and professional behavior.

A Mixed Methods Program Evaluation on the Effectiveness of a School Redesign Model on Teacher Empowerment and Student Achievement

ERIC Educational Resources Information Center

Costa, Ann Marie

2012-01-01

A recent law in a New England state allowed public schools to operate with increased flexibility and autonomy through the authorization of the creation of Innovation Schools. This project study, a program evaluation using a convergent parallel mixed methods research design, allowed for a comprehensive evaluation of the first Innovation School…

The Nursing Leadership Institute program evaluation: a critique

PubMed Central

Havaei, Farinaz; MacPhee, Maura

2015-01-01

A theory-driven program evaluation was conducted for a nursing leadership program, as a collaborative project between university faculty, the nurses’ union, the provincial Ministry of Health, and its chief nursing officers. A collaborative logic model process was used to engage stakeholders, and mixed methods approaches were used to answer evaluation questions. Despite demonstrated, successful outcomes, the leadership program was not supported with continued funding. This paper examines what happened during the evaluation process: What factors failed to sustain this program? PMID:29355180

Responding to the Crisis of Accountability: A Review of Program Assessment Methods.

ERIC Educational Resources Information Center

Haley, Eric; Jackson, DeForrest

The advertising program at the University of Tennessee, Knoxville (UTK) has at least 12 measures of program assessment, which serve as a basis for discussion rather than as a prescription for an effective assessment program. The program assessment methods are accrediting, internal program review, teaching evaluations, a university survey of…

Qualitative Evaluation of Emotional Intelligence In-Service Program for Secondary School Teachers

ERIC Educational Resources Information Center

Fer, Seval

2004-01-01

This paper is an attempt to evaluate the Emotional Intelligence (EQ) In-Service Program on the basis of experiences of 20 secondary school teachers who attended the program in a private school in Turkey. A phenomenological approach, with a focus group method was used. The first objective of this study was to evaluate EQ program on the basis of…

Chapter 8: Whole-Building Retrofit with Consumption Data Analysis Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W.; Agnew, Ken; Goldberg, Mimi

Whole-building retrofits involve the installation of multiple measures. Whole-building retrofit programs take many forms. With a focus on overall building performance, these programs usually begin with an energy audit to identify cost-effective energy efficiency measures for the home. Measures are then installed, either at no cost to the homeowner or partially paid for by rebates and/or financing. The methods described here may also be applied to evaluation of single-measure retrofit programs. Related methods exist for replace-on-failure programs and for new construction, but are not the subject of this chapter.

An Evaluation of the Right Choices Program to Determine Effectiveness in Delivering Constructive Interventions and Providing an Early Support Program in Order to Modify Behavior of First-Time Student Offenders Who Commit Drug and Violent Acts

ERIC Educational Resources Information Center

Barnes, Lisa B.

2010-01-01

The purpose of the study was to perform a program evaluation of the Right Choices Program to determine the program's effectiveness in delivering constructive interventions that modify student behavior once students have left the program and have returned to their regular learning environment. This mixed-method evaluation consisted of an…

Assessing Change in the Teaching Practice of Faculty in a Faculty Development Program for Primary Care Physicians: Toward a Mixed Method Evaluation Approach.

ERIC Educational Resources Information Center

Pinheiro, Sandro O.; Rohrer, Jonathan D.; Heimann, C. F. Larry

This paper describes a mixed method evaluation study that was developed to assess faculty teaching behavior change in a faculty development fellowship program for community-based hospital faculty. Principles of adult learning were taught to faculty participants over the fellowship period. These included instruction in teaching methods, group…

A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming.

PubMed

Hansen, William B; Derzon, James H; Reese, Eric L

2014-06-01

We propose a method for creating groups against which outcomes of local pretest-posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness. © The Author(s) 2012.

From Knowledge to Action: Tips for Encouraging and Measuring Program-Related Behavior Change

ERIC Educational Resources Information Center

Chazdon, Scott; Horntvedt, Jody; Templin, Elizabeth

2016-01-01

It is challenging to document the behavior changes that result from Extension programming. This article describes an evaluation method we call the "action items method." Unlike other approaches for measuring behavior change, this method requires program participants to define their own action plans as part of a program and then asks them…

Formative process evaluation for implementing a social marketing intervention to increase walking among African Americans in the Positive Action for Today's Health trial.

PubMed

Coulon, Sandra M; Wilson, Dawn K; Griffin, Sarah; St George, Sara M; Alia, Kassandra A; Trumpeter, Nevelyn N; Wandersman, Abraham K; Forthofer, Melinda; Robinson, Shamika; Gadson, Barney

2012-12-01

Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today's Health trial. We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities.

About the mechanism of ERP-system pilot test

NASA Astrophysics Data System (ADS)

Mitkov, V. V.; Zimin, V. V.

2018-05-01

In the paper the mathematical problem of defining the scope of pilot test is stated, which is a task of quadratic programming. The procedure of the problem solving includes the method of network programming based on the structurally similar network representation of the criterion and constraints and which reduces the original problem to a sequence of simpler evaluation tasks. The evaluation tasks are solved by the method of dichotomous programming.

Measuring Success in Obesity Prevention: A Synthesis of Health Promotion Switzerland's Long-Term Monitoring and Evaluation Strategy

PubMed Central

Ackermann, Günter; Kirschner, Michael; Guggenbühl, Lisa; Abel, Bettina; Klohn, Axel; Mattig, Thomas

2015-01-01

Aims Since 2007, Health Promotion Switzerland has implemented a national priority program for a healthy body weight. This article provides insight into the methodological challenges and results of the program evaluation. Methods Evaluation of the long-term program required targeted monitoring and evaluation projects addressing different outcome levels. The evaluation was carried out according to the Swiss Model for Outcome Classification (SMOC), a model designed to classify the effects of health promotion and prevention efforts. Results The results presented in this article emphasize both content and methods. The national program successfully achieved outcomes on many different levels within complex societal structures. The evaluation system built around the SMOC enabled assessment of program progress and the development of key indicators. However, it is not possible to determine definitively to what extent the national program helped stabilize the prevalence of obesity in Switzerland. Conclusion The model has shown its utility in providing a basis for evaluation and monitoring of the national program. Continuous analysis of data from evaluation and monitoring has made it possible to check the plausibility of suspected causal relationships as well as to establish an overall perspective and assessment of effectiveness supported by a growing body of evidence. PMID:25765161

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program

PubMed Central

Jaafaripooyan, Ebrahim

2014-01-01

Background: Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. Methods: A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Results: Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs’ over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Conclusion: Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs. PMID:25279381

Some Methods for Evaluating Program Implementation.

ERIC Educational Resources Information Center

Hardy, Roy A.

An approach to evaluating program implementation is described. This approach includes the development of a project description which includes a structure matrix, sampling from the structure matrix, and preparing an implementation evaluation plan. The implementation evaluation plan should include: (1) verification of implementation of planned…

The Evaluator's Perspective: Evaluating the State Capacity Building Program.

ERIC Educational Resources Information Center

Madey, Doren L.

A historical antagonism between the advocates of quantitative evaluation methods and the proponents of qualitative evaluation methods has stymied the recognition of the value to be gained by utilizing both methodologies in the same study. The integration of quantitative and qualitative methods within a single evaluation has synergistic effects in…

Evaluation of Supplemental Nutrition Assistance Program Education: Application of Behavioral Theory and Survey Validation

ERIC Educational Resources Information Center

Wyker, Brett A.; Jordan, Patricia; Quigley, Danielle L.

2012-01-01

Objective: Application of the Transtheoretical Model (TTM) to Supplemental Nutrition Assistance Program Education (SNAP-Ed) evaluation and development and validation of an evaluation tool used to measure TTM constructs is described. Methods: Surveys were collected from parents of children receiving food at Summer Food Service Program sites prior…

Evaluation of Mental Health Programs for the Aged.

ERIC Educational Resources Information Center

Kahn, Robert L.; Zarit, Steven H.

This paper highlights what the authors believe are the important issues and directions of change in the evaluation of mental health programs. The rationale for such evaluation is twofold. First, it provides a scientifically rigorous method of determining the therapeutic efficacy of the treatment or program, and secondly, these results can exercise…

Analyzing for Individual Differences in Evaluating Compensatory Education Programs. Occasional Paper No. 27.

ERIC Educational Resources Information Center

Krus, Patricia H.

Trait-Treatment Interaction (TTI), a research method for observing experimental effects of treatments on subjects of different aptitudes and learning characteristics, is suggested as an effective evaluation tool to provide evaluators and educators in compensatory education programs with information about which program is best for different kinds…

A Collaborative Approach to Family Literacy Evaluation Strategies.

ERIC Educational Resources Information Center

Landerholm, Elizabeth; Karr, Jo Ann; Mushi, Selina

A collaborative approach to program evaluation combined with the use of a variety of evaluation methods using currently available technology can yield valuable information about the effectiveness of family literacy programs. Such an approach was used for McCosh Even Start, a federally-funded family literacy program located at McCosh School in an…

A Program Evaluation of the Basic Supervisory Development Seminar

DTIC Science & Technology

1985-08-01

evaluation. - Hoyle , English, and Steffy (1985) describe various methods that can be used to accomplisn the appropriate objectives of the evaluation...Maynihan Figure 2 Alternative Methods of Assessing Student and Program Effectiveness ( Hoyle , English, and Steffy, 1985) 21 CHAPTER III METHODCLOGY S Design...Fiedler, Fred E. "Responses to Sergiovanni." Educational Leadership (March 1979), 394-96. Fiedler, Fred E., and Martin M. Chemers. Leadership and

Residency Programs' Evaluations of the Competencies: Data Provided to the ACGME About Types of Assessments Used by Programs

PubMed Central

Holt, Kathleen D.; Miller, Rebecca S.; Nasca, Thomas J.

2010-01-01

Background In 1999, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project began to focus on resident performance in the 6 competencies of patient care, medical knowledge, professionalism, practice-based learning and improvement, interpersonal communication skills, and professionalism. Beginning in 2007, the ACGME began collecting information on how programs assess these competencies. This report provides information on the nature and extent of those assessments. Methods Using data collected by the ACGME for site visits, we use descriptive statistics and percentages to describe the number and type of methods and assessors accredited programs (n  =  4417) report using to assess the competencies. Observed differences among specialties, methodologies, and assessors are tested with analysis of variance procedures. Results Almost all (>97%) of programs report assessing all of the competencies and using multiple methods and multiple assessors. Similar assessment methods and evaluator types were consistently used across the 6 competencies. However, there were some differences in the use of patient and family as assessors: Primary care and ambulatory specialties used these to a greater extent than other specialties. Conclusion Residency programs are emphasizing the competencies in their evaluation of residents. Understanding the scope of evaluation methodologies that programs use in resident assessment is important for both the profession and the public, so that together we may monitor continuing improvement in US graduate medical education. PMID:22132294

Student Perspectives: Evaluating a Higher Education Administration Program

ERIC Educational Resources Information Center

Roberts, Jalynn; Gentry, Debra; Townsend, Amy

2011-01-01

The purpose of this mixed-method study was to evaluate a Higher Education Administration doctoral program, to understand student perceptions of program quality, to improve both student satisfaction and retention to degree completion, and to plan for the future. Findings from survey data and focus group interviews indicated program strengths to…

Assessing the Value of the Enviroscape Watershed Learning Module

ERIC Educational Resources Information Center

Edwards, Warren Patrick

2013-01-01

Scope and Method of Study: The researcher's evaluation of the West Atlanta Watershed Alliance's (WAWA) programs highlighted that few if any of the offered educational programs included a program evaluation, especially the most promising, the Enviroscape® Watershed learning module. The education programs that were customized and developed by the…

Oasis Connections: Results from an Evaluation Study

ERIC Educational Resources Information Center

Czaja, Sara J.; Lee, Chin Chin; Branham, Janice; Remis, Peggy

2012-01-01

Purpose of the Study: The objectives of this study were to evaluate a community-based basic computer and Internet training program designed for older adults, provide recommendations for program refinement, and gather preliminary information on program sustainability. Design and Methods: The program was developed by the OASIS Institute, a nonprofit…

ADDENDUM TO SUPERFUND INNOVATIVE TECHNOLOGY EVALUATION PROGRAM TECHNOLOGY PROFILES, TENTH EDITION, VOLUME 1 - DEMONSTRATION PROGRAM

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its thirteenth year, is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine us...

Space Storable Rocket Technology program (SSRT). Option 1 program

NASA Astrophysics Data System (ADS)

Chazen, Melvin L.; Mueller, Thomas; Rust, Thomas

1993-08-01

The Space Storable Rocket Technology (SSRT) Option 1 Program was initiated in October 1991 after completion of the Basic Program. The program was restructured in mid-July 1992 to incorporate a Rhenium Technology Task and reduce the scope of the LO2-N2H4 engine development. The program was also extended to late February 1993 to allow for the Rhenium Technology Task completion. The Option 1 Program was devoted to evaluation of two new injector elements, evaluation of two different methods of thermal protection of the injector, evaluation of high temperature material properties of rhenium and evaluation of methods of joining the rhenium thrust chamber to the columbium injector and nozzle extension. In addition, critical experiments were conducted (Funded by Option 2) to evaluate mechanisms to understand the effects of GO2 injection into the chamber, helium injection into the main LO2, effect of the splash plate and effect of decreasing the aspect ratio of the 120-slot (-13a) element. The performance and thermal models were used to further correlate the test results with analyses. The results of the work accomplished are summarized.

Space Storable Rocket Technology program (SSRT). Option 1 program

NASA Technical Reports Server (NTRS)

Chazen, Melvin L.; Mueller, Thomas; Rust, Thomas

1993-01-01

The Space Storable Rocket Technology (SSRT) Option 1 Program was initiated in October 1991 after completion of the Basic Program. The program was restructured in mid-July 1992 to incorporate a Rhenium Technology Task and reduce the scope of the LO2-N2H4 engine development. The program was also extended to late February 1993 to allow for the Rhenium Technology Task completion. The Option 1 Program was devoted to evaluation of two new injector elements, evaluation of two different methods of thermal protection of the injector, evaluation of high temperature material properties of rhenium and evaluation of methods of joining the rhenium thrust chamber to the columbium injector and nozzle extension. In addition, critical experiments were conducted (Funded by Option 2) to evaluate mechanisms to understand the effects of GO2 injection into the chamber, helium injection into the main LO2, effect of the splash plate and effect of decreasing the aspect ratio of the 120-slot (-13a) element. The performance and thermal models were used to further correlate the test results with analyses. The results of the work accomplished are summarized.

Assessing the Subsequent Effect of a Formative Evaluation on a Program.

ERIC Educational Resources Information Center

Brown, J. Lynne; Kiernan, Nancy Ellen

2001-01-01

Conducted a formative evaluation of an osteoporosis prevention health education program using several methods, including questionnaires completed by 256 women, and then compared formative evaluation results to those of a summative evaluation focusing on the same target group. Results show the usefulness of formative evaluation for strengthening…

Chapter 11: Sample Design Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Khawaja, M. Sami; Rushton, Josh

Evaluating an energy efficiency program requires assessing the total energy and demand saved through all of the energy efficiency measures provided by the program. For large programs, the direct assessment of savings for each participant would be cost-prohibitive. Even if a program is small enough that a full census could be managed, such an undertaking would almost always be an inefficient use of evaluation resources. The bulk of this chapter describes methods for minimizing and quantifying sampling error. Measurement error and regression error are discussed in various contexts in other chapters.

Tailored program evaluation: Past, present, future.

PubMed

Suggs, L Suzanne; Cowdery, Joan E; Carroll, Jennifer B

2006-11-01

This paper discusses measurement issues related to the evaluation of computer-tailored health behavior change programs. As the first generation of commercially available tailored products is utilized in health promotion programming, programmers and researchers are becoming aware of the unique challenges that the evaluation of these programs presents. A project is presented that used an online tailored health behavior assessment (HBA) in a worksite setting. Process and outcome evaluation methods are described and include the challenges faced, and strategies proposed and implemented, for meeting them. Implications for future research in tailored program development, implementation, and evaluation are also discussed.

Methods for Evaluating Mammography Imaging Techniques

DTIC Science & Technology

1999-06-01

Distribution Unlimited 12b. DIS5TRIBUTION CODE 13. ABSTRACT (Maximum 200 words) This Department of Defense Breast Cancer Research Program Career...Development Award is enabling Dr. Rütter to develop bio’statistical methods for breast cancer research. Dr. Rutter is focusing on methods for...evaluating the accuracy of breast cancer screening. This four year program includes advanced training in the epidemiology of breast cancer , training in

Alternatives to Piloting Textbooks.

ERIC Educational Resources Information Center

Muther, Connie

1985-01-01

Using short-term pilot programs to evaluate textbooks can lead to unreliable results and interfere with effective education. Alternative methods for evaluating textbook-based programs include obtaining documented analyses of competitors' products from sales agents, visiting districts using programs being considered, and examining publishers' own…

Needs assessment and evaluation of a short course to improve faculties teaching skills at a former World Health Organization regional teacher training center

PubMed Central

KOJURI, JAVAD; AMINI, MITRA; KARIMIAN, ZAHRA; DEHGHANI, MOHAMMAD REZA; SABER, MAHBOOBEH; BAZRAFCAN, LEILA; EBRAHIMI, SEDIGHEH; REZAEE, RITA

2015-01-01

Introduction: In the design of educational programs, much attention has been paid to teaching methods, needs assessment, an important part of the development of educational programs, generally is neglected. Another important aspect in educational program design is assessing effectiveness. The aims of this study were to design a formal needs assessment program to define the core contents of a faculty development program, and to determine whether participation in the faculty development program reinforced new teaching skills. Methods: A teacher-training program was designed at Shiraz University of Medical Sciences to help medical instructors boost their teaching skills. Needs assessment was done with nominal group technique followed by a 5-point Likert scale questionnaire. The program, imparted in workshop format, covered effective teaching methods, feedback, assessing knowledge and time management. Instruction was in the form of lectures, group discussions, case simulations, video presentations and role-plays. The program was evaluated in several phases using data triangulation and multi-item assessments of overall program quality in three major dimensions: Kirkpatrick program evaluation model, evaluation of the educational environment and qualitative analysis with open-ended questions. All participants in the study belonged to the academic staff of Shiraz University of Medical Sciences (n=396). Results: Seven main categories were derived from nominal group techniques and questionnaires. After the program, participants rated the quality of the program highly. They felt that the educational intervention was appropriate and had a positive impact on their knowledge of effective teaching methods, feedback, knowledge assessment and time management. Assessment of the effectiveness of the program showed that participants reported significant improvements in their teaching abilities. Conclusions: Our faculty development program  have a significant positive effect on medical university teaching staff members’ competencies. Further research is needed to investigate whether the faculty development program actually results in improved teaching performance. PMID:25587548

Evaluating the effectiveness of clinical medical librarian programs: a systematic review of the literature*

PubMed Central

Wagner, Kay Cimpl; Byrd, Gary D.

2004-01-01

Objective: This study was undertaken to determine if a systematic review of the evidence from thirty years of literature evaluating clinical medical librarian (CML) programs could help clarify the effectiveness of this outreach service model. Methods: A descriptive review of the CML literature describes the general characteristics of these services as they have been implemented, primarily in teaching-hospital settings. Comprehensive searches for CML studies using quantitative or qualitative evaluation methods were conducted in the medical, allied health, librarianship, and social sciences literature. Findings: Thirty-five studies published between 1974 and 2001 met the review criteria. Most (30) evaluated single, active programs and used descriptive research methods (e.g., use statistics or surveys/questionnaires). A weighted average of 89% of users in twelve studies found CML services useful and of high quality, and 65% of users in another overlapping, but not identical, twelve studies said these services contributed to improved patient care. Conclusions: The total amount of research evidence for CML program effectiveness is not great and most of it is descriptive rather than comparative or analytically qualitative. Standards are needed to consistently evaluate CML or informationist programs in the future. A carefully structured multiprogram study including three to five of the best current programs is needed to define the true value of these services. PMID:14762460

Steps to a HealthierUS Cooperative Agreement Program: foundational elements for program evaluation planning, implementation, and use of findings.

PubMed

MacDonald, Goldie; Garcia, Danyael; Zaza, Stephanie; Schooley, Michael; Compton, Don; Bryant, Terry; Bagnol, Lulu; Edgerly, Cathy; Haverkate, Rick

2006-01-01

The Steps to a HealthierUS Cooperative Agreement Program (Steps Program) enables funded communities to implement chronic disease prevention and health promotion efforts to reduce the burden of diabetes, obesity, asthma, and related risk factors. At both the national and community levels, investment in surveillance and program evaluation is substantial. Public health practitioners engaged in program evaluation planning often identify desired outcomes, related indicators, and data collection methods but may pay only limited attention to an overarching vision for program evaluation among participating sites. We developed a set of foundational elements to provide a vision of program evaluation that informs the technical decisions made throughout the evaluation process. Given the diversity of activities across the Steps Program and the need for coordination between national- and community-level evaluation efforts, our recommendations to guide program evaluation practice are explicit yet leave room for site-specific context and needs. Staff across the Steps Program must consider these foundational elements to prepare a formal plan for program evaluation. Attention to each element moves the Steps Program closer to well-designed and complementary plans for program evaluation at the national, state, and community levels.

Why so many "rigorous" evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute.

PubMed

Bamberger, Michael; Tarsilla, Michele; Hesse-Biber, Sharlene

2016-04-01

Many widely-used impact evaluation designs, including randomized control trials (RCTs) and quasi-experimental designs (QEDs), frequently fail to detect what are often quite serious unintended consequences of development programs. This seems surprising as experienced planners and evaluators are well aware that unintended consequences frequently occur. Most evaluation designs are intended to determine whether there is credible evidence (statistical, theory-based or narrative) that programs have achieved their intended objectives and the logic of many evaluation designs, even those that are considered the most "rigorous," does not permit the identification of outcomes that were not specified in the program design. We take the example of RCTs as they are considered by many to be the most rigorous evaluation designs. We present a numbers of cases to illustrate how infusing RCTs with a mixed-methods approach (sometimes called an "RCT+" design) can strengthen the credibility of these designs and can also capture important unintended consequences. We provide a Mixed Methods Evaluation Framework that identifies 9 ways in which UCs can occur, and we apply this framework to two of the case studies. Copyright © 2016 Elsevier Ltd. All rights reserved.

Testing Mediators of Intervention Effects in Randomized Controlled Trials: An Evaluation of Three Depression Prevention Programs

ERIC Educational Resources Information Center

Stice, Eric; Rohde, Paul; Seeley, John R.; Gau, Jeff M.

2010-01-01

Objective: Evaluate a new 5-step method for testing mediators hypothesized to account for the effects of depression prevention programs. Method: In this indicated prevention trial, at-risk teens with elevated depressive symptoms were randomized to a group cognitive-behavioral (CB) intervention, group supportive expressive intervention, CB…

Comparing Propensity Score Methods in Balancing Covariates and Recovering Impact in Small Sample Educational Program Evaluations

ERIC Educational Resources Information Center

Stone, Clement A.; Tang, Yun

2013-01-01

Propensity score applications are often used to evaluate educational program impact. However, various options are available to estimate both propensity scores and construct comparison groups. This study used a student achievement dataset with commonly available covariates to compare different propensity scoring estimation methods (logistic…

Process Evaluation of a Parenting Program for Low-Income Families in South Africa

ERIC Educational Resources Information Center

Lachman, Jamie M.; Kelly, Jane; Cluver, Lucie; Ward, Catherine L.; Hutchings, Judy; Gardner, Frances

2018-01-01

Objective: This mixed-methods process evaluation examined the feasibility of a parenting program delivered by community facilitators to reduce the risk of child maltreatment in low-income families with children aged 3-8 years in Cape Town, South Africa (N = 68). Method: Quantitative measures included attendance registers, fidelity checklists,…

Development of a Program Logic Model and Evaluation Plan for a Participatory Ergonomics Intervention in Construction

PubMed Central

Jaegers, Lisa; Dale, Ann Marie; Weaver, Nancy; Buchholz, Bryan; Welch, Laura; Evanoff, Bradley

2013-01-01

Background Intervention studies in participatory ergonomics (PE) are often difficult to interpret due to limited descriptions of program planning and evaluation. Methods In an ongoing PE program with floor layers, we developed a logic model to describe our program plan, and process and summative evaluations designed to describe the efficacy of the program. Results The logic model was a useful tool for describing the program elements and subsequent modifications. The process evaluation measured how well the program was delivered as intended, and revealed the need for program modifications. The summative evaluation provided early measures of the efficacy of the program as delivered. Conclusions Inadequate information on program delivery may lead to erroneous conclusions about intervention efficacy due to Type III error. A logic model guided the delivery and evaluation of our intervention and provides useful information to aid interpretation of results. PMID:24006097

Connected Classroom: A Program Evaluation of the Professional Development Program of a One-to-One Educational Technology Initiative in South Carolina

ERIC Educational Resources Information Center

Grant, Kelly J.

2016-01-01

The purpose of this study was to evaluate the impact of the first year of a multi-year, district-wide professional development program for teachers that accompanied a one-to-one Apple device rollout for all students. A mixed-method research design was used to perform a logic model of program evaluation. Teacher self-reported proficiency in basic…

Cancer Education Program Evaluation: A Responsive Approach to Planning an Evaluation and Initial Results.

ERIC Educational Resources Information Center

Pearsol, James A.

This paper describes evaluation planning for the Cancer Education Program (CEP) at Ohio State University (OSU). The three-year OSU CEP project was designed as a multidisciplinary cancer education program. A responsive method, which trades off some measurement precision in order to increase the usefulness of the findings, was employed in the…

75 FR 57027 - National Toxicology Program (NTP); NTP Interagency Center for the Evaluation of Alternative...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-09-17

... for the Evaluation of Alternative Toxicological Methods (NICEATM); Availability of Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) Test Method Evaluation Reports: In Vitro Ocular Safety Testing Methods and Strategies, and Routine Use of Topical Anesthetics, Systemic...

Methods for comparative evaluation of propulsion system designs for supersonic aircraft

NASA Technical Reports Server (NTRS)

Tyson, R. M.; Mairs, R. Y.; Halferty, F. D., Jr.; Moore, B. E.; Chaloff, D.; Knudsen, A. W.

1976-01-01

The propulsion system comparative evaluation study was conducted to define a rapid, approximate method for evaluating the effects of propulsion system changes for an advanced supersonic cruise airplane, and to verify the approximate method by comparing its mission performance results with those from a more detailed analysis. A table look up computer program was developed to determine nacelle drag increments for a range of parametric nacelle shapes and sizes. Aircraft sensitivities to propulsion parameters were defined. Nacelle shapes, installed weights, and installed performance was determined for four study engines selected from the NASA supersonic cruise aircraft research (SCAR) engine studies program. Both rapid evaluation method (using sensitivities) and traditional preliminary design methods were then used to assess the four engines. The method was found to compare well with the more detailed analyses.

Evaluating the BK 21 Program. Research Brief

ERIC Educational Resources Information Center

Seong, Somi; Popper, Steven W.; Goldman, Charles A.; Evans, David K.; Grammich, Clifford A.

2008-01-01

The Brain Korea 21 program (BK21), an effort to improve Korean universities and research, has attracted a great deal of attention in Korea, producing the need to understand how well the program is meeting its goals. RAND developed a logic model for identifying program goals and dynamics, suggested quantitative and qualitative evaluation methods,…

The development and validation of The Inquiry Science Observation Coding Sheet.

PubMed

Brandon, P R; Taum, A K H; Young, D B; Pottenger, F M

2008-08-01

Evaluation reports increasingly document the degree of program implementation, particularly the extent to which programs adhere to prescribed steps and procedures. Many reports are cursory, however, and few, if any, fully portray the long and winding path taken when developing evaluation instruments, particularly observation instruments. In this article, we describe the development of an observational method for evaluating the degree to which K-12 inquiry science programs are implemented, including the many steps and decisions that occurred during the development, and present evidence for the reliability and validity of the data that we collected with the instrument. The article introduces a method for measuring the adherence of inquiry science implementation and gives evaluators a full picture of what they might expect when developing observation instruments for assessing the degree of program implementation.

Implementation and Evaluation of a Smartphone-Based Telemonitoring Program for Patients With Heart Failure: Mixed-Methods Study Protocol

PubMed Central

Ross, Heather J; Cafazzo, Joseph A; Laporte, Audrey; Seto, Emily

2018-01-01

Background Meta-analyses of telemonitoring for patients with heart failure conclude that it can lower the utilization of health services and improve health outcomes compared with the standard of care. A smartphone-based telemonitoring program is being implemented as part of the standard of care at a specialty care clinic for patients with heart failure in Toronto, Canada. Objective The objectives of this study are to (1) evaluate the impact of the telemonitoring program on health service utilization, patient health outcomes, and their ability to self-care; (2) identify the contextual barriers and facilitators of implementation at the physician, clinic, and institutional level; (3) describe patient usage patterns to determine adherence and other behaviors in the telemonitoring program; and (4) evaluate the costs associated with implementation of the telemonitoring program from the perspective of the health care system (ie, public payer), hospital, and patient. Methods The evaluation will use a mixed-methods approach. The quantitative component will include a pragmatic pre- and posttest study design for the impact and cost analyses, which will make use of clinical data and questionnaires administered to at least 108 patients at baseline and 6 months. Furthermore, outcome data will be collected at 1, 12, and 24 months to explore the longitudinal impact of the program. In addition, quantitative data related to implementation outcomes and patient usage patterns of the telemonitoring system will be reported. The qualitative component involves an embedded single case study design to identify the contextual factors that influenced the implementation. The implementation evaluation will be completed using semistructured interviews with clinicians, and other program staff at baseline, 4 months, and 12 months after the program start date. Interviews conducted with patients will be triangulated with usage data to explain usage patterns and adherence to the system. Results The telemonitoring program was launched in August 2016 and patient enrollment is ongoing. Conclusions The methods described provide an example for conducting comprehensive evaluations of telemonitoring programs. The combination of impact, implementation, and cost evaluations will inform the quality improvement of the existing program and will yield insights into the sustainability of smartphone-based telemonitoring programs for patients with heart failure within a specialty care setting. PMID:29724704

A combined impact-process evaluation of a program promoting active transport to school: understanding the factors that shaped program effectiveness.

PubMed

Crawford, S; Garrard, J

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts.

A Combined Impact-Process Evaluation of a Program Promoting Active Transport to School: Understanding the Factors That Shaped Program Effectiveness

PubMed Central

Crawford, S.; Garrard, J.

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts. PMID:23606865

Sediment Management Methods to Reduce Dredging: Part 2, Sediment Collector Technology

DTIC Science & Technology

2017-04-01

Approved for public release; distribution is unlimited. ERDC TN-DOER-T13 April 2017 Sediment Management Methods to Reduce Dredging: Part 2...DOER) Program technical note (TN) is the second in a series evaluating sediment management methods to reduce dredging through a research task (RT...in the DOER Program.1 This TN presents an evaluation of sediment collector technology, one promising new device that may help better manage

Children's Programs: A Comparative Evaluation Framework and Five Illustrations. Briefing Report to the Ranking Minority Member, Select Committee on Children, Youth, and Families, House of Representatives.

ERIC Educational Resources Information Center

General Accounting Office, Washington, DC. Program Evaluation and Methodology Div.

This general program evaluation framework provides a wide range of criteria that can be applied in the evaluation of diverse federal progams. The framework was developed from a literature search on program evaluation methods and their use, the experiences of the United States Government Accounting Office (GAO), and consideration of the types of…

Evaluation: An Imperative for Public Education and Training Programs. Professorial Inaugural Lecture Series.

ERIC Educational Resources Information Center

McCaslin, N. L.

Evaluation is important to public education and training programs if the United States is to be competitive in a global economy. Six major factors have had an impact on evaluation efforts: decreased financial resources, increased public dissatisfaction, changed management approaches, enhanced evaluation methods and procedures, redesigned…

Evaluation of an online continuing education program from the perspective of new graduate nurses.

PubMed

Karaman, Selcuk; Kucuk, Sevda; Aydemir, Melike

2014-05-01

The aim of this study is to evaluate the online continuing education program from the perspectives of new graduate nurses. An evaluation framework includes five factors (program and course structure, course materials, technology, support services and assessment). In this study, descriptive research methods were used. Participants of the study included 2.365 registered nurses enrolled in the first online nursing bachelor completion degree program in the country. Data were collected by survey. The findings indicated that students were mostly satisfied with this program. The results of this study suggest that well designed asynchronous online education methods can be effective and appropriate for registered nurses. However, the provision of effective support and technological infrastructure is as vital as the quality of teaching for online learners. © 2013.

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

The report is an overview of electric energy efficiency programs. It takes a concise look at what states are doing to encourage energy efficiency and how it impacts electric utilities. Energy efficiency programs began to be offered by utilities as a response to the energy crises of the 1970s. These regulatory-driven programs peaked in the early-1990s and then tapered off as deregulation took hold. Today, rising electricity prices, environmental concerns, and national security issues have renewed interest in increasing energy efficiency as an alternative to additional supply. In response, new methods for administering, managing, and delivering energy efficiency programs aremore » being implemented. Topics covered in the report include: Analysis of the benefits of energy efficiency and key methods for achieving energy efficiency; evaluation of the business drivers spurring increased energy efficiency; Discussion of the major barriers to expanding energy efficiency programs; evaluation of the economic impacts of energy efficiency; discussion of the history of electric utility energy efficiency efforts; analysis of the impact of energy efficiency on utility profits and methods for protecting profitability; Discussion of non-utility management of energy efficiency programs; evaluation of major methods to spur energy efficiency - systems benefit charges, resource planning, and resource standards; and, analysis of the alternatives for encouraging customer participation in energy efficiency programs.« less

An Approach for Selecting a Theoretical Framework for the Evaluation of Training Programs

ERIC Educational Resources Information Center

Tasca, Jorge Eduardo; Ensslin, Leonardo; Ensslin, Sandra Rolim; Alves, Maria Bernardete Martins

2010-01-01

Purpose: This research paper proposes a method for selecting references related to a research topic, and seeks to exemplify it for the case of a study evaluating training programs. The method is designed to identify references with high academic relevance in databases accessed via the internet, using a bibliometric analysis to sift the selected…

Family Assessment/Treatment/Evaluation Methods Integrated for Helping Teen Suicide Attempters/Families in Short Term Psychiatric Hospitalization Programs.

ERIC Educational Resources Information Center

Shepard, Suzanne

The assessment process can be integrated with treatment and evaluation for helping teenage suicide attempters and families in short term psychiatric hospitalization programs. The method is an extremely efficient way for the therapist to work within a given time constraint. During family assessment sufficient information can be gathered to…

Implementing a Service Learning Model for Teaching Research Methods and Program Evaluation

ERIC Educational Resources Information Center

Shannon, Patrick; Kim, Wooksoo; Robinson, Adjoa

2012-01-01

In an effort to teach students the basic knowledge of research methods and the realities of conducting research in the context of agencies in the community, faculty developed and implemented a service learning model for teaching research and program evaluation to foundation-year MSW students. A year-long foundation course was designed in which one…

Overcoming Language and Literacy Barriers: Using Student Response System Technology to Collect Quality Program Evaluation Data from Immigrant Participants

ERIC Educational Resources Information Center

Walker, Susan K.; Mao, Dung

2016-01-01

Student response system technology was employed for parenting education program evaluation data collection with Karen adults. The technology, with translation and use of an interpreter, provided an efficient and secure method that respected oral language and collective learning preferences and accommodated literacy needs. The method was popular…

The Evaluation of Health Education Program (HEP) of 9th Graders

ERIC Educational Resources Information Center

Erdogan, Polat; Gürol, Mehmet

2016-01-01

The main purpose of evaluation is to improve the quality of program. So, the purpose of this study was to evaluate the Health Education Program of 9th graders (HEP) with Educational Criticism Model of Eisner. The study was conducted with the qualitative research method. A holistic single case design was employed in this study. The typical case…

Evaluation of an Individualized Continuing Education Program for Physicians and Nurse Practitioners: An Example of the Situational Nature of Program Evaluation.

ERIC Educational Resources Information Center

Tresolini, Carol P.; Savage, Katherine D.; Hedgpeth, Marian Wells; Curtis, Peter

The Visiting Clinician Program (VCP) was established in 1996 at a public medical school to provide individualized continuing education to participants and to foster closer ties between academic health center faculty and community practitioners who serve as preceptors for health professions students. Various methods have been used to evaluate the…

78 FR 45253 - National Toxicology Program Scientific Advisory Committee on Alternative Toxicological Methods...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-07-26

... Scientific Advisory Committee on Alternative Toxicological Methods; Announcement of Meeting; Request for... Toxicological Methods (SACATM). SACATM advises the Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM), the National Toxicology Program (NTP) Interagency Center for the Evaluation of...

Client Assessment in an Industrial Setting: A Cross-Sectional Method.

ERIC Educational Resources Information Center

Yamatani, Hide

1988-01-01

Used cross-sectional method for evaluating social service programs in industrial setting to estimate numbers of workers needing social services, levels of program use, and penetration and to examine program outcome. Workers served by social service or employee assistance programs can be examined to determine additional services needed, adequacy of…

Partnering with the Local Livestock Market in Educational Programs.

ERIC Educational Resources Information Center

Jenkins, Jamie H.; Newman, Michael E.; Castellaw, Jimmy C.; Lane, Clyde D., Jr.

2000-01-01

Survey responses from 62 of 96 cattle producers evaluated educational methods of the extension service and the livestock market. Methods included tips distributed with the sale check, monthly and sale day programs, and Second Saturday cattle working program. The combination of programs offered influenced them to make changes in their production…

Care management program evaluation: constituents, conflicts, and moves toward standardization.

PubMed

Long, D Adam; Perry, Theodore L; Pelletier, Kenneth R; Lehman, Gregg O

2006-06-01

Care management program evaluations bring together constituents from finance, medicine, and social sciences. The differing assumptions and scientific philosophies that these constituents bring to the task often lead to frustrations and even contentions. Given the forms and variations of care management programs, the difficulty associated with program outcomes measurement should not be surprising. It is no wonder then that methods for clinical and economic evaluations of program efficacy continue to be debated and have yet to be standardized. We describe these somewhat hidden processes, examine where the industry stands, and provide recommendations for steps to standardize evaluation methodology.

Motivation for Evaluation: A roadmap for Improving Program Efficacy

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D. F.; Welti, R.; Davis, H. B.

2016-12-01

Over the past year, the Incorporated Research Institutions for Seismology (IRIS) Education and Public Outreach (EPO) program has undertaken a new effort to increase the rigor with which it evaluates its programs and products. More specifically we sought to make evaluation an integral part of our EPO staff's work, enable staff to demonstrate why we do the activities we do, enhance the impact or our products and programs, and empower staff to be able to make evidence-based claims. The challenges we faced included a modest budget, finding an applicable approach to both new and legacy programs ranging from formal and informal education to public outreach, and implementing the process without overwhelming staff. The Collaborative Impact Analysis Method (IAM; Davis and Scalice, 2015) was selected as it allowed us to combine the EPO staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make incremental improvements to the evaluation of programs over time. We have found that this approach promotes the development of staff knowledge and skills regarding evaluation, provides a common language among staff, increases enthusiasm to collect and share data, encourages discussions of evaluative approaches when planning new activities, and improves each program's ability to capture the intended and unintended effects on the behaviors, attitudes, skills, interests, and/or knowledge of users/participants. We will share the initial IAM Scores for products and programs in the EPO portfolio, along with examples of the action plans for several key products and programs, and the impact that implementing those actions plans has had on our evaluations. Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis Method (Invited). Abstract ED53D-0871 presented at 2015 Fall Meeting, AGU, San Francisco, Calif., 14 - 18 Dec.

77 FR 38071 - Council on Graduate Medical Education; Notice of Meeting

Federal Register 2010, 2011, 2012, 2013, 2014

2012-06-26

... graduate medical education, evaluation of teaching programs especially in terms of meeting community needs...' development of performance measures and methods of longitudinal evaluation specific to the training programs...

Lessons from a Concurrent Evaluation of Eight Antibullying Programs Used in Sweden

ERIC Educational Resources Information Center

Flygare, Erik; Gill, Peter Edward; Johansson, Bjorn

2013-01-01

Sweden has a low prevalence of bullying and Swedish schools are legally obliged to have anti-bullying policies. Many commercial programs are available. A mixed methods, quasi-experimental, concurrent evaluation of 8 programs, chosen from a pool of 21 widely used anti-bullying programs, was planned. Preliminary data, based on 835 stakeholder…

42 CFR 52d.6 - Grant awards.

Code of Federal Regulations, 2013 CFR

2013-10-01

...) The extent to which the program would involve innovative teaching techniques; (3) The availability of... and teaching and is allowed to participate in the planning, organization, operation, and evaluation of... program; (10) The adequacy of the methods for monitoring and evaluating the program on a continuing basis...

42 CFR 52d.6 - Grant awards.

Code of Federal Regulations, 2012 CFR

2012-10-01

...) The extent to which the program would involve innovative teaching techniques; (3) The availability of... and teaching and is allowed to participate in the planning, organization, operation, and evaluation of... program; (10) The adequacy of the methods for monitoring and evaluating the program on a continuing basis...

42 CFR 52d.6 - Grant awards.

Code of Federal Regulations, 2014 CFR

2014-10-01

...) The extent to which the program would involve innovative teaching techniques; (3) The availability of... and teaching and is allowed to participate in the planning, organization, operation, and evaluation of... program; (10) The adequacy of the methods for monitoring and evaluating the program on a continuing basis...

Evaluation in the Context of the Government Market Place: Implications for the Evaluation of Research

ERIC Educational Resources Information Center

Della-Piana, Connie Kubo; Della-Piana, Gabriel M.

2007-01-01

While the current debate in the evaluation community has concentrated on examining and explicating implications of the choice of methods for evaluating federal programs, the authors of this paper address the challenges faced by the government in the selection of funding mechanisms for supporting program evaluation efforts. The choice of funding…

The Role for an Evaluator: A Fundamental Issue for Evaluation of Education and Social Programs

ERIC Educational Resources Information Center

Luo, Heng

2010-01-01

This paper discusses one of the fundamental issues in education and social program evaluation: the proper role for an evaluator. Based on respective and comparative analysis of five theorists' positions on this fundamental issue, this paper reveals how different perspectives on other fundamental issues in evaluation such as value, methods, use and…

Practice guidelines for program evaluation in community-based rehabilitation.

PubMed

Grandisson, Marie; Hébert, Michèle; Thibeault, Rachel

2017-06-01

This paper proposes practice guidelines to evaluate community-based rehabilitation (CBR) programs. These were developed through a rigorous three-phase research process including a literature review on good practices in CBR program evaluation, a field study during which a South Africa CBR program was evaluated, and a Delphi study to generate consensus among a highly credible panel of CBR experts from a wide range of backgrounds and geographical areas. The 10 guidelines developed are summarized into a practice model highlighting key features of sound CBR program evaluation. They strongly indicate that sound CBR evaluations are those that give a voice and as much control as possible to the most affected groups, embrace the challenge of diversity, and foster use of evaluation processes and findings through a rigorous, collaborative and empowering approach. The practice guidelines should facilitate CBR evaluation decisions in respect to facilitating an evaluation process, using frameworks and designing methods. Implications for rehabilitation Ten practice guidelines provide guidance to facilitate sound community-based rehabilitation (CBR) program evaluation decisions. Key indications of good practice include: • being as participatory and empowering as possible; • ensuring that all, including the most affected, have a real opportunity to share their thoughts; • highly considering mixed methods and participatory tools; • adapting to fit evaluation context, local culture and language(s); • defining evaluation questions and reporting findings using shared CBR language when possible, which the framework offered may facilitate.

New approach in the evaluation of a fitness program at a worksite.

PubMed

Shirasaya, K; Miyakawa, M; Yoshida, K; Tanaka, C; Shimada, N; Kondo, T

1999-03-01

The most common methods for the economic evaluation of a fitness program at a worksite are cost-effectiveness, cost-benefit, and cost-utility analyses. In this study, we applied a basic microeconomic theory, "neoclassical firm's problems," as the new approach for it. The optimal number of physical-exercise classes that constitute the core of the fitness program are determined using the cubic health production function. The optimal number is defined as the number that maximizes the profit of the program. The optimal number corresponding to any willingness-to-pay amount of the participants for the effectiveness of the program is presented using a graph. For example, if the willingness-to-pay is $800, the optimal number of classes is 23. Our method can be applied to the evaluation of any health care program if the health production function can be estimated.

Using a Mixed-Methods RE-AIM Framework to Evaluate Community Health Programs for Older Latinas.

PubMed

Schwingel, Andiara; Gálvez, Patricia; Linares, Deborah; Sebastião, Emerson

2017-06-01

This study used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework to evaluate a promotora-led community health program designed for Latinas ages 50 and older that sought to improve physical activity, nutrition, and stress management. A mixed-methods evaluation approach was administered at participant and organizational levels with a focus on the efficacy, adoption, implementation, and maintenance components of the RE-AIM theoretical model. The program was shown to be effective at improving participants' eating behaviors, increasing their physical activity levels, and lowering their depressive symptoms. Promotoras felt motivated and sufficiently prepared to deliver the program. Some implementation challenges were reported. More child care opportunities and an increased focus on mental well-being were suggested. The promotora delivery model has promise for program sustainability with both promotoras and participants alike expressing interest in leading future programs.

Applied Pluralism in the Evaluation of Employee Counselling.

ERIC Educational Resources Information Center

Goss, Stephen; Mearns, Dave

1997-01-01

Outlines the method, findings, and philosophical approach taken in a 22-month evaluation of an Employee Assistance Program. The program offered free counseling sessions, telephone support, in-service training, and conciliation work. Using an integrated pluralist evaluation, found that clients reported high satisfaction. Reduced absenteeism…

Dig into Learning: A Program Evaluation of an Agricultural Literacy Innovation

ERIC Educational Resources Information Center

Edwards, Erica Brown

2016-01-01

This study is a mixed-methods program evaluation of an agricultural literacy innovation in a local school district in rural eastern North Carolina. This evaluation describes the use of a theory-based framework, the Concerns-Based Adoption Model (CBAM), in accordance with Stufflebeam's Context, Input, Process, Product (CIPP) model by evaluating the…

Participatory Evaluation and Learning: A Case Example Involving Ripple Effects Mapping of a Tourism Assessment Program

ERIC Educational Resources Information Center

Bhattacharyya, Rani; Templin, Elizabeth; Messer, Cynthia; Chazdon, Scott

2017-01-01

Engaging communities through research-based participatory evaluation and learning methods can be rewarding for both a community and Extension. A case study of a community tourism development program evaluation shows how participatory evaluation and learning can be mutually reinforcing activities. Many communities value the opportunity to reflect…

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2013 CFR

2013-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2014 CFR

2014-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2012 CFR

2012-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2011 CFR

2011-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

Evaluation of a Sandplay Program for Preschoolers in a Multiethnic Neighborhood

ERIC Educational Resources Information Center

Rousseau, Cecile; Benoit, Maryse; Lacroix, Louise; Gauthier, Marie-France

2009-01-01

Background: This evaluative study assesses the efficacy of a school-based secondary prevention program consisting of creative expression workshops for immigrant and refugee preschoolers in a predominantly South Asian multiethnic neighborhood. Coincidentally, the program began in the wake of the tsunami. Method: Pretest and posttest data were…

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2010 CFR

2010-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... the reading ability of students in a program over time to a similar group of students not... example, a performance measure for a literacy program may include the percentage of students receiving...

Charting the Impact of Federal Spending for Education Research: A Bibliometric Approach

ERIC Educational Resources Information Center

Milesi, Carolina; Brown, Kevin L.; Hawkley, Louise; Dropkin, Eric; Schneider, Barbara L.

2014-01-01

Impact evaluation plays a critical role in determining whether federally funded research programs in science, technology, engineering, and mathematics are wise investments. This paper develops quantitative methods for program evaluation and applies this approach to a flagship National Science Foundation-funded education research program, Research…

75 FR 3642 - Special Evaluation Assistance for Rural Communities and Households Program

Federal Register 2010, 2011, 2012, 2013, 2014

2010-01-22

... the Special Evaluation Assistance for Rural Communities and Households (SEARCH) Program as authorized.... 1926 (a)(2)). The amendment added the new SEARCH grant program under which the Secretary is authorized... following methods: Federal eRulemaking Portal: Go to http://www.regulations.gov and in the ``Search...

Supporting School Leaders in Blended Learning with Blended Learning

ERIC Educational Resources Information Center

Acree, Lauren; Gibson, Theresa; Mangum, Nancy; Wolf, Mary Ann; Kellogg, Shaun; Branon, Suzanne

2017-01-01

This study provides a mixed-methods case-study design evaluation of the Leadership in Blended Learning (LBL) program. The LBL program uses blended approaches, including face-to-face and online, to prepare school leaders to implement blended learning initiatives in their schools. This evaluation found that the program designers effectively…

Review of Evaluation, Measurement and Verification Approaches Used to Estimate the Load Impacts and Effectiveness of Energy Efficiency Programs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Messenger, Mike; Bharvirkar, Ranjit; Golemboski, Bill

Public and private funding for end-use energy efficiency actions is expected to increase significantly in the United States over the next decade. For example, Barbose et al (2009) estimate that spending on ratepayer-funded energy efficiency programs in the U.S. could increase frommore » $3.1 billion in 2008 to $$7.5 and 12.4 billion by 2020 under their medium and high scenarios. This increase in spending could yield annual electric energy savings ranging from 0.58% - 0.93% of total U.S. retail sales in 2020, up from 0.34% of retail sales in 2008. Interest in and support for energy efficiency has broadened among national and state policymakers. Prominent examples include {approx}$$18 billion in new funding for energy efficiency programs (e.g., State Energy Program, Weatherization, and Energy Efficiency and Conservation Block Grants) in the 2009 American Recovery and Reinvestment Act (ARRA). Increased funding for energy efficiency should result in more benefits as well as more scrutiny of these results. As energy efficiency becomes a more prominent component of the U.S. national energy strategy and policies, assessing the effectiveness and energy saving impacts of energy efficiency programs is likely to become increasingly important for policymakers and private and public funders of efficiency actions. Thus, it is critical that evaluation, measurement, and verification (EM&V) is carried out effectively and efficiently, which implies that: (1) Effective program evaluation, measurement, and verification (EM&V) methodologies and tools are available to key stakeholders (e.g., regulatory agencies, program administrators, consumers, and evaluation consultants); and (2) Capacity (people and infrastructure resources) is available to conduct EM&V activities and report results in ways that support program improvement and provide data that reliably compares achieved results against goals and similar programs in other jurisdictions (benchmarking). The National Action Plan for Energy Efficiency (2007) presented commonly used definitions for EM&V in the context of energy efficiency programs: (1) Evaluation (E) - The performance of studies and activities aimed at determining the effects and effectiveness of EE programs; (2) Measurement and Verification (M&V) - Data collection, monitoring, and analysis associated with the calculation of gross energy and demand savings from individual measures, sites or projects. M&V can be a subset of program evaluation; and (3) Evaluation, Measurement, and Verification (EM&V) - This term is frequently seen in evaluation literature. EM&V is a catchall acronym for determining both the effectiveness of program designs and estimates of load impacts at the portfolio, program and project level. This report is a scoping study that assesses current practices and methods in the evaluation, measurement and verification (EM&V) of ratepayer-funded energy efficiency programs, with a focus on methods and practices currently used for determining whether projected (ex-ante) energy and demand savings have been achieved (ex-post). M&V practices for privately-funded energy efficiency projects (e.g., ESCO projects) or programs where the primary focus is greenhouse gas reductions were not part of the scope of this study. We identify and discuss key purposes and uses of current evaluations of end-use energy efficiency programs, methods used to evaluate these programs, processes used to determine those methods; and key issues that need to be addressed now and in the future, based on discussions with regulatory agencies, policymakers, program administrators, and evaluation practitioners in 14 states and national experts in the evaluation field. We also explore how EM&V may evolve in a future in which efficiency funding increases significantly, innovative mechanisms for rewarding program performance are adopted, the role of efficiency in greenhouse gas mitigation is more closely linked, and programs are increasingly funded from multiple sources often with multiple program administrators and intended to meet multiple purposes.« less

A Cost-Effectiveness Evaluation Approach to Improving Resource Allocations for School Systems. Administering for Change Program. A Professional Paper.

ERIC Educational Resources Information Center

Temkin, Sanford

This dissertation begins with a description of some methods employed in making public-sector resource-allocation decisions, with conclusions on the appropriateness of each method for evaluating the ongoing programs of a school system. The second section has been rewritten and published as "A Comprehensive Theory of Cost-Effectiveness" (EA 002…

A Cost-Effectiveness/Benefit Analysis Model for Postsecondary Vocational Programs. Technical Report.

ERIC Educational Resources Information Center

Kim, Jin Eun

A cost-effectiveness/benefit analysis is defined as a technique for measuring the outputs of existing and new programs in relation to their specified program objectives, against the costs of those programs. In terms of its specific use, the technique is conceptualized as a systems analysis method, an evaluation method, and a planning tool for…

Evaluating an Enrichment Program in Early Childhood: A Multi-Methods Approach

ERIC Educational Resources Information Center

van Aswegen, Christa; Pendergast, Donna

2015-01-01

This article reports on the evaluation of one topic in an enrichment program designed for children in their early years of learning. The program is responsive to an increased understanding of the benefits for very young children of programs that not only take advantage of the sensitive periods for learning but that also assist parents to a take a…

Goal Attainment Scaling: A Primary Method of Treatment and Program Evaluation in Project SAIL - A Special Education Dropout Prevention Program.

ERIC Educational Resources Information Center

Harris, Linda Hall; Thompson, John L.

The manual discusses Project SAIL's (a special dropout prevention program) use of Goal Attainment Scaling as part of individualized education plans in the treatment of troubled adolescents and in overall program evaluation. The scaling is characterized as an explicit, respectful treatment contact through which the adolescent can learn to set…

Planning an integrated agriculture and health program and designing its evaluation: Experience from Western Kenya.

PubMed

Cole, Donald C; Levin, Carol; Loechl, Cornelia; Thiele, Graham; Grant, Frederick; Girard, Aimee Webb; Sindi, Kirimi; Low, Jan

2016-06-01

Multi-sectoral programs that involve stakeholders in agriculture, nutrition and health care are essential for responding to nutrition problems such as vitamin A deficiency among pregnant and lactating women and their infants in many poor areas of lower income countries. Yet planning such multi-sectoral programs and designing appropriate evaluations, to respond to different disciplinary cultures of evidence, remain a challenge. We describe the context, program development process, and evaluation design of the Mama SASHA project (Sweetpotato Action for Security and Health in Africa) which promoted production and consumption of a bio-fortified, orange-fleshed sweetpotato (OFSP). In planning the program we drew upon information from needs assessments, stakeholder consultations, and a first round of the implementation evaluation of a pilot project. The multi-disciplinary team worked with partner organizations to develop a program theory of change and an impact pathway which identified aspects of the program that would be monitored and established evaluation methods. Responding to the growing demand for greater rigour in impact evaluations, we carried out quasi-experimental allocation by health facility catchment area, repeat village surveys for assessment of change in intervention and control areas, and longitudinal tracking of individual mother-child pairs. Mid-course corrections in program implementation were informed by program monitoring, regular feedback from implementers and partners' meetings. To assess economic efficiency and provide evidence for scaling we collected data on resources used and project expenses. Managing the multi-sectoral program and the mixed methods evaluation involved bargaining and trade-offs that were deemed essential to respond to the array of stakeholders, program funders and disciplines involved. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Planning an integrated agriculture and health program and designing its evaluation: Experience from Western Kenya

PubMed Central

Cole, Donald C.; Levin, Carol; Loechl, Cornelia; Thiele, Graham; Grant, Frederick; Girard, Aimee Webb; Sindi, Kirimi; Low, Jan

2016-01-01

Multi-sectoral programs that involve stakeholders in agriculture, nutrition and health care are essential for responding to nutrition problems such as vitamin A deficiency among pregnant and lactating women and their infants in many poor areas of lower income countries. Yet planning such multi-sectoral programs and designing appropriate evaluations, to respond to different disciplinary cultures of evidence, remain a challenge. We describe the context, program development process, and evaluation design of the Mama SASHA project (Sweetpotato Action for Security and Health in Africa) which promoted production and consumption of a bio-fortified, orange-fleshed sweetpotato (OFSP). In planning the program we drew upon information from needs assessments, stakeholder consultations, and a first round of the implementation evaluation of a pilot project. The multi-disciplinary team worked with partner organizations to develop a program theory of change and an impact pathway which identified aspects of the program that would be monitored and established evaluation methods. Responding to the growing demand for greater rigour in impact evaluations, we carried out quasi-experimental allocation by health facility catchment area, repeat village surveys for assessment of change in intervention and control areas, and longitudinal tracking of individual mother-child pairs. Mid-course corrections in program implementation were informed by program monitoring, regular feedback from implementers and partners’ meetings. To assess economic efficiency and provide evidence for scaling we collected data on resources used and project expenses. Managing the multi-sectoral program and the mixed methods evaluation involved bargaining and trade-offs that were deemed essential to respond to the array of stakeholders, program funders and disciplines involved. PMID:27003730

Bystander programs addressing sexual violence on college campuses: A systematic review and meta-analysis of program outcomes and delivery methods.

PubMed

Jouriles, Ernest N; Krauss, Alison; Vu, Nicole L; Banyard, Victoria L; McDonald, Renee

2018-02-06

This systematic review and meta-analysis evaluates the effectiveness of bystander programs that address sexual violence on college campuses. Program effects on student attitudes/beliefs and bystander behavior were examined. Durability of program outcomes and the influence of program-delivery methods (e.g., facilitator-led programs vs. video, online or poster campaign programs) and program-parameters (e.g., program length) were also evaluated. Twenty-four studies met criteria for inclusion in the meta-analysis, and 207 separate results from these studies were coded. Students who participated in a bystander program, compared to those who had not, had more pro-social attitudes/beliefs about sexual violence and intervening to prevent it, and engaged in more bystander behavior. Program effects diminished over time, but meaningful changes persisted for at least three months following program delivery. Longer programs had greater effects than shorter programs on attitudes/beliefs. Bystander programs can be a valuable addition to colleges' violence prevention efforts.

Evaluation of verifiability in HAL/S. [programming language for aerospace computers

NASA Technical Reports Server (NTRS)

Young, W. D.; Tripathi, A. R.; Good, D. I.; Browne, J. C.

1979-01-01

The ability of HAL/S to write verifiable programs, a characteristic which is highly desirable in aerospace applications, is lacking since many of the features of HAL/S do not lend themselves to existing verification techniques. The methods of language evaluation are described along with the means in which language features are evaluated for verifiability. These methods are applied in this study to various features of HAL/S to identify specific areas in which the language fails with respect to verifiability. Some conclusions are drawn for the design of programming languages for aerospace applications and ongoing work to identify a verifiable subset of HAL/S is described.

The opportunities and challenges of multi-site evaluations: lessons from the jail diversion and trauma recovery national cross-site evaluation.

PubMed

Stainbrook, Kristin; Penney, Darby; Elwyn, Laura

2015-06-01

Multi-site evaluations, particularly of federally funded service programs, pose a special set of challenges for program evaluation. Not only are there contextual differences related to project location, there are often relatively few programmatic requirements, which results in variations in program models, target populations and services. The Jail Diversion and Trauma Recovery-Priority to Veterans (JDTR) National Cross-Site Evaluation was tasked with conducting a multi-site evaluation of thirteen grantee programs that varied along multiple domains. This article describes the use of a mixed methods evaluation design to understand the jail diversion programs and client outcomes for veterans with trauma, mental health and/or substance use problems. We discuss the challenges encountered in evaluating diverse programs, the benefits of the evaluation in the face of these challenges, and offer lessons learned for other evaluators undertaking this type of evaluation. Copyright © 2015 Elsevier Ltd. All rights reserved.

FIELD ANALYTICAL SCREENING PROGRAM: PCP METHOD - INNOVATIVE TECHNOLOGY EVALUATION REPORT

EPA Science Inventory

The Field Analytical Screening Program (FASP) pentachlorophenol (PCP) method uses a gas chromatograph (GC) equipped with a megabore capillary column and flame ionization detector (FID) and electron capture detector (ECD) to identify and quantify PCP. The FASP PCP method is design...

ORE's GENeric Evaluation SYStem: GENESYS 1988-89.

ERIC Educational Resources Information Center

Baenen, Nancy; And Others

GENESYS--GENeric Evaluation SYStem--is a method of streamlining data collection and evaluation through the use of computer technology. GENESYS has allowed the Office of Research and Evaluation (ORE) of the Austin (Texas) Independent School District to evaluate a multitude of contrasting programs with limited resources. By standardizing methods and…

Development and application of course-embedded assessment system for program outcome evaluation in the Korean nursing education: A pilot study.

PubMed

Park, Jee Won; Seo, Eun Ji; You, Mi-Ae; Song, Ju-Eun

2016-03-01

Program outcome evaluation is important because it is an indicator for good quality of education. Course-embedded assessment is one of the program outcome evaluation methods. However, it is rarely used in Korean nursing education. The study purpose was to develop and apply preliminarily a course-embedded assessment system to evaluate one program outcome and to share our experiences. This was a methodological study to develop and apply the course-embedded assessment system based on the theoretical framework in one nursing program in South Korea. Scores for 77 students generated from the three practicum courses were used. The course-embedded assessment system was developed following the six steps suggested by Han's model as follows. 1) One program outcome in the undergraduate program, "nursing process application ability", was selected and 2) the three clinical practicum courses related to the selected program outcome were identified. 3) Evaluation tools including rubric and items were selected for outcome measurement and 4) performance criterion, the educational goal level for the program, was established. 5) Program outcome was actually evaluated using the rubric and evaluation items in the three practicum courses and 6) the obtained scores were analyzed to identify the achievement rate, which was compared with the performance criterion. Achievement rates for the selected program outcome in adult, maternity, and pediatric nursing practicum were 98.7%, 100%, and 66.2% in the case report and 100% for all three in the clinical practice, and 100%, 100%, and 87% respectively for the conference. These are considered as satisfactory levels when compared with the performance criterion of "at least 60% or more". Course-embedded assessment can be used as an effective and economic method to evaluate the program outcome without running an integrative course additionally. Further studies to develop course-embedded assessment systems for other program outcomes in nursing education are needed. Copyright © 2016 Elsevier Ltd. All rights reserved.

The Use of Mixed Methods in Randomized Control Trials

ERIC Educational Resources Information Center

White, Howard

2013-01-01

Evaluations should be issues driven, not methods driven. The starting point should be priority programs to be evaluated or policies to be tested. From this starting point, a list of evaluation questions is identified. For each evaluation question, the task is to identify the best available method for answering that question. Hence it is likely…

[Development of a program theory as a basis for the evaluation of a dementia special care unit].

PubMed

Adlbrecht, Laura; Bartholomeyczik, Sabine; Mayer, Hanna

2018-06-01

Background: An existing dementia special care unit should be evaluated. In order to build a sound foundation of the evaluation a deep theoretical understanding of the implemented intervention is needed, which has not been explicated yet. One possibility to achieve this is the development of a program theory. Aim: The aim is to present a method to develop a program theory for the existing living and care concept of the dementia special care unit, which is used in a larger project to evaluate the concept theory-drivenly. Method: The evaluation is embedded in the framework of van Belle et al. (2010) and an action model and a change model (Chen, 2015) is created. For the specification of the change model the contribution analysis (Mayne, 2011) is applied. Data were collected in workshops with the developers and the nurses of the dementia special care unit and a literature research concerning interventions and outcomes was carried out. The results were synthesized in a consens workshop. Results: The action model describes the interventions of the dementia special care unit, the implementers, the organization and the context. The change model compromises the mechanisms through which interventions achieve outcomes. Conclusions: The results of the program theory can be employed to choose data collection methods and instruments for the evaluation. On the basis of the results of the evaluation the program theory can be refined and adapted.

The Evaluation of Industry Practical of Mechanical Engineering in Vocational Education: A CIPP Model Approach

NASA Astrophysics Data System (ADS)

Kamaludin, M.; Munawar, W.; Mahdan, D.; Simanjuntak, M. V.; Wendi, H. F.

2018-02-01

The learning system is not only studied on campus but also practicing in the world of work. Industry Practical aims to enable students to develop their skills in accordance with the real world of work. To know the success of the implementation of industry practical program then held evaluation. The evaluation of the program in this study used the CIPP evaluation approach (Context, Input, Process, Product). The purpose of this research is to know the extent of achievement and success of industry practical program at vocational school in Bandung with descriptive research method using mix method approach. The sample in this research is students majoring in mechanical engineering in the city of Bandung who have done industry practical.

Chapter 17: Residential Behavior Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W.; Stewart, James; Todd, Annika

Residential behavior-based (BB) programs use strategies grounded in the behavioral and social sciences to influence household energy use. These may include providing households with real-time or delayed feedback about their energy use; supplying energy efficiency education and tips; rewarding households for reducing their energy use; comparing households to their peers; and establishing games, tournaments, and competitions. BB programs often target multiple energy end uses and encourage energy savings, demand savings, or both. Savings from BB programs are usually a small percentage of energy use, typically less than 5 percent. Utilities will continue to implement residential BB programs as large-scale, randomizedmore » control trials (RCTs); however, some are now experimenting with alternative program designs that are smaller scale; involve new communication channels such as the web, social media, and text messaging; or that employ novel strategies for encouraging behavior change (for example, Facebook competitions). These programs will create new evaluation challenges and may require different evaluation methods than those currently employed to verify any savings they generate. Quasi-experimental methods, however, require stronger assumptions to yield valid savings estimates and may not measure savings with the same degree of validity and accuracy as randomized experiments.« less

Exploring Robust Methods for Evaluating Treatment and Comparison Groups in Chronic Care Management Programs

PubMed Central

Hamar, Brent; Bradley, Chastity; Gandy, William M.; Harrison, Patricia L.; Sidney, James A.; Coberley, Carter R.; Rula, Elizabeth Y.; Pope, James E.

2013-01-01

Abstract Evaluation of chronic care management (CCM) programs is necessary to determine the behavioral, clinical, and financial value of the programs. Financial outcomes of members who are exposed to interventions (treatment group) typically are compared to those not exposed (comparison group) in a quasi-experimental study design. However, because member assignment is not randomized, outcomes reported from these designs may be biased or inefficient if study groups are not comparable or balanced prior to analysis. Two matching techniques used to achieve balanced groups are Propensity Score Matching (PSM) and Coarsened Exact Matching (CEM). Unlike PSM, CEM has been shown to yield estimates of causal (program) effects that are lowest in variance and bias for any given sample size. The objective of this case study was to provide a comprehensive comparison of these 2 matching methods within an evaluation of a CCM program administered to a large health plan during a 2-year time period. Descriptive and statistical methods were used to assess the level of balance between comparison and treatment members pre matching. Compared with PSM, CEM retained more members, achieved better balance between matched members, and resulted in a statistically insignificant Wald test statistic for group aggregation. In terms of program performance, the results showed an overall higher medical cost savings among treatment members matched using CEM compared with those matched using PSM (-$25.57 versus -$19.78, respectively). Collectively, the results suggest CEM is a viable alternative, if not the most appropriate matching method, to apply when evaluating CCM program performance. (Population Health Management 2013;16:35–45) PMID:22788834

Exploring robust methods for evaluating treatment and comparison groups in chronic care management programs.

PubMed

Wells, Aaron R; Hamar, Brent; Bradley, Chastity; Gandy, William M; Harrison, Patricia L; Sidney, James A; Coberley, Carter R; Rula, Elizabeth Y; Pope, James E

2013-02-01

Evaluation of chronic care management (CCM) programs is necessary to determine the behavioral, clinical, and financial value of the programs. Financial outcomes of members who are exposed to interventions (treatment group) typically are compared to those not exposed (comparison group) in a quasi-experimental study design. However, because member assignment is not randomized, outcomes reported from these designs may be biased or inefficient if study groups are not comparable or balanced prior to analysis. Two matching techniques used to achieve balanced groups are Propensity Score Matching (PSM) and Coarsened Exact Matching (CEM). Unlike PSM, CEM has been shown to yield estimates of causal (program) effects that are lowest in variance and bias for any given sample size. The objective of this case study was to provide a comprehensive comparison of these 2 matching methods within an evaluation of a CCM program administered to a large health plan during a 2-year time period. Descriptive and statistical methods were used to assess the level of balance between comparison and treatment members pre matching. Compared with PSM, CEM retained more members, achieved better balance between matched members, and resulted in a statistically insignificant Wald test statistic for group aggregation. In terms of program performance, the results showed an overall higher medical cost savings among treatment members matched using CEM compared with those matched using PSM (-$25.57 versus -$19.78, respectively). Collectively, the results suggest CEM is a viable alternative, if not the most appropriate matching method, to apply when evaluating CCM program performance.

Integrating quantitative and qualitative evaluation methods to compare two teacher inservice training programs

NASA Astrophysics Data System (ADS)

Lawrenz, Frances; McCreath, Heather

Qualitative and quantitative evaluation procedures were used to compare two physical-science teacher inservice training programs. The two programs followed the master teacher training model espoused by NSF but used different types of master teachers and types of activities. The two evaluation procedures produced different results and together they provided a much clearer picture of the strengths and weaknesses of the two programs. Using only one approach or the other would have substantially altered the conclusions.

Process Evaluation of a Workers' Health Surveillance Program for Meat Processing Workers.

PubMed

van Holland, Berry J; Brouwer, Sandra; de Boer, Michiel R; Reneman, Michiel F; Soer, Remko

2017-09-01

Objective To evaluate the implementation process of a workers' health surveillance (WHS) program in a Dutch meat processing company. Methods Workers from five plants were eligible to participate in the WHS program. The program consisted of four evaluative components and an intervention component. Qualitative and quantitative methods were used to evaluate seven process aspects. Data were gathered by interviews with stakeholders, participant questionnaires, and from registries of the company and occupational health service. Results Two recruitment strategies were used: open invitation or automatic participation. Of the 986 eligible workers, 305 participated in the program. Average reach was 53 %. Two out of five program components could not be assessed on dose delivered, dose received and fidelity. If components were assessable, 85-100 % of the components was delivered, 66-100 % of the components was received by participants, and fidelity was 100 %. Participants were satisfied with the WHS program (mean score 7.6). Contextual factors that facilitated implementation were among others societal developments and management support. Factors that formed barriers were program novelty and delayed follow-up. Conclusion The WHS program was well received by participants. Not all participants were offered the same number of program components, and not all components were performed according to protocol. Deviation from protocol is an indication of program failure and may affect program effectiveness.

Physiotherapists supporting self-management through health coaching: a mixed methods program evaluation.

PubMed

Dufour, Sinéad Patricia; Graham, Shane; Friesen, Josh; Rosenblat, Michael; Rous, Colin; Richardson, Julie

2015-01-01

To evaluate a program in support of chronic disease self-management (CDSM) that is founded on a health coaching (HC) approach, includes supervised exercise and mindfulness-based stress reduction components and is delivered within a private practice physiotherapy setting. An explanatory mixed method design, framed by theory-based program evaluation, was employed to evaluate an eight-week group-based program. Standardized self-rated and performance measures were evaluated pre- and post intervention. Additionally, participant focus groups were conducted following the intervention period. An inductive thematic approach was undertaken to analyze the qualitative data. Seventeen participants (N = 17) completed the study. Improvements were seen in both self-report and performance outcomes. Participants explained how and why they felt the program was beneficial. Six themes were generated: (1) group dynamic; (2) learning versus doing; (3) holism and comprehensive care; (4) self-efficacy and empowerment; (5) previous solutions versus new management strategies; and (6) healthcare provider support. This study established that a group program in support of CDSM founded on a HC approach demonstrated potential value from participants as well as favorable outcomes. A pragmatic randomized control trial is required to determine efficacy of this intervention.

A Participatory Action Research Approach To Evaluating Inclusive School Programs.

ERIC Educational Resources Information Center

Dymond, Stacy K.

2001-01-01

This article proposes a model for evaluating inclusive schools. Key elements of the model are inclusion of stakeholders in the evaluation process through a participatory action research approach, analysis of program processes and outcomes, use of multiple methods and measures, and obtaining perceptions from diverse stakeholder groups. (Contains…

The Efficacy of Stuttering Measurement Training: Evaluating Two Training Programs

ERIC Educational Resources Information Center

Bainbridge, Lauren A.; Stavros, Candace; Ebrahimian, Mineh; Wang, Yuedong; Ingham, Roger J.

2015-01-01

Purpose: Two stuttering measurement training programs currently used for training clinicians were evaluated for their efficacy in improving the accuracy of total stuttering event counting. Method: Four groups, each with 12 randomly allocated participants, completed a pretest-posttest design training study. They were evaluated by their counts of…

Learning for All: The Value of Field Experience in Training a New Generation of Program Evaluators

ERIC Educational Resources Information Center

Buitrag, Carolina

2015-01-01

Professionals in various fields have the opportunity to train in real-life situations before they take on a job by themselves. In program evaluation, however, preservice training usually focuses on coursework oriented to research design and methods. After completing these studies, evaluators go off to conduct evaluations guided primarily by their…

Evaluation of Adult Literacy Education in the United States: A Review of Methodological Issues

ERIC Educational Resources Information Center

Shi, Yan; Tsang, Mun C.

2008-01-01

This is a critical review of methodological issues in the evaluation of adult literacy education programs in the United States. It addresses the key research questions: What are the appropriate methods for evaluating these programs under given circumstances. It identifies 15 evaluation studies that are representative of a range of adult literacy…

Evaluating portfolio use as a tool for assessment and professional development in graduate nursing education.

PubMed

Ryan, Marybeth

2011-01-01

The portfolio is emerging as an efficient and effective method for evaluating program outcomes and professional development in nursing education. Although there is a host of literature about the use of portfolios in undergraduate nursing programs, fewer reports exist about their use in graduate nursing education. This article presents the results of a formative evaluation process, using student and faculty focus groups, conducted at a midsized university's graduate nursing education program to determine the effectiveness of portfolio use. Content analysis of the focus group data yielded three student themes and two faculty themes with associated theme clusters that revealed similarities and unique perceptions of students and faculty regarding the portfolio process. The information gleaned will provide direction to faculty as they make decisions about the use of this evaluation method in the graduate program. Copyright © 2011. Published by Elsevier Inc.

Postadoption and Guardianship: An Evaluation of the Adoption Preservation, Assessment, and Linkage Program

ERIC Educational Resources Information Center

Liao, Minli; Testa, Mark

2016-01-01

Objectives: This study evaluated the effects of the Adoption Preservation, Assessment, and Linkage (APAL) postpermanency program. Method: A quasi-experimental, posttest-only design was used to estimate the program's effects on youth discharged from foster care to adoption or legal guardianship. A random sample was surveyed (female = 44.7%; African…

Cost Savings Threshold Analysis of a Capacity-Building Program for HIV Prevention Organizations

ERIC Educational Resources Information Center

Dauner, Kim Nichols; Oglesby, Willie H.; Richter, Donna L.; LaRose, Christopher M.; Holtgrave, David R.

2008-01-01

Although the incidence of HIV each year remains steady, prevention funding is increasingly competitive. Programs need to justify costs in terms of evaluation outcomes, including economic ones. Threshold analyses set performance standards to determine program effectiveness relative to that threshold. This method was used to evaluate the potential…

A Text-Computer Assisted Instruction Program as a Viable Alternative for Continuing Education in Laboratory Medicine.

ERIC Educational Resources Information Center

Bruce, A. Wayne

1986-01-01

Describes reasons for developing combined text and computer assisted instruction (CAI) teaching programs for delivery of continuing education to laboratory professionals, and mechanisms used for developing a CAI program on method evaluation in the clinical laboratory. Results of an evaluation of the software's cost effectiveness and instructional…

Multidimensional Implementation Evaluation of a Residential Treatment Program for Adolescent Substance Abuse

ERIC Educational Resources Information Center

Faw, Leyla; Hogue, Aaron; Liddle, Howard A.

2005-01-01

The authors applied contemporary methods from the evaluation literature to measure implementation in a residential treatment program for adolescent substance abuse. A logic model containing two main components was measured. Program structure (adherence to the intended framework of service delivery) was measured using data from daily activity logs…

Baylor SBIRT Medical Residency Training Program: Model Description and Initial Evaluation

ERIC Educational Resources Information Center

Bray, James H.; Kowalchuk, Alicia; Waters, Vicki; Laufman, Larry; Shilling, Elizabeth H.

2012-01-01

The Baylor College of Medicine SBIRT Medical Residency Training Program is a multilevel project that trains residents and faculty in evidenced-based screening, brief intervention, and referral to treatment (SBIRT) methods for alcohol and substance use problems. This paper describes the training program and provides initial evaluation after the…

Outcome Evaluation of a Community Center-Based Program for Mothers at High Psychosocial Risk

ERIC Educational Resources Information Center

Rodrigo, Maria Jose; Maiquez, Maria Luisa; Correa, Ana Delia; Martin, Juan Carlos; Rodriguez, Guacimara

2006-01-01

Objective: This study reported the outcome evaluation of the "Apoyo Personal y Familiar" (APF) program for poorly-educated mothers from multi-problem families, showing inadequate behavior with their children. APF is a community-based multi-site program delivered through weekly group meetings in municipal resource centers. Method: A total…

Controlled Multivariate Evaluation of Open Education: Application of a Critical Model.

ERIC Educational Resources Information Center

Sewell, Alan F.; And Others

This paper continues previous reports of a controlled multivariate evaluation of a junior high school open-education program. A new method of estimating program objectives and implementation is presented, together with the nature and degree of obtained student outcomes. Open-program students were found to approve more highly of their learning…

Multiple-Methods Needs Assessment of California 4-H Science Education Programming

ERIC Educational Resources Information Center

Worker, Steven M.; Schmitt-McQuitty, Lynn; Ambrose, Andrea; Brian, Kelley; Schoenfelder, Emily; Smith, Martin H.

2017-01-01

The California 4-H Science Leadership Team conducted a statewide assessment to evaluate the needs of county-based 4-H programs related to the key areas of the 4-H Science Initiative: program development and design, professional development, curricula, evaluation, partnerships, and fund development. The use of multiple qualitative data sources…

Evaluating Predictors of Program Attrition among Women Mandated into Batterer Intervention Treatment

ERIC Educational Resources Information Center

Buttell, Frederick P.; Powers, Dolores; Wong, Asia

2012-01-01

Objective: The purpose of the present study was to evaluate pretreatment differences between treatment completers and dropouts among a large sample of women ordered into a 26-week batterer intervention program (BIP). Method: The study employed a nonequivalent, control-group design (comparing program completers to dropouts) in a secondary analysis…

Evaluation of a Postdischarge Call System Using the Logic Model.

PubMed

Frye, Timothy C; Poe, Terri L; Wilson, Marisa L; Milligan, Gary

2018-02-01

This mixed-method study was conducted to evaluate a postdischarge call program for congestive heart failure patients at a major teaching hospital in the southeastern United States. The program was implemented based on the premise that it would improve patient outcomes and overall quality of life, but it had never been evaluated for effectiveness. The Logic Model was used to evaluate the input of key staff members to determine whether the outputs and results of the program matched the expectations of the organization. Interviews, online surveys, reviews of existing patient outcome data, and reviews of publicly available program marketing materials were used to ascertain current program output. After analyzing both qualitative and quantitative data from the evaluation, recommendations were made to the organization to improve the effectiveness of the program.

Rational drug therapy education in clinical phase carried out by task-based learning

PubMed Central

Bilge, S. Sırrı; Akyüz, Bahar; Ağrı, Arzu Erdal; Özlem, Mıdık

2017-01-01

Objectives: Irrational drug use results in drug interactions, treatment noncompliance, and drug resistance. Rational pharmacotherapy education is being implemented in many faculties of medicine. Our aim is to introduce rational pharmacotherapy education by clinicians and to evaluate task-based rational drug therapy education in the clinical context. Methods: The Kirkpatrick's evaluation model was used for the evaluation of the program. The participants evaluated the program in terms of constituents of the program, utilization, and contribution to learning. Voluntary participants responded to the evaluation forms after the educational program. Data are evaluated using both quantitative and qualitative tools. SPSS (version 21) used for quantitative data for determining mean and standard deviation values. Descriptive qualitative analysis approach is used for the analysis of open-ended questions. Results: It was revealed that the program and its components have been favorable. A total 95.9% of the students consider the education to be beneficial. Simulated patients practice and personal drug choice/problem-based learning sessions were appreciated by the students in particular. 93.9% of the students stated that all students of medicine should undergo this educational program. Among the five presentations contained in the program, “The Principles of Prescribing” received the highest points (9 ± 1.00) from participating students in general evaluation of the educational program. Conclusion: This study was carried out to improve task-based rational drug therapy education. According to feedback from the students concerning content, method, resource, assessment, and program design; some important changes, especially in number of facilitators and indications, are made in rational pharmacotherapy education in clinical task-based learning program. PMID:28458432

Evaluation Methods Used during the Assessment of an Academic Program at Micro-Level

ERIC Educational Resources Information Center

Chinembiri, France

2017-01-01

This paper majors on the evaluation criteria and methods used for the assessment of health education programs in the United States. The choice of the topic is dictated by the fact that there is need to improve on the quality of the graduates that the different nursing and medical institutions in the United States produce. By looking at the…

A pilot study of Aboriginal health promotion from an ecological perspective

PubMed Central

2011-01-01

Background For health promotion to be effective in Aboriginal and Torres Strait Islander Communities, interventions (and their evaluation) need to work within a complex social environment and respect Indigenous knowledge, culture and social systems. At present, there is a lack of culturally appropriate evaluation methods available to practitioners that are capable of capturing this complexity. As an initial response to this problem, we used two non-invasive methods to evaluate a community-directed health promotion program, which aimed to improve nutrition and physical activity for members of the Aboriginal community of the Goulburn-Murray region of northern Victoria, Australia. The study addressed two main questions. First, for members of an Aboriginal sporting club, what changes were made to the nutrition environment in which they meet and how is this related to national guidelines for minimising the risk of chronic disease? Second, to what degree was the overall health promotion program aligned with an ecological model of health promotion that addresses physical, social and policy environments as well as individual knowledge and behaviour? Methods Rather than monitoring individual outcomes, evaluation methods reported on here assessed change in the nutrition environment (sports club food supply) as a facilitator of dietary change and the 'ecological' nature of the overall program (that is, its complexity with respect to numbers of targets, settings and strategies). Results There were favourable changes towards the provision of a food supply consistent with Australian guidelines at the sports club. The ecological analysis indicated that the design and implementation of the program were consistent with an ecological model of health promotion. Conclusions The evaluation was useful for assessing the impact of the program on the nutrition environment and for understanding the ecological nature of program activities. PMID:21961906

Analyzing Empirical Evaluations of Non-Experimental Methods in Field Settings

ERIC Educational Resources Information Center

Steiner, Peter M.; Wong, Vivian

2016-01-01

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

FIELD ANALYTICAL SCREENING PROGRAM: PCB METHOD - INNOVATIVE TECHNOLOGY REPORT

EPA Science Inventory

This innovative technology evaluation report (ITER) presents information on the demonstration of the U.S. Environmental Protection Agency (EPA) Region 7 Superfund Field Analytical Screening Program (FASP) method for determining polychlorinated biphenyl (PCB) contamination in soil...

Engaging partners to initiate evaluation efforts: tactics used and lessons learned from the prevention research centers program.

PubMed

Wright, Demia Sundra; Anderson, Lynda A; Brownson, Ross C; Gwaltney, Margaret K; Scherer, Jennifer; Cross, Alan W; Goodman, Robert M; Schwartz, Randy; Sims, Tom; White, Carol R

2008-01-01

The Centers for Disease Control and Prevention's (CDC's) Prevention Research Centers (PRC) Program underwent a 2-year evaluation planning project using a participatory process that allowed perspectives from the national community of PRC partners to be expressed and reflected in a national logic model. The PRC Program recognized the challenge in developing a feasible, useable, and relevant evaluation process for a large, diverse program. To address the challenge, participatory and utilization-focused evaluation models were used. Four tactics guided the evaluation planning process: 1) assessing stakeholders' communication needs and existing communication mechanisms and infrastructure; 2) using existing mechanisms and establishing others as needed to inform, educate, and request feedback; 3) listening to and using feedback received; and 4) obtaining adequate resources and building flexibility into the project plan to support multifaceted mechanisms for data collection. Participatory methods resulted in buy-in from stakeholders and the development of a national logic model. Benefits included CDC's use of the logic model for program planning and development of a national evaluation protocol and increased expectations among PRC partners for involvement. Challenges included the time, effort, and investment of program resources required for the participatory approach and the identification of whom to engage and when to engage them for feedback on project decisions. By using a participatory and utilization-focused model, program partners positively influenced how CDC developed an evaluation plan. The tactics we used can guide the involvement of program stakeholders and help with decisions on appropriate methods and approaches for engaging partners.

Are We There Yet? Evaluating Library Collections, Reference Services, Programs, and Personnel.

ERIC Educational Resources Information Center

Robbins-Carter, Jane; Zweizig, Douglas L.

1985-01-01

This second in a five-lesson tutorial on library evaluation focuses on the evaluation of library collections. Highlights include the seven-step evaluation process described in lesson one; quantitative methods (total size, unfilled requests, circulation, turnover rate); and qualitative methods (impressionistic, list-checking). One required and…

Collaborative Evaluation of a High School Prevention Curriculum: How Methods of Collaborative Evaluation Enhanced a Randomized Control Trial to Inform Program Improvement

ERIC Educational Resources Information Center

Orsini, Muhsin Michael; Wyrick, David L.; Milroy, Jeffrey J.

2012-01-01

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome…

Indigenous health program evaluation design and methods in Australia: a systematic review of the evidence.

PubMed

Lokuge, Kamalini; Thurber, Katherine; Calabria, Bianca; Davis, Meg; McMahon, Kathryn; Sartor, Lauren; Lovett, Raymond; Guthrie, Jill; Banks, Emily

2017-10-01

Indigenous Australians experience a disproportionately higher burden of disease compared to non-Indigenous Australians. High-quality evaluation of Indigenous health programs is required to inform health and health services improvement. We aimed to quantify methodological and other characteristics of Australian Indigenous health program evaluations published in the peer-reviewed literature. Systematic review of peer-reviewed literature (November 2009-2014) on Indigenous health program evaluation. We identified 118 papers describing evaluations of 109 interventions; 72.0% were university/research institution-led. 82.2% of evaluations included a quantitative component; 49.2% utilised quantitative data only and 33.1% used both quantitative and qualitative data. The most common design was a before/after comparison (30.5%, n=36/118). 7.6% of studies (n=9/118) used an experimental design: six individual-level and three cluster-randomised controlled trials. 56.8% (67/118) reported on service delivery/process outcomes (versus health or health risk factor outcomes) only. Given the number of Indigenous health programs that are implemented, few evaluations overall are published in the peer-reviewed literature and, of these, few use optimal methodologies such as mixed methods and experimental design. Implications for public health: Multiple strategies are required to increase high-quality, accessible evaluation in Indigenous health, including supporting stronger research-policy-practice partnerships and capacity building for evaluation by health services and government. © 2017 The Authors.

A novel resident-as-teacher training program to improve and evaluate obstetrics and gynecology resident teaching skills.

PubMed

Ricciotti, Hope A; Dodge, Laura E; Head, Julia; Atkins, K Meredith; Hacker, Michele R

2012-01-01

Residents play a significant role in teaching, but formal training, feedback, and evaluation are needed. Our aims were to assess resident teaching skills in the resident-as-teacher program, quantify correlations of faculty evaluations with resident self-evaluations, compare resident-as-teacher evaluations with clinical evaluations, and evaluate the resident-as-teacher program. The resident-as-teacher training program is a simulated, videotaped teaching encounter with a trained medical student and standardized teaching evaluation tool. Evaluations from the resident-as-teacher training program were compared to evaluations of resident teaching done by faculty, residents, and medical students from the clinical setting. Faculty evaluation of resident teaching skills in the resident-as-teacher program showed a mean total score of 4.5 ± 0.5 with statistically significant correlations between faculty assessment and resident self-evaluations (r = 0.47; p < 0.001). However, resident self-evaluation of teaching skill was lower than faculty evaluation (mean difference: 0.4; 95% CI 0.3-0.6). When compared to the clinical setting, resident-as-teacher evaluations were significantly correlated with faculty and resident evaluations, but not medical student evaluations. Evaluations from both the resident-as-teacher program and the clinical setting improved with duration of residency. The resident-as-teacher program provides a method to train, give feedback, and evaluate resident teaching.

EVALUATING PROGRAMMED TEST INTERPRETATION USING EMOTIONAL AROUSAL AS A CRITERION.

ERIC Educational Resources Information Center

FORSTER, JERALD R.

THE QUESTION AS TO WHETHER OR NOT THE EMOTIONAL ASPECTS OF TEST INTERPRETATION MAKE IT INAPPROPRIATE FOR PROGRAMED METHODS WAS EXAMINED BY TWO METHODS OF TEST RESULT COMMUNICATION--PROGRAMED MATERIALS AND VERBAL COMMUNICATION BY A COUNSELOR. DEPENDENT VARIABLES WERE MEASURES OF EMOTIONAL AROUSAL RECORDED BY SKIN CONDUCTANCE UNITS AND THE GAIN IN…

SRM Manual, Selected Resource Materials: Description and Evaluation.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Special Educational Services Branch.

Provided in the manual is descriptive and evaluative information on 67 instructional materials or programs frequently used with exceptional children in Alberta schools. A flow chart describes the three methods of using the manual: l) to obtain descriptive and evaluative information of an instruction program in the manual 2) to classify and record…

Does the Kids Cafe Program's nutrition education improve children's dietary intake? A pilot evaluation study

USDA-ARS?s Scientific Manuscript database

Our objective was to evaluate the Kids Café Program (KCP) nutrition education intervention and assess its impact on children's diet quality and body mass index (BMI) percentile. An experimental design consisting of pretest-posttest comparison groups using mixed methods was used to evaluate the 6-ses...

Sexuality Education: An Evaluation of Programs and Their Effects.

ERIC Educational Resources Information Center

Kirby, Douglas

This document is the first volume of a six-volume report on sexuality education. This volume summarizes the structure and content of sexuality education in the United States, reviews the literature on the effects of sexuality education, describes the evaluation methods, provides a description of and the evaluation data for each program, and…

Evaluation Techniques for the Sandy Point Discovery Center, Great Bay National Estuarine Research Reserve.

ERIC Educational Resources Information Center

Heffernan, Bernadette M.

1998-01-01

Describes work done to provide staff of the Sandy Point Discovery Center with methods for evaluating exhibits and interpretive programming. Quantitative and qualitative evaluation measures were designed to assess the program's objective of estuary education. Pretest-posttest questionnaires and interviews are used to measure subjects' knowledge and…

"Evaluation as Illumination: A New Approach to the Study of Innovatory Programs". Occasional Paper.

ERIC Educational Resources Information Center

Parlett, Malcolm; Hamilton, David

Conventional approaches to program evaluation have followed the experimental and psychometric traditions dominant in educational research. Their aim (unfulfilled) of achieving fully objective methods had led to studies that are artificial and restricted in scope. Illuminative evaluation is introduced as a total re-appraisal of the rationale and…

Improving Human Services: Methods for Systematic Planning, Evaluation, and Staff Development.

ERIC Educational Resources Information Center

Jones, G. Brian; And Others

This monograph speaks to the basic problems confronting human services programs, and offers suggestions for improving guidance services. Specific concerns addressed are: (1) the need for more systematic program planning and evaluation; (2) an approach to effective planning and evaluation; (3) development of staff skills for use in the…

Critical evaluation of international health programs: Reframing global health and evaluation.

PubMed

Chi, Chunhuei; Tuepker, Anaïs; Schoon, Rebecca; Núñez Mondaca, Alicia

2018-04-01

Striking changes in the funding and implementation of international health programs in recent decades have stimulated debate about the role of communities in deciding which health programs to implement. An important yet neglected piece of that discussion is the need to change norms in program evaluation so that analysis of community ownership, beyond various degrees of "participation," is seen as central to strong evaluation practices. This article challenges mainstream evaluation practices and proposes a framework of Critical Evaluation with 3 levels: upstream evaluation assessing the "who" and "how" of programming decisions; midstream evaluation focusing on the "who" and "how" of selecting program objectives; and downstream evaluation, the focus of current mainstream evaluation, which assesses whether the program achieved its stated objectives. A vital tenet of our framework is that a community possesses the right to determine the path of its health development. A prerequisite of success, regardless of technical outcomes, is that programs must address communities' high priority concerns. Current participatory methods still seldom practice community ownership of program selection because they are vulnerable to funding agencies' predetermined priorities. In addition to critiquing evaluation practices and proposing an alternative framework, we acknowledge likely challenges and propose directions for future research. Copyright © 2018 John Wiley & Sons, Ltd.

Evaluating disease management program effectiveness: an introduction to survival analysis.

PubMed

Linden, Ariel; Adams, John L; Roberts, Nancy

2004-01-01

Currently, the most widely used method in the disease management industry for evaluating program effectiveness is the "total population approach." This model is a pretest-posttest design, with the most basic limitation being that without a control group, there may be sources of bias and/or competing extraneous confounding factors that offer plausible rationale explaining the change from baseline. Survival analysis allows for the inclusion of data from censored cases, those subjects who either "survived" the program without experiencing the event (e.g., achievement of target clinical levels, hospitalization) or left the program prematurely, due to disenrollement from the health plan or program, or were lost to follow-up. Additionally, independent variables may be included in the model to help explain the variability in the outcome measure. In order to maximize the potential of this statistical method, validity of the model and research design must be assured. This paper reviews survival analysis as an alternative, and more appropriate, approach to evaluating DM program effectiveness than the current total population approach.

Evaluation of a Classroom Program of Creative Expression Workshops for Refugee and Immigrant Children

ERIC Educational Resources Information Center

Rousseau, Cecile; Drapeau, Aline; Lacroix, Louise; Bagilishya, Deogratias; Heusch, Nicole

2005-01-01

Objective: This evaluative study assessed the effect of a creative expression program designed to prevent emotional and behavioral problems and to enhance self-esteem in immigrant and refugee children attending multiethnic schools. Method: The 12-week program involved 138 children, aged 7 to 13, registered in both integration classes designed for…

Self-Evaluation Handbook for Hotlines and Youth Crisis Centers.

ERIC Educational Resources Information Center

Baizerman, Michael; And Others

This handbook is a response to Hotline and Youth Crisis Center staff who asked for a guidebook for doing program evaluation. A cross-section of Hotline workers in three Minnesota centers was monitored to define issues and problems or concerns about their programs. The actual program methods used were determined and utilized to formalize a method…

Implementation and Evaluation of a Smartphone-Based Telemonitoring Program for Patients With Heart Failure: Mixed-Methods Study Protocol.

PubMed

Ware, Patrick; Ross, Heather J; Cafazzo, Joseph A; Laporte, Audrey; Seto, Emily

2018-05-03

Meta-analyses of telemonitoring for patients with heart failure conclude that it can lower the utilization of health services and improve health outcomes compared with the standard of care. A smartphone-based telemonitoring program is being implemented as part of the standard of care at a specialty care clinic for patients with heart failure in Toronto, Canada. The objectives of this study are to (1) evaluate the impact of the telemonitoring program on health service utilization, patient health outcomes, and their ability to self-care; (2) identify the contextual barriers and facilitators of implementation at the physician, clinic, and institutional level; (3) describe patient usage patterns to determine adherence and other behaviors in the telemonitoring program; and (4) evaluate the costs associated with implementation of the telemonitoring program from the perspective of the health care system (ie, public payer), hospital, and patient. The evaluation will use a mixed-methods approach. The quantitative component will include a pragmatic pre- and posttest study design for the impact and cost analyses, which will make use of clinical data and questionnaires administered to at least 108 patients at baseline and 6 months. Furthermore, outcome data will be collected at 1, 12, and 24 months to explore the longitudinal impact of the program. In addition, quantitative data related to implementation outcomes and patient usage patterns of the telemonitoring system will be reported. The qualitative component involves an embedded single case study design to identify the contextual factors that influenced the implementation. The implementation evaluation will be completed using semistructured interviews with clinicians, and other program staff at baseline, 4 months, and 12 months after the program start date. Interviews conducted with patients will be triangulated with usage data to explain usage patterns and adherence to the system. The telemonitoring program was launched in August 2016 and patient enrollment is ongoing. The methods described provide an example for conducting comprehensive evaluations of telemonitoring programs. The combination of impact, implementation, and cost evaluations will inform the quality improvement of the existing program and will yield insights into the sustainability of smartphone-based telemonitoring programs for patients with heart failure within a specialty care setting. ©Patrick Ware, Heather J Ross, Joseph A Cafazzo, Audrey Laporte, Emily Seto. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 03.05.2018.

Future animal improvement programs applied to global populations

USDA-ARS?s Scientific Manuscript database

Breeding programs evolved gradually from within-herd phenotypic selection to local and regional cooperatives to national evaluations and now international evaluations. In the future, breeders may adapt reproductive, computational, and genomic methods to global populations as easily as with national ...

Evaluation of traffic signal timing optimization methods using a stochastic and microscopic simulation program.

DOT National Transportation Integrated Search

2003-01-01

This study evaluated existing traffic signal optimization programs including Synchro,TRANSYT-7F, and genetic algorithm optimization using real-world data collected in Virginia. As a first step, a microscopic simulation model, VISSIM, was extensively ...

78 FR 47307 - Submission for OMB Review; Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2013-08-05

... obtain data on the pedagogical methods of National Guard Youth ChalleNGe program teachers. The data will be used by DoD to evaluate how differences in classroom teaching methods impact program outcomes. The... September 4, 2013. Title; Associated Form; and OMB Number: How Differences in Pedagogical Methods Impact...

The TMDL Program Results Analysis Project: Matching Results Measures with Program Expectations

EPA Pesticide Factsheets

The paper provides a detailed description of the aims, methods and outputs of the program evaluation project undertaken by EPA in order to generate the insights needed to make TMDL program improvements.

Designing an evaluation for a multiple-strategy community intervention: the North Coast Stay on Your Feet program.

PubMed

van Beurden, E; Kempton, A; Sladden, T; Garner, E

1998-02-01

Evaluation of the North Coast Stay on Your Feet falls prevention program is described as a case study of a comprehensive evaluation design for multi-strategic community interventions. Qualitative and quantitative methods were used to evaluate the program at formative, process and outcome levels. Formative evaluation used literature review, focus groups, mail-out and telephone survey methods to gather evidence from publications, older people, health workers, local business, media and government bodies. It included an analysis of demographic and hospital databases and identified incidence, causal pathways, knowledge, attitudes, behaviour, consequences and effectiveness of potential strategies. Process evaluation employed auditing, monitoring and telephone surveys to maintain an inventory of intervention activities and to track the reach of the program. Outcome evaluation involved a longitudinal study of intervention and control cohorts, surveyed before, during and after the intervention by telephone to monitor changes in knowledge, attitudes, risk and falls incidence. The survey instrument was designed for both formative and outcome evaluation, and analysis reflected the research design by incorporating repeat measures and adjusting for bias and confounding. Outcome validity was cross-checked via hospital admission rates. A novel, integrated framework for presenting inputs, activities and outcomes from all stages of the program is described. This framework facilitated feedback to stakeholders and enabled subsequent rapid adjustment of the intervention. Rigorous evaluation combined with clear presentation of findings helped to engender intersectoral support and obtain funding grants for extended implementation and evaluation. It also helped Stay on Your Feet to become a model for other falls prevention programs within Australia and internationally.

Evaluation of Time Domain EM Coupling Techniques. Volume II.

DTIC Science & Technology

1980-08-01

tool for the analysis of elec- tromangetic coupling and shielding problems: the finite-difference, time-domain (FD- TD ) solution of Maxwell’s equations...The objective of the program was to evaluate the suitability of the FD- TD method to determine the amount of electromagnetic coupling through an...specific questfiowwere addressed during this program: 1. Can the FD- TD method accurately model electromagnetic coupling into a conducting structure for

Collaborative evaluation of a high school prevention curriculum: How methods of collaborative evaluation enhanced a randomized control trial to inform program improvement.

PubMed

Orsini, Muhsin Michael; Wyrick, David L; Milroy, Jeffrey J

2012-11-01

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome variables that can be used for program improvement. Throughout this project there were many examples of client learning from evaluator and evaluator learning from client. This article presents convincing evidence that collaborative evaluation can improve the design, implementation, and findings of the randomized control trial. Throughout this paper, we discuss many examples of good science, good evaluation, and other practical benefits of practicing collaborative evaluation. Ultimately, the authors created the term pre-formative evaluation to describe the period prior to data collection and before program implementation, when collaborative evaluation can inform program improvement. Copyright © 2012 Elsevier Ltd. All rights reserved.

76 FR 29752 - Nomination of In Vitro Test Methods for Detection and Quantification of Botulinum Neurotoxins and...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-05-23

... DEPARTMENT OF HEALTH AND HUMAN SERVICES Nomination of In Vitro Test Methods for Detection and... Evaluated by These Test Methods AGENCY: Division of National Toxicology Program (NTP), National Institute of... Methods (ICCVAM), the NTP Interagency Center for the Evaluation of Alternative Toxicological Methods...

Chapter 21: Estimating Net Savings - Common Practices. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Violette, Daniel M.; Rathbun, Pamela

This chapter focuses on the methods used to estimate net energy savings in evaluation, measurement, and verification (EM and V) studies for energy efficiency (EE) programs. The chapter provides a definition of net savings, which remains an unsettled topic both within the EE evaluation community and across the broader public policy evaluation community, particularly in the context of attribution of savings to a program. The chapter differs from the measure-specific Uniform Methods Project (UMP) chapters in both its approach and work product. Unlike other UMP resources that provide recommended protocols for determining gross energy savings, this chapter describes and comparesmore » the current industry practices for determining net energy savings but does not prescribe methods.« less

Evaluating a culturally tailored peer-mentoring and education pilot intervention among Chinese breast cancer survivors using a mixed-methods approach.

PubMed

Lu, Qian; You, Jin; Man, Jenny; Loh, Alice; Young, Lucy

2014-11-01

To evaluate a social support intervention that was culturally tailored for Chinese Americans who face many challenges because of cultural and linguistic barriers. Intervention with a one-group pre- or post-test design, mixed methods, and a community-based participatory research (CBPR) approach. Southern California. 14 Chinese American breast cancer survivors post-treatment and eight breast cancer peer mentors. The intervention was a 10-week program to provide emotional and informational support through peer mentoring and education. Health outcomes were assessed before and after the intervention. Eight weekly process evaluations and two focus group interviews also were conducted. Depressive and anxiety symptoms. The program was associated with a decrease in depressive symptoms. Participants valued the program highly. Inductive analysis suggested possible mechanisms for effectiveness, such as reducing stigma, empowerment, and increased sense of belonging. The peer-mentoring and education program has the potential to serve as a model intervention for ethnic minorities. Mixed methods and CBPR are valuable in evaluating pilot interventions with minorities. Focusing on relationships may be fruitful for designing novel interventions for cancer survivors from collectivistic cultures. Peer-mentoring and education programs can be integrated into communities and clinics to improve care for underserved minority cancer survivors and to reduce health disparities.

Establishing monitoring programs for travel time reliability.

DOT National Transportation Integrated Search

2014-01-01

Within the second Strategic Highway Research Program (SHRP 2), Project L02 focused on creating a suite of methods by which transportation agencies could monitor and evaluate travel time reliability. Creation of the methods also produced an improved u...

SUPERFUND INNOVATIVE TECHNOLOGY EVALUATION: TECHNOLOGY PROFILES, NINTH EDITION

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its eleventh year, is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine use o...

How Do Mode and Timing of Follow-up Surveys Affect Evaluation Success?

ERIC Educational Resources Information Center

Koundinya, Vikram; Klink, Jenna; Deming, Philip; Meyers, Andrew; Erb, Kevin

2016-01-01

This article presents the analysis of evaluation methods used in a well-designed and comprehensive evaluation effort of a significant Extension program. The evaluation data collection methods were analyzed by questionnaire mode and timing of follow-up surveys. Response rates from the short- and long-term follow-ups and different questionnaire…

Evaluating the Implementation of an Olympic Education Program in Greece

NASA Astrophysics Data System (ADS)

Grammatikopoulos, Vasilios; Tsigilis, Nikolaos; Koustelios, Athanasios; Theodorakis, Yannis

2005-11-01

The aim of this study was to develop an instrument for evaluating how an education program has been implemented. Such evaluation can provide insight into the effectiveness of a program. Examined here was the Olympic Education Program used in Greek schools since 2000. In it, students learn the history of the Olympic games and the importance of exercise for health along with the principles and values of sports and volunteerism. The evaluation instrument underlying this study addressed the following six factors: `facilities', `administration', `educational material', `student-teacher relationships', `educational procedures', and `training'. Results indicate that the instrument, while adequate for assessing effectiveness, should be combined with advanced statistical methods.

Socioeconomic impact assessment in ex ante evaluations: a case study on the rural development programs of the European Union.

PubMed

Vidueira, Pablo; Díaz-Puente, José M; Rivera, María

2014-08-01

Ex ante impact assessment has become a fundamental tool for effective program management, and thus, a compulsory task when establishing a new program in the European Union (EU). This article aims to analyze benefits from ex ante impact assessment, methodologies followed, and difficulties encountered. This is done through the case study on the rural development programs (RDPs) in the EU. Results regarding methodologies are then contrasted with the international context in order to provide solid insights to evaluators and program managing authorities facing ex ante impact assessment. All European RDPs from the period 2007 through 2013 (a total of 88) and their corresponding available ex ante evaluations (a total of 70) were analyzed focusing on the socioeconomic impact assessment. Only 46.6% of the regions provide quantified impact estimations on socioeconomic impacts in spite of it being a compulsory task demanded by the European Commission (EC). Recommended methods by the EC are mostly used, but there is a lack of mixed method approaches since qualitative methods are used in substitution of quantitative ones. Two main difficulties argued were the complexity of program impacts and the lack of needed program information. Qualitative approaches on their own have been found as not suitable for ex ante impact assessment, while quantitative approaches-such as microsimulation models-provide a good approximation to actual impacts. However, time and budgetary constraints make that quantitative and mixed methods should be mainly applied on the most relevant impacts for the program success. © The Author(s) 2014.

Qualitative and Semiquantitative Assessment of Exposure to Engineered Nanomaterials within the French EpiNano Program: Inter- and Intramethod Reliability Study.

PubMed

Guseva Canu, Irina; Jezewski-Serra, Delphine; Delabre, Laurène; Ducamp, Stéphane; Iwatsubo, Yuriko; Audignon-Durand, Sabine; Ducros, Cécile; Radauceanu, Anca; Durand, Catherine; Witschger, Olivier; Flahaut, Emmanuel

2017-01-01

The relatively recent development of industries working with nanomaterials has created challenges for exposure assessment. In this article, we propose a relatively simple approach to assessing nanomaterial exposures for the purposes of epidemiological studies of workers in these industries. This method consists of an onsite industrial hygiene visit of facilities carried out individually and a description of workstations where nano-objects and their agglomerates and aggregates (NOAA) are present using a standardized tool, the Onsite technical logbook. To assess its reliability, we implemented this approach for assessing exposure to NOAA in workplaces at seven workstations which synthesize and functionalize carbon nanotubes. The prediction of exposure to NOAA using this method exhibited substantial agreement with that of the reference method, the latter being based on an onsite group visit, an expert's report and exposure measurements (Cohen kappa = 0.70, sensitivity = 0.88, specificity = 0.92). Intramethod comparison of results for exposure prediction showed moderate agreement between the three evaluators (two program team evaluators and one external evaluator) (weighted Fleiss kappa = 0.60, P = 0.003). Interevaluator reliability of the semiquantitative exposure characterization results was excellent between the two evaluators from the program team (Spearman rho = 0.93, P = 0.03) and fair when these two evaluators' results were compared with the external evaluator's results. The project was undertaken within the framework of the French epidemiological surveillance program EpiNano. This study allowed a first reliability assessment of the EpiNano method. However, to further validate this method a comparison with robust quantitative exposure measurement data is necessary. © The Author 2017. Published by Oxford University Press on behalf of the British Occupational Hygiene Society.

An evaluation of undergraduate family care programs.

PubMed

Silberstein, E B; Scott, C J

1978-01-01

Evaluation of the many new family care programs (FCP), and others of similar intent, however defined, is as essential as determining the value of any other curriculum change. Replies to a questionnaire from 101 U.S. and 15 Canadian medical schools indicated that 80% of the former and 93% of the latter had FCPs; 35% and 29%, respectively, were not evaluating their program by any method. No single evaluative technique was used by more than 42% of the American medical schools. A review of the literature on FCPs frequently indicated that the conclusions that could be drawn about the programs were ambiguous. Students in the University of Cincinnati Medical Center FCPs elected family practice or pediatric internships significantly more often than did the FCP nonparticipants, but they indicated that the program had little effect on this choice, despite almost uniformly favorable testimonials. We discuss the possibility that FCPs may be educating the wrong students, that FCPs, if not reinforced in other clinical areas, may have few lasting effects on student attitudes or career choice, and that we may be asking the wrong questions, and with inadequate methods.

Using Work Action Analysis to Identify Web-Portal Requirements for a Professional Development Program

ERIC Educational Resources Information Center

Nickles, George

2007-01-01

This article describes using Work Action Analysis (WAA) as a method for identifying requirements for a web-based portal that supports a professional development program. WAA is a cognitive systems engineering method for modeling multi-agent systems to support design and evaluation. A WAA model of the professional development program of the…

Enhancing the Possibility of Success by Measuring the Probability of Failure in an Educational Program.

ERIC Educational Resources Information Center

Brookhart, Susan M.; And Others

1997-01-01

Process Analysis is described as a method for identifying and measuring the probability of events that could cause the failure of a program, resulting in a cause-and-effect tree structure of events. The method is illustrated through the evaluation of a pilot instructional program at an elementary school. (SLD)

Evaluating Integrative Cancer Clinics With the Claim Assessment Profile: An Example With the InspireHealth Clinic

PubMed Central

Hilton, Lara; Elfenbaum, Pamela; Jain, Shamini; Sprengel, Meredith; Jonas, Wayne B.

2016-01-01

Background: The evaluation of freestanding integrative cancer clinical programs is challenging and is rarely done. We have developed an approach called the Claim Assessment Profile (CAP) to identify whether evaluation of a practice is justified, feasible, and likely to provide useful information. Objectives: A CAP was performed in order to (1) clarify the healing claims at InspireHealth, an integrative oncology treatment program, by defining the most important impacts on its clients; (2) gather information about current research capacity at the clinic; and (3) create a program theory and path model for use in prospective research. Study Design/Methods: This case study design incorporates methods from a variety of rapid assessment approaches. Procedures included site visits to observe the program, structured qualitative interviews with 26 providers and staff, surveys to capture descriptive data about the program, and observational data on program implementation. Results: The InspireHealth program is a well-established, multi-site, thriving integrative oncology clinical practice that focuses on patient support, motivation, and health behavior engagement. It delivers patient-centered care via a standardized treatment protocol. There arehigh levels of research interest from staff and resources by which to conduct research. Conclusions: This analysis provides the primary descriptive and claims clarification of an integrative oncology treatment program, an evaluation readiness report, a detailed logic model explicating program theory, and a clinical outcomes path model for conducting prospective research. Prospective evaluation of this program would be feasible and valuable, adding to our knowledge base of integrative cancer therapies. PMID:29444602

FIRES: Fire Information Retrieval and Evaluation System - A program for fire danger rating analysis

Treesearch

Patricia L. Andrews; Larry S. Bradshaw

1997-01-01

A computer program, FIRES: Fire Information Retrieval and Evaluation System, provides methods for evaluating the performance of fire danger rating indexes. The relationship between fire danger indexes and historical fire occurrence and size is examined through logistic regression and percentiles. Historical seasonal trends of fire danger and fire occurrence can be...

Design and Implementation of a Multi-Strategy, Collegewide Program of Evaluation and Planning: The Mercy College Self-Study Project.

ERIC Educational Resources Information Center

Kraetzer, Mary C.; And Others

The rationale, strategies, and methods of The Mercy College Self-Study Project are considered, and evaluation instruments are provided. This program of institutional evaluation and planning was initiated in 1980 and consists of: standardized surveys, a 10-year longitudinal (panel) study, and academic department self-studies. Questionnaires…

Programs to improve adolescent sexual and reproductive health in the US: a review of the evidence

PubMed Central

Manlove, Jennifer; Fish, Heather; Moore, Kristin Anderson

2015-01-01

Background US adolescents have high rates of teen pregnancy, childbearing, and sexually transmitted infections (STIs), highlighting the need to identify and implement effective programs that will help improve teen sexual and reproductive health. Materials and methods This review identified 103 random-assignment evaluations of 85 programs that incorporated intent-to-treat analyses and assessed impacts on pregnancy, childbearing, STIs, and their key determinants – sexual activity, number of sexual partners, condom use, and other contraceptive use – among teens. This review describes the evidence base for five broad program approaches, including abstinence education, comprehensive sex education, clinic-based programs, youth development programs, and parent–youth relationship programs. We also describe programs with impacts on key outcomes, including pregnancy/childbearing, STIs, and those that found impacts on both sexual activity and contraceptive use. Results Our review identified 52 effective programs: 38 with consistent impacts on reproductive health outcomes, and 14 with mixed findings (across subpopulations, follow-ups, or multiple measures of a single outcome). We found that a variety of program approaches produced impacts on sexual and reproductive health outcomes. Parent–youth relationship programs and clinic-based program evaluations more frequently showed impacts than other program approaches, although we also identified a number of abstinence-education, comprehensive sex education, and youth-development programs with impacts on sexual and reproductive health outcomes. Overall, we identified nine program evaluations with impacts on teen pregnancies or births, five with impacts on reducing STIs, and 15 with impacts on both delaying/reducing sexual activity and increasing contraceptive use (including condom use). Conclusion Future efforts should conduct replications of existing program evaluations, identify implementation components linked to impacts, rigorously evaluate programs that appear promising, and expand the evidence base on programs that impact hormonal and long-acting contraceptive method use. PMID:25897271

Defining a staged-based process for economic and financial evaluations of mHealth programs.

PubMed

LeFevre, Amnesty E; Shillcutt, Samuel D; Broomhead, Sean; Labrique, Alain B; Jones, Tom

2017-01-01

Mobile and wireless technology for health (mHealth) has the potential to improve health outcomes by addressing critical health systems constraints that impede coverage, utilization, and effectiveness of health services. To date, few mHealth programs have been implemented at scale and there remains a paucity of evidence on their effectiveness and value for money. This paper aims to improve understanding among mHealth program managers and key stakeholders of how to select methods for economic evaluation (comparative analysis for determining value for money) and financial evaluation (determination of the cost of implementing an intervention, estimation of costs for sustaining or expanding an intervention, and assessment of its affordability). We outline a 6 stage-based process for selecting and integrating economic and financial evaluation methods into the monitoring and evaluation of mHealth solutions including (1) defining the program strategy and linkages with key outcomes, (2) assessment of effectiveness, (3) full economic evaluation or partial evaluation, (4) sub-group analyses, (5) estimating resource requirements for expansion, (6) affordability assessment and identification of models for financial sustainability. While application of these stages optimally occurs linearly, finite resources, limited technical expertise, and the timing of evaluation initiation may impede this. We recommend that analysts prioritize economic and financial evaluation methods based on programmatic linkages with health outcomes; alignment with an mHealth solution's broader stage of maturity and stage of evaluation; overarching monitoring and evaluation activities; stakeholder evidence needs; time point of initiation; and available resources for evaluations.

Evolution and outcomes of a quality improvement program.

PubMed

Thor, Johan; Herrlin, Bo; Wittlöv, Karin; Øvretveit, John; Brommels, Mats

2010-01-01

The purpose of this paper is to examine the outcomes and evolution over a five-year period of a Swedish university hospital quality improvement program in light of enduring uncertainty regarding the effectiveness of such programs in healthcare and how best to evaluate it. The paper takes the form of a case study, using data collected as part of the program, including quality indicators from clinical improvement projects and participants' program evaluations. Overall, 58 percent of the program's projects (39/67) demonstrated success. A greater proportion of projects led by female doctors demonstrated success (91 percent, n=11) than projects led by male doctors (51 percent, n=55). Facilitators at the hospital continuously adapted the improvement methods to the local context. A lack of dedicated time for improvement efforts was the participants' biggest difficulty. The dominant benefits included an increased ability to see the "bigger picture" and the improvements achieved for patients and employees. Quality measurement, which is important for conducting and evaluating improvement efforts, was weak with limited reliability. Nevertheless, the present study adds evidence about the effectiveness of healthcare improvement programs. Gender differences in improvement team leadership merit further study. Improvement program evaluation should assess the extent to which improvement methods are locally adapted and applied. This case study reports the outcomes of all improvement projects undertaken in one healthcare organization over a five-year period and provides in-depth insight into an improvement program's changeable nature.

Evaluating disease management program effectiveness: an introduction to time-series analysis.

PubMed

Linden, Ariel; Adams, John L; Roberts, Nancy

2003-01-01

Currently, the most widely used method in the disease management (DM) industry for evaluating program effectiveness is referred to as the "total population approach." This model is a pretest-posttest design, with the most basic limitation being that without a control group, there may be sources of bias and/or competing extraneous confounding factors that offer a plausible rationale explaining the change from baseline. Furthermore, with the current inclination of DM programs to use financial indicators rather than program-specific utilization indicators as the principal measure of program success, additional biases are introduced that may cloud evaluation results. This paper presents a non-technical introduction to time-series analysis (using disease-specific utilization measures) as an alternative, and more appropriate, approach to evaluating DM program effectiveness than the current total population approach.

Evaluation of Arabic Language Learning Program for Non-Native Speakers in Saudi Electronic University According to Total Quality Standards

ERIC Educational Resources Information Center

Alowaydhi, Wafa Hafez

2016-01-01

The current study aimed at standardizing the program of learning Arabic for non-native speakers in Saudi Electronic University according to certain standards of total quality. To achieve its purpose, the study adopted the descriptive analytical method. The author prepared a measurement tool for evaluating the electronic learning programs in light…

The Wide-Scale Implementation of a Support Program for Parents of Children with an Intellectual Disability and Difficult Behaviour

ERIC Educational Resources Information Center

Hudson, Alan; Cameron, Christine; Matthews, Jan

2008-01-01

Background: While there have been several evaluations of programs to help parents manage difficult behaviour of their child with an intellectual disability, little research has focused on the evaluation of such programs when delivered to large populations. Method: The benchmarks recommended by Wiese, Stancliffe, and Hemsley (2005) were used to…

Suicide Prevention among High School Students: Evaluation of a Nonrandomized Trial of a Multi-Stage Suicide Screening Program

ERIC Educational Resources Information Center

Torcasso, Gina; Hilt, Lori M.

2017-01-01

Background: Suicide is a leading cause of death among youth. Suicide screening programs aim to identify mental health issues and prevent death by suicide. Objective: The present study evaluated outcomes of a multi-stage screening program implemented over 3 school years in a moderately-sized Midwestern high school. Methods: One hundred ninety-three…

Adding to the Pool of Methods for Program Evaluation: A Comparison of Latent Class Analysis and Propensity Score Analysis

ERIC Educational Resources Information Center

Taylor, Lauren Christine

2013-01-01

Considering the amount of funding that is distributed to educational research each year, leaders and policymakers have a vested interest in finding scientifically based evidence that answers causal questions regarding program effectiveness. The importance of program evaluation has long been recognized in many fields of research; however, the most…

Algunos Criterios para Evaluar Programas de Educacion Superior a Nivel de Posgrado: El Caso Particular de la Administracion Publica (Some Criteria to Evaluate Higher Education Programs at the Graduate Level: The Special Case of Public Administration).

ERIC Educational Resources Information Center

Valle, Victor M.

Intended as a contribution to a workshop discussion on program evaluation in higher education, the paper covers five major evaluation issues. First, it deals with evaluation concepts, explaining the purposes of evaluation; pertinent terms; and the sources of evaluation in public health procedures, the scientific method, the systems approach, and…

A mixed-methods study of teachers of the deaf learning to integrate computers into their teaching.

PubMed

Kluwin, Thomas N; Noretsky, Martin

2005-01-01

A MIXED-METHODS STUDY was conducted of 47 teachers of the deaf in seven urban and suburban programs around the country as they participated in an online training program. The goal of the training program was to encourage the teachers to integrate technology into their teaching of children who were deaf or hard of hearing. In their evaluation of the project, the researchers sought to identify factors that supported the success of the program. Data used in the evaluation of the project were drawn from responses to formal questionnaires, information on retention in the program, and an analysis of comments during online asynchronous discussions. Anxiety about using technology, access to technology, and initial expertise did not determine whether teachers were retained in the program; however, the degree of local support for participants did differentiate those who continued in the program from those who dropped out.

Novel Method Of Preparing Vaccines | NCI Technology Transfer Center | TTC

Cancer.gov

This invention from the NCI Cancer and Inflammation Program describes methods to prepare vaccines for the treatment of human immunodeficiency virus (HIV) infections. The National Cancer Institute's Cancer and Inflammation Program seeks parties interested in licensing or collaborative research to further develop, evaluate, or commercialize novel methods of preparing vaccines.

An Evaluation of the Effectiveness of U.S. Naval Aviation Crew Resource Management Training Programs: A Reassessment for the Twenty-First Century Operating Environment

DTIC Science & Technology

2009-06-01

3. Previous Navy CRM Assessments ....................................................24 4. Applying Kirkpatrick’s Topology of Evaluation...development within each aviation community. Kirkpatrick’s (1976) hierarchy of training evaluation technique was applied to examine three levels of... Applying methods and techniques used in previous CRM evaluation research, this thesis provided an updated evaluation of the Naval CRM program to fill

Methods and Purposes for Conducting Students' Course Evaluations Reported by North American Dental School and Dental Hygiene Program Leaders: A Preliminary Survey Study.

PubMed

Reinke, Robin E; Enright, Tonya; Love, Rebecca; Patel, Shalizeh A; Ali, Ala Omar; Horvath, Zsuzsa

2018-03-01

The aim of this preliminary survey study was to determine the perceptions of leaders of dental schools and dental hygiene programs regarding methods of and purposes for conducting students' course evaluations and their role in course improvement, curriculum design, and faculty assessment. A short electronic survey was distributed in 2016 to the academic deans of all 76 dental schools in the U.S. and Canada and a convenience sample of program directors of 232 of the total 332 accredited dental hygiene programs. Individuals from 93 institutions responded for an overall response rate of 30%: 30 of 76 dental schools (39.5% response rate) and 63 of the 232 dental hygiene programs (27% response rate). All of the respondents (100%) reported that their institutions' full-time faculty members were assessed by students in course evaluations for each course and semester they taught. However, only 78% reported that their part-time faculty members were evaluated by students. Course evaluations were mandatory in 62% (n=58) of the responding institutions, with the remaining 38% (n=35) optional. Respondents indicated course directors received the evaluation results for purposes of annual review (n=73, 78%) and instructional review (n=70, 75%). Further investigation of the use and effects of student evaluations is needed to better understand their role in faculty assessment and other aspects of the administration of dental schools and dental hygiene programs.

Assessment of village health worker training program in Tuguegarao, Philippine.

PubMed

Kim, Jung Min; Koh, Kwang Wook; Oak, Chul Ho; Jung, Woo Hyuk; Kim, Sung Hyun; Park, Dae Hee

2009-11-01

This study was performed to evaluate the effectiveness of 'village health worker training program' which aimed to build community participatory health promotion capacity of community leaders in villages of low developed country and to develop methods for further development of the program. The intervention group were 134 community leaders from 25 barangays (village). Control group were 149 form 4 barangays. Intervention group participated 3-day training program. Questionnaire was developed based on 'Health Promotion Capacity Checklist' which assessed capacity in 4 feathers; 'knowledge', 'skill', 'commitment', and 'resource'. Each feather was assessed in 4 point rating scale. Capacity scores between intervention group and control group were examined to identify changes between the pre- and post-intervention periods. A qualitative evaluation of the program was conducted to assess the appropriateness of the program. The program was conducted in Tuguegarao city, Philippine in January, 2009. The result showed significant increases in the total health promotion capacity and each feather of health promotion capacities between pre and post assessment of intervention group. But there was no significant change in that of control group. Participants marked high level of satisfaction for preparedness, selection of main subjects and education method. Qualitative evaluation revealed that training program facilitated community participatory health promotion capacity of participants. This study suggested that the Village health worker training program is effective for building health promotion capacity of community leaders and it can be a main method for helping low developed countries with further development.

Assessment and Evaluation in Adapted Physical Education.

ERIC Educational Resources Information Center

Kennedy, Charlotte I.; Bundschuh, Ernie

Intended for physical education teachers, the booklet describes informal and formal methods for evaluating handicapped children to determine whether they can participate in a regular physical education program with nonhandicapped students, in a regular physical education program with modification, or in a specially designed physical education…

Canadian Pacific Railway mechanical services' 5-Alive safety program shows promise in reducing injuries.

DOT National Transportation Integrated Search

2006-09-01

The Federal Railroad Administration (FRA) Human Factors Research and Development (R&D) Program is sponsoring an Alternative Safety Measures Program to explore alternative methods for evaluating whether safety programs improve safety outcomes and the ...

Milestone-specific, Observed data points for evaluating levels of performance (MODEL) assessment strategy for anesthesiology residency programs.

PubMed

Nagy, Christopher J; Fitzgerald, Brian M; Kraus, Gregory P

2014-01-01

Anesthesiology residency programs will be expected to have Milestones-based evaluation systems in place by July 2014 as part of the Next Accreditation System. The San Antonio Uniformed Services Health Education Consortium (SAUSHEC) anesthesiology residency program developed and implemented a Milestones-based feedback and evaluation system a year ahead of schedule. It has been named the Milestone-specific, Observed Data points for Evaluating Levels of performance (MODEL) assessment strategy. The "MODEL Menu" and the "MODEL Blueprint" are tools that other anesthesiology residency programs can use in developing their own Milestones-based feedback and evaluation systems prior to ACGME-required implementation. Data from our early experience with the streamlined MODEL blueprint assessment strategy showed substantially improved faculty compliance with reporting requirements. The MODEL assessment strategy provides programs with a workable assessment method for residents, and important Milestones data points to programs for ACGME reporting.

Building capacity for water, sanitation, and hygiene programming: Training evaluation theory applied to CLTS management training in Kenya.

PubMed

Crocker, Jonny; Shields, Katherine F; Venkataramanan, Vidya; Saywell, Darren; Bartram, Jamie

2016-10-01

Training and capacity building are long established critical components of global water, sanitation, and hygiene (WaSH) policies, strategies, and programs. Expanding capacity building support for WaSH in developing countries is one of the targets of the Sustainable Development Goals. There are many training evaluation methods and tools available. However, training evaluations in WaSH have been infrequent, have often not utilized these methods and tools, and have lacked rigor. We developed a conceptual framework for evaluating training in WaSH by reviewing and adapting concepts from literature. Our framework includes three target outcomes: learning, individual performance, and improved programming; and two sets of influences: trainee and context factors. We applied the framework to evaluate a seven-month community-led total sanitation (CLTS) management training program delivered to 42 government officials in Kenya from September 2013 to May 2014. Trainees were given a pre-training questionnaire and were interviewed at two weeks and seven months after initial training. We qualitatively analyzed the data using our conceptual framework. The training program resulted in trainees learning the CLTS process and new skills, and improving their individual performance through application of advocacy, partnership, and supervision soft skills. The link from trainees' performance to improved programming was constrained by resource limitations and pre-existing rigidity of trainees' organizations. Training-over-time enhanced outcomes and enabled trainees to overcome constraints in their work. Training in soft skills is relevant to managing public health programs beyond WaSH. We make recommendations on how training programs can be targeted and adapted to improve outcomes. Our conceptual framework can be used as a tool both for planning and evaluating training programs in WaSH. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Evaluating Realized Impacts of DOE/EERE R&D Programs. Standard impact evaluation method

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ruegg, Rosalie; O'Connor, Alan C.; Loomis, Ross J.

2014-08-01

This document provides guidance for evaluators who conduct impact assessments of research and development (R&D) programs for the U.S. Department of Energy’s (DOE) Office of Energy Efficiency and Renewable Energy (EERE). It is also targeted at EERE program staff responsible for initiating and managing commissioned impact studies. The guide specifies how to estimate economic benefits and costs, energy saved and installed or generated, environmental impacts, energy security impacts, and knowledge impacts of R&D investments in advanced energy technologies.

Can We Afford Not to Evaluate Services for Elderly Persons with Dementia?

PubMed Central

Worrall, Graham; Chambers, Larry W.

1989-01-01

With the increasing expenditure on health-care programs for seniors, there is an urgent need to evaluate such programs. The Measurement Iterative Loop is a tool that can provide both health administrators and health researchers with a method of evaluation of existing programs and identification of gaps in knowledge, and forms a rational basis for health-care policy decisions. In this article, the Loop is applied to one common problem of the elderly: dementia. PMID:21248993

Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods

PubMed Central

2010-01-01

Background Schools are key settings for health promotion (HP) but the development of suitable approaches for evaluating HP in schools is still a major topic of discussion. This article presents a research protocol of a program developed to evaluate HP. After reviewing HP evaluation issues, the various possible approaches are analyzed and the importance of a realistic evaluation framework and a mixed methods (MM) design are demonstrated. Methods/Design The design is based on a systemic approach to evaluation, taking into account the mechanisms, context and outcomes, as defined in realistic evaluation, adjusted to our own French context using an MM approach. The characteristics of the design are illustrated through the evaluation of a nationwide HP program in French primary schools designed to enhance children's social, emotional and physical health by improving teachers' HP practices and promoting a healthy school environment. An embedded MM design is used in which a qualitative data set plays a supportive, secondary role in a study based primarily on a different quantitative data set. The way the qualitative and quantitative approaches are combined through the entire evaluation framework is detailed. Discussion This study is a contribution towards the development of suitable approaches for evaluating HP programs in schools. The systemic approach of the evaluation carried out in this research is appropriate since it takes account of the limitations of traditional evaluation approaches and considers suggestions made by the HP research community. PMID:20109202

Evaluating adolescent pregnancy programs: rethinking our priorities.

PubMed

Stahler, G J; DuCette, J P

1991-01-01

Noting that impact evaluations of adolescent pregnancy programs are characterized by poor quality, the authors recommend using a different standard in assessing the value of programs. While the number of adolescent pregnancy programs has multiplied during the last 3 decades, little is known about their impact in ameliorating the negative consequences of too-early childbearing. An ideal evaluation of these programs would randomly select and randomly assign subjects to experimental and control groups. But evaluations conducted by individual program generally face obstacles that limit the randomness of the study. most individual programs lack the financial resources and do not employ the full-time professional evaluators needed to carry out a valid evaluation. These factors result in too short an evaluation period, incomplete and inaccurate data, and lack of randomness in the assignment of control groups. To more accurately assess the impact of the programs, the authors recommend that individual programs focus on process evaluation and collection of complete and reliable data on their clients. From the onset, a program should have a clear description of its content, logic of intervention, and method of implementation. It should maintain thorough records on client characteristics, service utilization, and should conduct long-term follow-ups. For rigorous impact evaluations, programs should rely on 3rd party entities. These independent organizations -- universities or research institutes -- do not have a stake in the outcome of the evaluation, making the study all the more objective. Furthermore, they provide experienced researchers.

Job Evaluation's Role in Employee Relations

ERIC Educational Resources Information Center

Dick, Arthur H.

1974-01-01

Jobs are evaluated to provide a fair basis for payment of wages; this procedure must be communicated to employees so they can see they are being treated equitably. Job ranking, grading or job classification, factor comparison, the point method, and direct pricing are methods used in job evaluation programs. (AG)

Evaluation of programs to improve complementary feeding in infants and young children.

PubMed

Frongillo, Edward A

2017-10-01

Evaluation of complementary feeding programs is needed to enhance knowledge on what works, to document responsible use of resources, and for advocacy. Evaluation is done during program conceptualization and design, implementation, and determination of effectiveness. This paper explains the role of evaluation in the advancement of complementary feeding programs, presenting concepts and methods and illustrating them through examples. Planning and investments for evaluations should occur from the beginning of the project life cycle. Essential to evaluation is articulation of a program theory on how change would occur and what program actions are required for change. Analysis of program impact pathways makes explicit the dynamic connections in the program theory and accounts for contextual factors that could influence program effectiveness. Evaluating implementation functioning is done through addressing questions about needs, coverage, provision, and utilization using information obtained from process evaluation, operations research, and monitoring. Evaluating effectiveness is done through assessing impact, efficiency, coverage, process, and causality. Plausibility designs ask whether the program seemed to have an effect above and beyond external influences, often using a nonrandomized control group and baseline and end line measures. Probability designs ask whether there was an effect using a randomized control group. Evaluations may not be able to use randomization, particularly for programs implemented at a large scale. Plausibility designs, innovative designs, or innovative combinations of designs sometimes are best able to provide useful information. Further work is needed to develop practical designs for evaluation of large-scale country programs on complementary feeding. © 2017 John Wiley & Sons Ltd.

[Evaluating programs for preparation to old age: some theoretical thoughts and a practical example (author's transl)].

PubMed

Schneider, H D

1977-07-01

In the first part methodically satisfactory experimental designs for evaluating programs for preparation to old age on the basis of Cambell & Stanely (1963) are sketched including conditions which hinder the realization of the ideal notions. In the second part an experiment for comparing two preparatory courses to old age using different methods of instruction is described. It was observed that the more demanding teaching method resulted in more anxiety for old age. This led to the conclusion, that programs for preparation to old age should be planned as a sequence of several units in which the mastering of the new ideas may be learned and in which this learning process may be supervised.

Findings of the District English Program Evaluation Committee.

ERIC Educational Resources Information Center

Price, A. Rae; And Others

In summer 1990, the English Department of the Metropolitan Community College District (MCCD) in Missouri conducted a self-study to determine whether the English program's subject matter, academic standards, and methods of instruction were consistent with objectives. An Evaluation Committee, consisting of three English instructors representing each…

Wildland Fire Prevention: Today, Intuition--Tomorrow, Management

Treesearch

Albert J. Simard; Linda R. Donoghue

1987-01-01

Describes, from a historical perspective, methods used to characterize fire prevention problems and evaluate prevention programs and discusses past research efforts to bolster these analytical and management efforts. Highlights research on the sociological perspectives of the wildfire problem and on quantitative fire occurrence prediction and program evaluation systems...

Evaluating Writing Programs: Paradigms, Problems, Possibilities.

ERIC Educational Resources Information Center

McLeod, Susan H.

1992-01-01

Describes two methodological approaches (qualitative and quantitative) that grow out of two different research examples. Suggests the problems these methods present. Discusses the ways in which an awareness of these problems can help teachers to understand how to work with researchers in designing useful evaluations of writing programs. (PRA)

Design and Evaluation of the NFL PLAY 60 FITNESSGRAM® Partnership Project

ERIC Educational Resources Information Center

Welk, Gregory J.; Bai, Yang; Saint-Maurice, Pedro F.; Allums-Featherston, Kelly; Candelaria, Norma

2016-01-01

This article describes the conceptual design and evaluation strategies used in the NFL PLAY 60 FITNESSGRAM® Partnership Project, a large participatory research network focused on building effective school physical education programming. The article summarizes the unique participatory design, recruitment methods, programming strategies, and…

The Evaluation of School-Based Violence Prevention Programs: A Meta-Analysis

ERIC Educational Resources Information Center

Park-Higgerson, Hyoun-Kyoung; Perumean-Chaney, Suzanne E.; Bartolucci, Alfred A.; Grimley, Diane M.; Singh, Karan P.

2008-01-01

Background: Youth violence and related aggressive behaviors have become serious public health issues with physical, economic, social, and psychological impacts and consequences. This study identified and evaluated the characteristics of successful school-based violence prevention programs. Methods: Twenty-six randomized controlled trial (RCT),…

A Controlled Evaluation of a High School Biomedical Pipeline Program: Design and Methods

NASA Astrophysics Data System (ADS)

Winkleby, Marilyn A.; Ned, Judith; Ahn, David; Koehler, Alana; Fagliano, Kathleen; Crump, Casey

2014-02-01

Given limited funding for school-based science education, non-school-based programs have been developed at colleges and universities to increase the number of students entering science- and health-related careers and address critical workforce needs. However, few evaluations of such programs have been conducted. We report the design and methods of a controlled trial to evaluate the Stanford Medical Youth Science Program's Summer Residential Program (SRP), a 25-year-old university-based biomedical pipeline program. This 5-year matched cohort study uses an annual survey to assess educational and career outcomes among four cohorts of students who participate in the SRP and a matched comparison group of applicants who were not chosen to participate in the SRP. Matching on sociodemographic and academic background allows control for potential confounding. This design enables the testing of whether the SRP has an independent effect on educational- and career-related outcomes above and beyond the effects of other factors such as gender, ethnicity, socioeconomic background, and pre-intervention academic preparation. The results will help determine which curriculum components contribute most to successful outcomes and which students benefit most. After 4 years of follow-up, the results demonstrate high response rates from SRP participants and the comparison group with completion rates near 90 %, similar response rates by gender and ethnicity, and little attrition with each additional year of follow-up. This design and methods can potentially be replicated to evaluate and improve other biomedical pipeline programs, which are increasingly important for equipping more students for science- and health-related careers.

A CONTROLLED EVALUATION OF A HIGH SCHOOL BIOMEDICAL PIPELINE PROGRAM: DESIGN AND METHODS.

PubMed

Winkleby, Marilyn A; Ned, Judith; Ahn, David; Koehler, Alana; Fagliano, Kathleen; Crump, Casey

2014-02-01

Given limited funding for school-based science education, non-school-based programs have been developed at colleges and universities to increase the number of students entering science- and health-related careers and address critical workforce needs. However, few evaluations of such programs have been conducted. We report the design and methods of a controlled trial to evaluate the Stanford Medical Youth Science Program's Summer Residential Program (SRP), a 25-year-old university-based biomedical pipeline program. This 5-year matched cohort study uses an annual survey to assess educational and career outcomes among four cohorts of students who participate in the SRP and a matched comparison group of applicants who were not chosen to participate in the SRP. Matching on sociodemographic and academic background allows control for potential confounding. This design enables the testing of whether the SRP has an independent effect on educational- and career-related outcomes above and beyond the effects of other factors such as gender, ethnicity, socioeconomic background, and pre-intervention academic preparation. The results will help determine which curriculum components contribute most to successful outcomes and which students benefit most. After 4 years of follow-up, the results demonstrate high response rates from SRP participants and the comparison group with completion rates near 90%, similar response rates by gender and ethnicity, and little attrition with each additional year of follow-up. This design and methods can potentially be replicated to evaluate and improve other biomedical pipeline programs, which are increasingly important for equipping more students for science- and health-related careers.

A Professional Development Program for Dental Medical Educators in Kuwait: Needs Assessment, Program Design and Formative Evaluation

ERIC Educational Resources Information Center

Alyaseen, Haneen

2017-01-01

New innovative methods of teaching and learning adopted from mainstream research and development in educational theory and practice are being adapted to serve the unique needs of the medical professions. The success of these methods requires careful planning and establishment of faculty development programs. The purpose of this study is to perform…

Pediatric Resident-as-Teacher Curricula: A National Survey of Existing Programs and Future Needs.

PubMed

Fromme, H Barrett; Whicker, Shari A; Paik, Steve; Konopasek, Lyuba; Koestler, Jennifer L; Wood, Beverly; Greenberg, Larrie

2011-06-01

We conducted a national survey of US pediatric program directors to explore the current status, content, and teaching methods of Resident-as-Teacher (RAT) curricula. The purposes of the survey were to (1) determine the level and method of evaluation of such curricula, and (2) assess the need for a national curricular resource in this area. A survey was sent to US pediatric program directors that asked questions regarding demographics, support, design, development, content, and evaluation of RAT curricula, as well as existing needs and desires for RAT curricular resources. Sixty-two percent of pediatric program directors completed our survey. Eighty-seven percent have a formal RAT curriculum, but more than 50% allocate 10 hours or less to it during residency. The primary teaching modalities are lectures and workshops. Content areas include feedback, in-patient teaching, communication skills, case-based teaching, role modeling, evaluation, leadership skills, 1-minute preceptors, teaching/learning styles, professionalism, and small-group teaching. Sixty-three percent of programs report evaluating their curricula, but only 27% perceive their program to be very/extremely effective. Nearly all respondents expressed interest in a national RAT curriculum, preferring web-based modules for dissemination. Despite an Accreditation Council for Graduate Medical Education requirement for a RAT curriculum, some pediatrics programs still lack one, and some consider their program only moderately effective. A wealth of curricular material exists across programs, which could be shared nationally. Establishing a national RAT curriculum would offer programs resources to meet educational mandates and the ability to tailor programs to best fit their own program needs.

Reflections on Empowerment Evaluation: Learning from Experience.

ERIC Educational Resources Information Center

Fetterman, David M.

1999-01-01

Reflects on empowerment evaluation, the use of evaluation to foster improvement and self-determination. Empowerment evaluation uses quantitative and qualitative methods, and usually focuses on program evaluation. Discusses the growth in empowerment evaluation as a result of interest in participatory evaluation. (SLD)

Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates. NCEE 2012-4019

ERIC Educational Resources Information Center

Fortson, Kenneth; Verbitsky-Savitz, Natalya; Kopa, Emma; Gleason, Philip

2012-01-01

Randomized controlled trials (RCTs) are widely considered to be the gold standard in evaluating the impacts of a social program. When an RCT is infeasible, researchers often estimate program impacts by comparing outcomes of program participants with those of a nonexperimental comparison group, adjusting for observable differences between the two…

Evaluation of Three Osteoporosis Prevention Programs for Young Women: Application of the Health Belief Model

ERIC Educational Resources Information Center

Lein, Donald H.; Turner, Lori; Wilroy, Jereme

2016-01-01

Purpose: The purpose of this study was to evaluate the effectiveness of theory-based osteoporosis prevention programs on calcium and vitamin D intakes and osteoporosis health beliefs in young women. Methods: Women (N = 152) aged 19 to 25 years were randomly assigned to one of 3 groups: a brochure group (n = 51), a computer-tailored program group…

A Hospice Rotation for Military Medical Residents: A Mixed Methods, Multi-Perspective Program Evaluation

PubMed Central

Boyden, Jackelyn Y.; Kalish, Virginia B.; Muir, J. Cameron; Richardson, Suzanne; Connor, Stephen R.

2016-01-01

Abstract Background: An estimated 6,000 to 18,000 additional hospice and palliative medicine (HPM) physicians are needed in the United States. A source could be the military graduate medical education system where 15% of U.S. medical residents are trained. A community-based hospice and palliative care organization created a one-week rotation for military residents including participation in interdisciplinary group visits at patients' homes, facilities, and an inpatient hospice unit. Objective: Our goal was to evaluate the effectiveness of a one-week community HPM rotation for military medical residents. Methods: A mixed-methods, multi-stakeholder perspective program evaluation model was used for program years 2011 to 2013. Data were managed and analyzed using Microsoft Excel and Atlas.ti. Participants in the rotation were residents training at two local military hospitals. Program evaluation data were collected from residents, military program liaisons, and hospice clinical preceptors. Quantitative data included pre- and post-tests based on Accreditation Council for Graduate Medical Education competencies completed by residents. Qualitative data included resident essays and semi-structured interviews with hospice preceptors and military program liaisons. Results: Quantitative and qualitative data suggested that the rotation increased military residents' knowledge, attitudes, and comfort level with HPM. Quantitative analysis of test scores indicated improvements from pre- to post-tests in each of five areas of learning. Qualitative data indicated the rotation created a greater appreciation for the overall importance of HPM and increased understanding of eligibility and methods for pain and symptom management. Conclusions: A one-week community hospice rotation for medical military residents impacts participant's knowledge of and attitudes toward HPM. PMID:27139524

Visualizing ultrasound through computational modeling

NASA Technical Reports Server (NTRS)

Guo, Theresa W.

2004-01-01

The Doppler Ultrasound Hematocrit Project (DHP) hopes to find non-invasive methods of determining a person s blood characteristics. Because of the limits of microgravity and the space travel environment, it is important to find non-invasive methods of evaluating the health of persons in space. Presently, there is no well developed method of determining blood composition non-invasively. This projects hopes to use ultrasound and Doppler signals to evaluate the characteristic of hematocrit, the percentage by volume of red blood cells within whole blood. These non-invasive techniques may also be developed to be used on earth for trauma patients where invasive measure might be detrimental. Computational modeling is a useful tool for collecting preliminary information and predictions for the laboratory research. We hope to find and develop a computer program that will be able to simulate the ultrasound signals the project will work with. Simulated models of test conditions will more easily show what might be expected from laboratory results thus help the research group make informed decisions before and during experimentation. There are several existing Matlab based computer programs available, designed to interpret and simulate ultrasound signals. These programs will be evaluated to find which is best suited for the project needs. The criteria of evaluation that will be used are 1) the program must be able to specify transducer properties and specify transmitting and receiving signals, 2) the program must be able to simulate ultrasound signals through different attenuating mediums, 3) the program must be able to process moving targets in order to simulate the Doppler effects that are associated with blood flow, 4) the program should be user friendly and adaptable to various models. After a computer program is chosen, two simulation models will be constructed. These models will simulate and interpret an RF data signal and a Doppler signal.

Cost effectiveness of the stream-gaging program in North Dakota

USGS Publications Warehouse

Ryan, Gerald L.

1989-01-01

This report documents results of a cost-effectiveness study of the stream-gaging program In North Dakota. It is part of a nationwide evaluation of the stream-gaging program of the U.S. Geological Survey.One phase of evaluating cost effectiveness is to identify less costly alternative methods of simulating streamflow records. Statistical or hydro logic flow-routing methods were used as alternative methods to simulate streamflow records for 21 combinations of gaging stations from the 94-gaging-station network. Accuracy of the alternative methods was sufficient to consider discontinuing only one gaging station.Operation of the gaging-station network was evaluated by using associated uncertainty in streamflow records. The evaluation was limited to the nonwinter operation of 29 gaging stations in eastern North Dakota. The current (1987) travel routes and measurement frequencies require a budget of about $248/000 and result in an average equivalent Gaussian spread in streamflow records of 16.5 percent. Changes in routes and measurement frequencies optimally could reduce the average equivalent Gaussian spread to 14.7 percent.Budgets evaluated ranged from $235,000 to $400,000. A $235,000 budget would increase the optimal average equivalent Gaussian spread from 14.7 to 20.4 percent, and a $400,000 budget could decrease it to 5.8 percent.

Cultural and Linguistic Adaptation of a Healthy Diet Text Message Intervention for Hispanic Adults

PubMed Central

Cameron, Linda D.; Durazo, Arturo; Ramirez, A. Susana; Corona, Roberto; Ultreras, Mayra; Piva, Sonia

2017-01-01

Hispanics represent a critical target for culturally-adapted diet interventions. In this formative research, we translated HealthyYouTXT, an mHealth program developed by the U.S. National Cancer Institute, into HealthyYouTXT en Español, a linguistically and culturally appropriate version for Spanish speakers. We report a three-stage, mixed-methods process through which we culturally adapted the text messages, evaluated their acceptability, and revised the program based on the findings. In Stage 1, we conducted initial translations and adaptations of the text libraries using an iterative, principle-guided process. In Stage 2, we used mixed methods including focus groups and surveys with 109 Hispanic adults to evaluate the acceptability and cultural appropriateness of the program. Further, we used survey data to evaluate whether Self-Determination Theory factors (used to develop HealthyYouTXT) of autonomous motivation, controlled motivation, and amotivation and Hispanic cultural beliefs about familism, fatalism, and destiny predict program interest and its perceived efficacy. Mixed-methods analyses revealed substantial interest in HealthyYouTXT, with most participants expressing substantial interest in using it and viewing it as highly efficacious. Both cultural beliefs (i.e., beliefs in destiny and, for men, high familism) and SDT motivations (i.e., autonomy) predicted HealthyYouTXT evaluations, suggesting utility in emphasizing them in messages. Higher destiny beliefs predicted lower interest and perceived efficacy, suggesting they could impede program use. In Stage 3, we implemented the mixed-methods findings to generate a revised HealthyYouTXT en Español. The emergent linguistic principles and multi-stage, multi-methods process can be applied beneficially in health communication adaptations. PMID:28248628

INTEGRATION OF SPATIAL DATA: EVALUATION OF METHODS BASED ON DATA ISSUES AND ASSESSMENT QUESTIONS

EPA Science Inventory

EPA's Regional Vulnerability Assessment (ReVA) Program has focused initially on the synthesis of existing data. We have used the same set of spatial data and synthesized these data using a total of 11 existing and newly developed integration methods. These methods were evaluated ...

75 FR 25867 - National Toxicology Program (NTP) Interagency Center for the Evaluation of Alternative...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-05-10

... validation studies. NICEATM and ICCVAM work collaboratively to evaluate new and improved test methods... for nomination of test methods for validation studies, and guidelines for submission of test methods... for human and veterinary vaccine post-licensing potency and safety testing. Plenary and breakout...

A systematic review on how to conduct evaluations in community-based rehabilitation

PubMed Central

Hébert, Michèle; Thibeault, Rachel

2014-01-01

Purpose Community-based rehabilitation (CBR) must prove that it is making a significant difference for people with disabilities in low- and middle-income countries. Yet, evaluation is not a common practice and the evidence for its effectiveness is fragmented and largely insufficient. The objective of this article was to review the literature on best practices in program evaluation in CBR in relation to the evaluative process, the frameworks, and the methods of data collection. Method A systematic search was conducted on five rehabilitation databases and the World Health Organization website with keywords associated with CBR and program evaluation. Two independent researchers selected the articles. Results Twenty-two documents were included. The results suggest that (1) the evaluative process needs to be conducted in close collaboration with the local community, including people with disabilities, and to be followed by sharing the findings and taking actions, (2) many frameworks have been proposed to evaluate CBR but no agreement has been reached, and (3) qualitative methodologies have dominated the scene in CBR so far, but their combination with quantitative methods has a lot of potential to better capture the effectiveness of this strategy. Conclusions In order to facilitate and improve evaluations in CBR, there is an urgent need to agree on a common framework, such as the CBR matrix, and to develop best practice guidelines based on the literature available and consensus among a group of experts. These will need to demonstrate a good balance between community development and standards for effective evaluations. Implications for Rehabilitation In the quest for evidence of the effectiveness of community-based rehabilitation (CBR), a shared program evaluation framework would better enable the combination of findings from different studies. The evaluation of CBR programs should always include sharing findings and taking action for the sake of the local community. Although qualitative methodologies have dominated the scene in CBR and remain highly relevant, there is also a call for the inclusion of quantitative indicators in order to capture the progress made by people participating in CBR programs. The production of best practice guidelines for evaluation in CBR could foster accountable and empowering program evaluations that are congruent with the principles at the heart of CBR and the standards for effective evaluations. PMID:23614357

Department of Defense Financial Management Education and Training Programs: A Survey of Quality Assurance Methods

DTIC Science & Technology

1992-06-01

CRITERIA TO HIRE CIVILIANS 10 21. PROFESSIONAL QUALIFICATION STANDARDS 18 22. CLASSROOM OBSERVATION 19 23. OTHER METHODS TO EVALUATE 18 INSTRUCTION 24. OTHER...other methods used to evaluate classroom instruction? (Note: Question 23 asks whether respondents use classroom observation to evaluate instruction] (15...number of affirmative responses are as follows: "* Question 22: Do you use classroom observation to evaluate instruction? (17 responses) "* Question

The effectiveness of environmental education programs from the perspectives of three stakeholders: Participants, sponsors, and professionals

NASA Astrophysics Data System (ADS)

Luera, Gail Rose

As the field of environmental education has matured since its inception in 1970, so have the number and variety of environmental programs. Along with the increased number of programs has come a need for in-depth program evaluations. This is especially critical because of reductions in educational funding and competition for a place in an already crowded curriculum. Evaluation is essential to convince the educational community that environmental education can improve the curriculum, ensure cost effectiveness, and become more relevant to students. Drawing on program evaluation research, action research, and interpretive (qualitative) research to strengthen the research design and methodology, this dissertation explores how different stakeholders of two environmental education programs determine program effectiveness. Effectiveness was investigated primarily through the use of in-depth interviews of participants, program sponsors, and professionals. Program success at the professional field level was determined by criteria set by the North American Association for Environmental Education. Characteristics considered in the selection of programs for this study included: method of dissemination, sponsor, subject focus, the method of action which they encourage, and their vision for the future. Using the qualitative case study approach, the Orange County Outdoor Science School in California and Project WILD in Michigan were evaluated. The results show that all levels of stakeholders perceived the two programs to be effective. Areas of effectiveness varied with each program and by stakeholder level. Issues facing each program also differed. At the Orange County Outdoor Science School, program cost was named as the major issue. The most often cited issues for Project WILD was public awareness of the program and stabilizing long term financial support for the Michigan program. Interview data were analyzed at the question level in addition to the thematic level. Themes which emerged from the interview responses included cost of the program, respondent and program receptivity to change, reconciling program philosophy with personal philosophy (Orange County program), perception of environmental education, definition of the program's boundaries and barriers to implementation (Project WILD). The dissertation concludes with a presentation of a preliminary model for environmental education program evaluation and recommendations for future related research.

Hey girlfriend: an evaluation of AIDS prevention among women in the sex industry.

PubMed

Dorfman, L E; Derish, P A; Cohen, J B

1992-01-01

Increasingly, acquired immunodeficiency syndrome (AIDS) prevention programs have been developed to reach and influence street-based populations. Standard methods of evaluation do not fit the conditions of such programs. This article describes a process and outcome evaluation of an AIDS prevention program for sex workers in which qualitative and quantitative methods were combined in order to mediate research problems endemic to street-based populations. Methods included epidemiological questionnaires, open-ended interviews with participants, and ethnographic field notes. Process evaluation findings show that field staff who were indigenous to the neighborhood and population readily gained access to the community of sex workers and simultaneously became role models for positive behavior change. Outcome findings show that sex workers do feel at risk for AIDS, but usually from clients rather than from husbands or boyfriends. Accordingly, they use condoms more frequently with clients than with steady partners. Increasing condom use among sex workers with their steady partners remains an important challenge for AIDS prevention. Combining qualitative and quantitative research data provided a more comprehensive assessment of how to reach sex workers with effective AIDS risk reduction messages than either method could have provided alone.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Russell, J.S.

Several factors in the development of the East Wilmington oil field by THUMS Long Beach Co. are described. These include: critical path scheduling, complex stratigraphy, reservoir engineering, drilling program, production methods, pressure maintenance, crude oil processing, automation, transportation facilities, service lines, and electrical facilities. The complexity and closely scheduled operational events interwoven in the THUMS project demands a method for the carefully planned sequence of jobs to be done, beginning with island construction up through routine production and to the LACT system. These demanding requirements necessitated the use of a critical path scheduling program. It was decided to use themore » program evaluation technique. This technique is used to assign responsibilities for individual assignments to time assignments, and to keep the overall program on schedule. The stratigraphy of East Wilmington complicates all engineering functions associated with recovery methods and reservoir evaluation. At least 5 major faults are anticipated.« less

Training and capacity building evaluation: Maximizing resources and results with Success Case Method.

PubMed

Medina, L; Acosta-Pérez, E; Velez, C; Martínez, G; Rivera, M; Sardiñas, L; Pattatucci, A

2015-10-01

This article describes the use of Success Case Method (Brinkerhoff, 2003) to evaluate health promotion and public health training programs. The goal of the Office Community Research and Engagement (OCRE) of the Puerto Rico Clinical and Translational Research Consortium (PRCTRC) is to establish a stable and sustainable translational research capacity. Early efforts toward achieving this goal included sponsoring two independent research training programs. A description of the implementation of the five step Success Case Method is presented. Results reveal that SCM would deem both trainings as highly successful, based upon the overall impact of a low number of success cases. However, a traditional summative evaluation would consider this disappointing. Strengths of SCM are discussed. It was concluded that the Success Case Method is a useful and valuable evaluative method for measuring the success of health promotion and public health training initiatives and provides sufficient information for decision-making processes. Copyright © 2015 Elsevier Ltd. All rights reserved.

An Evaluation of Kernel Equating: Parallel Equating with Classical Methods in the SAT Subject Tests[TM] Program. Research Report. ETS RR-09-06

ERIC Educational Resources Information Center

Grant, Mary C.; Zhang, Lilly; Damiano, Michele

2009-01-01

This study investigated kernel equating methods by comparing these methods to operational equatings for two tests in the SAT Subject Tests[TM] program. GENASYS (ETS, 2007) was used for all equating methods and scaled score kernel equating results were compared to Tucker, Levine observed score, chained linear, and chained equipercentile equating…

Increasing the use of evaluation data collection in an EPO program

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dordevic, M.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D.; Welti, R.; Davis, H. B.

2017-12-01

Over the past two years, the Incorporated Research Institutions for Seismology Education and Public Outreach (EPO) program has sought to increase the evaluation rigor of its programs and products. Specifically we sought to make evaluation an integral part of our work; enabling staff to demonstrate why we do the activities we do, enhancing the impact or our products/programs, and empowering staff to make evidence-based claims. The Collaborative Impact Analysis Method (Davis and Scalice, 2015) was selected as it allowed us to combine staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make improvements to the programs over time. A key part of the initial action plans has been the collection and analysis of new evaluation data. The most frequently used tools were surveys as they were relatively straightforward to implement and analyze, and could be adapted for different situations. Examples include: brand awareness, value of booth interactions, assessing community interest in a data app, and user surveys of social media and specific web pages. Other evaluation activities included beta testing of new software, and interviews with students and faculty involved in summer field experiences. The surveys have allowed us to document increased impact in some areas, to improve the usability of products and activities, and to provide baseline impact data. The direct involvement of staff in the process has helped staff appreciate the value of evaluation, but there are also challenges to this approach. Since many of the surveys are developed and conducted by EPO staff, rather than being primarily handled by the evaluator, the process takes considerably more staff time to implement. We are still determining how to best manage and present the data and analysis; our current approach is to post evaluation reports on our EPO website so that other groups may be able to benefit from our evaluation results. Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis Method. Abstract ED53D-0871, 2015 Fall Meeting, AGU.

Program Evaluation of a School District's Multisensory Reading Initiative

ERIC Educational Resources Information Center

Asip, Michael Patrick

2012-01-01

The purpose of this study was to conduct a formative program evaluation of a school district's multisensory reading initiative. The mixed methods study involved semi-structured interviews, online survey, focus groups, document review, and analysis of extant special education student reading achievement data. Participants included elementary…

Peace Works: A Program Evaluation Model

ERIC Educational Resources Information Center

LeBlanc, Patrice; Lacey, Candace

2002-01-01

The purpose of this paper is to describe the model used to evaluate the two-year Allegany Grant between the Peace Education Foundation and the Miami-Dade County Public Schools that focused on reducing school violence through implementing the "Peace Works" program. A mixed method design using simultaneous methodologies was employed. For…

New-Teacher Induction 2.0

ERIC Educational Resources Information Center

Taranto, Greg

2011-01-01

The purpose of this program evaluation study was to design, implement, and evaluate the effectiveness of incorporating an online learning community as part of a comprehensive new-teacher induction program. The researcher, who serves as the middle school principal and new induction coordinator for the school district, used a mixed-method approach…

45 CFR 2516.820 - What types of internal evaluation activities are required of programs?

Code of Federal Regulations, 2011 CFR

2011-10-01

... required to: (a) Continuously assess management effectiveness, the quality of services provided, and the... evaluation methods, such as community advisory councils, participant advisory councils, peer reviews, quality... following data: (1) The total number of participants in each program and basic demographic characteristics...

Colorado Even Start. 1998-1999 Progress Report.

ERIC Educational Resources Information Center

Anderson, Beckie

Even Start programs integrate early childhood education, adult literacy or basic education, parenting education and support, and parent and child time to help break the cycle of poverty and illiteracy. This report describes the Even Start program in Colorado, and includes evaluation questions and methods. The report presents evaluation findings…

Perspectives on Evaluation in Financial Education: Landscape, Issues, and Studies

ERIC Educational Resources Information Center

Walstad, William; Urban, Carly; Asarta, Carlos J.; Breitbach, Elizabeth; Bosshardt, William; Heath, Julie; O'Neill, Barbara; Wagner, Jamie; Xiao, Jing Jian

2017-01-01

This review discusses the heterogeneity in the effectiveness of financial education programs that occurs because of the unique conditions for programs and methods to evaluate them. The authors define six groups served by financial education: children, youth, college students and young adults, working adults, military personnel, and low-income…

Evaluation of a Secure Laptop-Based Testing Program in an Undergraduate Nursing Program: Students' Perspective.

PubMed

Tao, Jinyuan; Gunter, Glenda; Tsai, Ming-Hsiu; Lim, Dan

2016-01-01

Recently, the many robust learning management systems, and the availability of affordable laptops, have made secure laptop-based testing a reality on many campuses. The undergraduate nursing program at the authors' university began to implement a secure laptop-based testing program in 2009, which allowed students to use their newly purchased laptops to take quizzes and tests securely in classrooms. After nearly 5 years' secure laptop-based testing program implementation, a formative evaluation, using a mixed method that has both descriptive and correlational data elements, was conducted to seek constructive feedback from students to improve the program. Evaluation data show that, overall, students (n = 166) believed the secure laptop-based testing program helps them get hands-on experience of taking examinations on the computer and gets them prepared for their computerized NCLEX-RN. Students, however, had a lot of concerns about laptop glitches and campus wireless network glitches they experienced during testing. At the same time, NCLEX-RN first-time passing rate data were analyzed using the χ2 test, and revealed no significant association between the two testing methods (paper-and-pencil testing and the secure laptop-based testing) and students' first-time NCLEX-RN passing rate. Based on the odds ratio, however, the odds of students passing NCLEX-RN the first time was 1.37 times higher if they were taught with the secure laptop-based testing method than if taught with the traditional paper-and-pencil testing method in nursing school. It was recommended to the institution that better quality of laptops needs to be provided to future students, measures needed to be taken to further stabilize the campus wireless Internet network, and there was a need to reevaluate the Laptop Initiative Program.

Re-evaluation of Montana's air quality program.

DOT National Transportation Integrated Search

2013-08-01

This project examined the Montana DOTs current methods for determining projects for the Montana Air and Congestion Initiative (MACI) program, and made recommendations to improve and implement this program. A major project objective was to keep the...

Ethics Education in Midwifery Education Programs in the United States.

PubMed

Megregian, Michele

2016-09-01

Familiarity with ethical concepts is a required competency for new graduates and a component of accreditation for midwifery education programs in the United States. While midwifery educators have acknowledged the importance of ethics education in midwifery programs, little is known about current methods, format, or evaluation of ethics education. A Web-based survey was developed for program directors of accredited midwifery education programs in the United States. Clock hours, formats, venues, content topics, barriers, and evaluation methods were evaluated by descriptive analysis. Fifty-one percent of programs completed the online survey (25/49). Of these, only 7 (28%) offer ethics as a stand-alone class, although all responding programs integrate some ethics education into other core classes. Programs show variation in format, venue, resources, and clock hours dedicated to ethics education. The most frequent barrier to ethics education is an already crowded curriculum (60%), although 32% of programs denied any barriers at all. The majority of programs include the ethical concepts of informed consent, shared decision making, and effective communication in curriculum content. This survey found that there is considerable variation in ethics education in terms of content, format, and evaluation among accredited midwifery education programs in the United States. Midwifery educators have an opportunity to explore the ethical dilemmas unique to maternity care from a midwifery perspective. There is also the opportunity to create a comprehensive and dynamic midwifery ethics curriculum, which incorporates both stand-alone ethics courses and ethics concepts that are woven throughout the core midwifery curriculum. © 2016 by the American College of Nurse-Midwives.

Optimizing Department of Defense Acquisition Development Test and Evaluation Scheduling

DTIC Science & Technology

2015-06-01

CPM Critical Path Method DOD Department of Defense DT&E development test and evaluation EMD engineering and manufacturing development GAMS...these, including the Program Evaluation Review Technique (PERT), the Critical Path Method ( CPM ), and the resource- constrained project-scheduling...problem (RCPSP). These are of particular interest to this thesis as the current scheduling method uses elements of the PERT/ CPM , and the test

Pyrochemical and Dry Processing Methods Program. A selected bibliography

DOE Office of Scientific and Technical Information (OSTI.GOV)

McDuffie, H.F.; Smith, D.H.; Owen, P.T.

1979-03-01

This selected bibliography with abstracts was compiled to provide information support to the Pyrochemical and Dry Processing Methods (PDPM) Program sponsored by DOE and administered by the Argonne National Laboratory. Objectives of the PDPM Program are to evaluate nonaqueous methods of reprocessing spent fuel as a route to the development of proliferation-resistant and diversion-resistant methods for widespread use in the nuclear industry. Emphasis was placed on the literature indexed in the ERDA--DOE Energy Data Base (EDB). The bibliography includes indexes to authors, subject descriptors, EDB subject categories, and titles.

Using MathCad to Evaluate Exact Integral Formulations of Spacecraft Orbital Heats for Primitive Surfaces at Any Orientation

NASA Technical Reports Server (NTRS)

Pinckney, John

2010-01-01

With the advent of high speed computing Monte Carlo ray tracing techniques has become the preferred method for evaluating spacecraft orbital heats. Monte Carlo has its greatest advantage where there are many interacting surfaces. However Monte Carlo programs are specialized programs that suffer from some inaccuracy, long calculation times and high purchase cost. A general orbital heating integral is presented here that is accurate, fast and runs on MathCad, a generally available engineering mathematics program. The integral is easy to read, understand and alter. The integral can be applied to unshaded primitive surfaces at any orientation. The method is limited to direct heating calculations. This integral formulation can be used for quick orbit evaluations and spot checking Monte Carlo results.

[Psychological Effects of Forest Therapy Program on Workers].

PubMed

Ikei, Harumi; Koizumi, Haruka; Song, Chorong; Kouzuki, Mitsunori; Teratani, Seiichiro; Sakuma, Takahiro; Miyazaki, Yoshifumi

2015-01-01

To examine the psychological effects of forest therapy program on workers. The subjective symptoms index, a shortened version of the profile of mood states (POMS), and a semantic differential (SD) method were used to measure the psychological effects. The evaluations were performed 3 days before, during, and 1, 3, and 5 days after the forest therapy. The following results were obtained: (1) the subjective symptoms improved before breakfast and continued for 5 days; (2) the mood evaluated using POMS improved before breakfast and continued for 3 days; and (3) “comfortable,” “relaxed,” and “natural” feelings evaluated using the SD method were enhanced before breakfast, lunch, and dinner during forest therapy. These results provided scientific evidence of the psychological effects of forest therapy program on workers.

BOKASUN: A fast and precise numerical program to calculate the Master Integrals of the two-loop sunrise diagrams

NASA Astrophysics Data System (ADS)

Caffo, Michele; Czyż, Henryk; Gunia, Michał; Remiddi, Ettore

2009-03-01

We present the program BOKASUN for fast and precise evaluation of the Master Integrals of the two-loop self-mass sunrise diagram for arbitrary values of the internal masses and the external four-momentum. We use a combination of two methods: a Bernoulli accelerated series expansion and a Runge-Kutta numerical solution of a system of linear differential equations. Program summaryProgram title: BOKASUN Catalogue identifier: AECG_v1_0 Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AECG_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 9404 No. of bytes in distributed program, including test data, etc.: 104 123 Distribution format: tar.gz Programming language: FORTRAN77 Computer: Any computer with a Fortran compiler accepting FORTRAN77 standard. Tested on various PC's with LINUX Operating system: LINUX RAM: 120 kbytes Classification: 4.4 Nature of problem: Any integral arising in the evaluation of the two-loop sunrise Feynman diagram can be expressed in terms of a given set of Master Integrals, which should be calculated numerically. The program provides a fast and precise evaluation method of the Master Integrals for arbitrary (but not vanishing) masses and arbitrary value of the external momentum. Solution method: The integrals depend on three internal masses and the external momentum squared p. The method is a combination of an accelerated expansion in 1/p in its (pretty large!) region of fast convergence and of a Runge-Kutta numerical solution of a system of linear differential equations. Running time: To obtain 4 Master Integrals on PC with 2 GHz processor it takes 3 μs for series expansion with pre-calculated coefficients, 80 μs for series expansion without pre-calculated coefficients, from a few seconds up to a few minutes for Runge-Kutta method (depending on the required accuracy and the values of the physical parameters).

Examining mixing methods in an evaluation of a smoking cessation program.

PubMed

Betzner, Anne; Lawrenz, Frances P; Thao, Mao

2016-02-01

Three different methods were used in an evaluation of a smoking cessation study: surveys, focus groups, and phenomenological interviews. The results of each method were analyzed separately and then combined using both a pragmatic and dialectic stance to examine the effects of different approaches to mixing methods. Results show that the further apart the methods are philosophically, the more diverse the findings. Comparisons of decision maker opinions and costs of the different methods are provided along with recommendations for evaluators' uses of different methods. Copyright © 2015. Published by Elsevier Ltd.

Shortening the Miles to the Milestones: Connecting EPA-Based Evaluations to ACGME Milestone Reports for Internal Medicine Residency Programs.

PubMed

Choe, John H; Knight, Christopher L; Stiling, Rebekah; Corning, Kelli; Lock, Keli; Steinberg, Kenneth P

2016-07-01

The Next Accreditation System requires internal medicine training programs to provide the Accreditation Council for Graduate Medical Education (ACGME) with semiannual information about each resident's progress in 22 subcompetency domains. Evaluation of resident "trustworthiness" in performing entrustable professional activities (EPAs) may offer a more tangible assessment construct than evaluations based on expectations of usual progression toward competence. However, translating results from EPA-based evaluations into ACGME milestone progress reports has proven to be challenging because the constructs that underlay these two systems differ.The authors describe a process to bridge the gap between rotation-specific EPA-based evaluations and ACGME milestone reporting. Developed at the University of Washington in 2012 and 2013, this method involves mapping EPA-based evaluation responses to "milestone elements," the narrative descriptions within the columns of each of the 22 internal medicine subcompetencies. As faculty members complete EPA-based evaluations, the mapped milestone elements are automatically marked as "confirmed." Programs can maintain a database that tallies the number of times each milestone element is confirmed for a resident; these data can be used to produce graphical displays of resident progress along the internal medicine milestones.Using this count of milestone elements allows programs to bridge the gap between faculty assessments of residents based on rotation-specific observed activities and semiannual ACGME reports based on the internal medicine milestones. Although potentially useful for all programs, this method is especially beneficial to large programs where clinical competency committee members may not have the opportunity for direct observation of all residents.

Alternative Methods in the Evaluation of School District Cash Management Programs.

ERIC Educational Resources Information Center

Dembowski, Frederick L.

1980-01-01

Empirically evaluates three measures of effectiveness of school district cash management: the rate of return method in common use and two new measures--efficiency rating and Net Present Value (NPV). The NPV approach allows examination of efficiency and provides a framework for evaluating other areas of educational policy. (Author/IRT)

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program.

PubMed

Jaafaripooyan, Ebrahim

2014-09-01

Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs' over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs.

"Can't We Just Have Some Sazón?" Student, Family, and Staff Perspectives on a New School Food Program at a Boston High School

ERIC Educational Resources Information Center

Chatterjee, Avik; Daftary, Genevieve; Campbell, Meg; Gatison, Lenward; Day, Liam; Ramsey, Kibret; Goldman, Roberta; Gillman, Matthew W.

2016-01-01

Background: In September 2013, a Massachusetts high school launched a nutrition program in line with 2013 United States Department of Agriculture requirements. We sought to understand attitudes of stakeholders toward the new program. Methods: We employed community-based participatory research methods in a qualitative evaluation of the food program…

Culturally Competent Research with American Indians and Alaska Natives: Findings and Recommendations of the First Symposium of the Work Group on American Indian Research and Program Evaluation Methodology

ERIC Educational Resources Information Center

Caldwell, Joyce Y.; Davis, Jamie D.; Du Bois, Barbara; Echo-Hawk, Holly; Erickson, Jill Shepard; Goins, R. Turner; Hill, Calvin; Hillabrant, Walter; Johnson, Sharon R.; Kendall, Elizabeth; Keemer, Kelly; Manson, Spero M.; Marshall, Catherine A.; Running Wolf, Paulette; Santiago, Rolando L.; Schacht, Robert; Stone, Joseph B.

2005-01-01

This article describes the collective experience of a multidisciplinary network of researchers, practitioners, and program evaluators who support appropriate research and evaluation methods in working with Native peoples. Our experience underlines the critical importance of culture in understanding and conducting research with the diverse…

PubMed Central

Worrall, Graham; Chambers, Larry W.

1990-01-01

With the increasing expenditure on health care programs for seniors, there is an urgent need to evaluate such programs. The Measurement Iterative Loop is a tool that can provide both health administrators and health researchers with a method of evaluation of existing programs and identification of gaps in knowledge, and forms a rational basis for health-care policy decisions. In this article, the Loop is applied to one common problem of the elderly: dementia. PMID:21233998

Evaluating employee assistance programs. A review of methods, outcomes, and future directions.

PubMed

Jerrell, J M; Rightmyer, J F

1982-01-01

Renewed interest in assisting troubled employees has led to an upsurge in the development of employee assistance programs, coupled with demands for demonstrable effectiveness. This review examines the nature and scope of these programs, their administrative and methodological context, and the types and outcomes of evaluation studies conducted thus far. Proposals for improving future investigations through a number of different approaches and strategies are then made.

Effectiveness of the Surviving the Teens® Suicide Prevention and Depression Awareness Program: An Impact Evaluation Utilizing a Comparison Group

ERIC Educational Resources Information Center

Strunk, Catherine M.; King, Keith A.; Vidourek, Rebecca A.; Sorter, Michael T.

2014-01-01

Youth suicide is a serious public health issue in the United States. It is currently the third leading cause of death for youth aged 10 to 19. School-based prevention programs may be an effective method of educating youth and enhancing their help-seeking. Most school-based suicide prevention programs have not been rigorously evaluated for their…

Early Lessons Learned from Extramural School Programs That Offer HPV Vaccine

ERIC Educational Resources Information Center

Hayes, Kim A.; Entzel, Pamela; Berger, Wendy; Caskey, Rachel N.; Shlay, Judith C.; Stubbs, Brenda W.; Smith, Jennifer S.; Brewer, Noel T.

2013-01-01

Background: There has been little evaluation of school-located vaccination programs that offer human papillomavirus (HPV) vaccine in US schools without health centers (ie, extramural programs). This article summarizes lessons learned from such programs. Methods: In July to August 2010, 5 programs were identi?ed. Semistructured, in-depth telephone…

Evaluating Effectiveness of Pair Programming as a Teaching Tool in Programming Courses

ERIC Educational Resources Information Center

Faja, Silvana

2014-01-01

This study investigates the effectiveness of pair programming on student learning and satisfaction in introductory programming courses. Pair programming, used in the industry as a practice of an agile development method, can be adopted in classroom settings to encourage peer learning, increase students' social skills, and enhance student…

Development of a Peer Teaching-Assessment Program and a Peer Observation and Evaluation Tool

PubMed Central

Trujillo, Jennifer M.; Barr, Judith; Gonyeau, Michael; Van Amburgh, Jenny A.; Matthews, S. James; Qualters, Donna

2008-01-01

Objectives To develop a formalized, comprehensive, peer-driven teaching assessment program and a valid and reliable assessment tool. Methods A volunteer taskforce was formed and a peer-assessment program was developed using a multistep, sequential approach and the Peer Observation and Evaluation Tool (POET). A pilot study was conducted to evaluate the efficiency and practicality of the process and to establish interrater reliability of the tool. Intra-class correlation coefficients (ICC) were calculated. Results ICCs for 8 separate lectures evaluated by 2-3 observers ranged from 0.66 to 0.97, indicating good interrater reliability of the tool. Conclusion Our peer assessment program for large classroom teaching, which includes a valid and reliable evaluation tool, is comprehensive, feasible, and can be adopted by other schools of pharmacy. PMID:19325963

The application of generalized, cyclic, and modified numerical integration algorithms to problems of satellite orbit computation

NASA Technical Reports Server (NTRS)

Chesler, L.; Pierce, S.

1971-01-01

Generalized, cyclic, and modified multistep numerical integration methods are developed and evaluated for application to problems of satellite orbit computation. Generalized methods are compared with the presently utilized Cowell methods; new cyclic methods are developed for special second-order differential equations; and several modified methods are developed and applied to orbit computation problems. Special computer programs were written to generate coefficients for these methods, and subroutines were written which allow use of these methods with NASA's GEOSTAR computer program.

Mixed method evaluation of a community-based physical activity program using the RE-AIM framework: practical application in a real-world setting.

PubMed

Koorts, Harriet; Gillison, Fiona

2015-11-06

Communities are a pivotal setting in which to promote increases in child and adolescent physical activity behaviours. Interventions implemented in these settings require effective evaluation to facilitate translation of findings to wider settings. The aims of this paper are to i) present findings from a RE-AIM evaluation of a community-based physical activity program, and ii) review the methodological challenges faced when applying RE-AIM in practice. A single mixed-methods case study was conducted based on a concurrent triangulation design. Five sources of data were collected via interviews, questionnaires, archival records, documentation and field notes. Evidence was triangulated within RE-AIM to assess individual and organisational-level program outcomes. Inconsistent availability of data and a lack of robust reporting challenged assessment of all five dimensions. Reach, Implementation and setting-level Adoption were less successful, Effectiveness and Maintenance at an individual and organisational level were moderately successful. Only community-level Adoption was highly successful, reflecting the key program goal to provide community-wide participation in sport and physical activity. This research highlighted important methodological constraints associated with the use of RE-AIM in practice settings. Future evaluators wishing to use RE-AIM may benefit from a mixed-method triangulation approach to offset challenges with data availability and reliability.

Review of the Agard S&M panel evaluation program of the NASA-Lewis 'SRP' approach to high-temperature LCF life prediction. [Strainrange Partitioning for Low Cycle Fatigue

NASA Technical Reports Server (NTRS)

Hirschberg, M. H.

1978-01-01

Twenty laboratories in six countries participated in this program, each testing its own materials of interest under its own laboratory conditions. In this way the results obtained provided validation of the Strainrange Partitioning (SRP) method for a wide range of materials and insured maximum usefulness to each of the participating laboratories. The first, very necessary step in the evaluation of any life prediction approach - assessing the ability of the method to predict life of simple laboratory specimens subjected to complex loading, was thereby taken. The culmination of this program was the Specialists Meeting that was held in Aalborg, Denmark in April of 1978. At that meeting the various investigators shared their findings, thus providing the basis for an in-depth evaluation of the SRP method. While the results were variable from laboratory to laboratory, most investigators agreed that the SRP method was a significant step toward life prediction in the presence of high temperature and cyclic stresses.

Center for Nondestructive Evaluation - Center for Nondestructive Evaluation

Science.gov Websites

available for the full range of inspection methods, housed in a 52,000 sq. ft. facility with over $5M in - 1990): Development of NDE methods for application to DOE energy and weapons programs, including multi for enhanced frequency bandwidth and improved flaw reconstruction, and novel methods for poling

Draft version 1.0 final report : evaluation methods and lessons learned from the Minnesota Department of Transportation intelligent vehicle initiative field operational test

DOT National Transportation Integrated Search

2003-09-26

This report on the Evaluation Methods and Lessons Learned for the Mn/DOT Intelligent Vehicle Initiative (IVI) Field Operational Test (FOT) documents the goals and objectives, research approach, methods, and findings of a program to measure the feasib...

Evaluating Ethanol-based Sample Preservation to Facilitate Use of DNA Barcoding in Routine Freshwater Biomonitoring Programs Using Benthic Macroinvertebrates

EPA Science Inventory

Molecular methods, such as DNA barcoding, have the potential in enhance biomonitoring programs worldwide. Altering routinely used sample preservation methods to protect DNA from degradation may pose a potential impediment to application of DNA barcoding and metagenomics for biom...

Development, implementation and evaluation of a clinical research engagement and leadership capacity building program in a large Australian health care service.

PubMed

Misso, Marie L; Ilic, Dragan; Haines, Terry P; Hutchinson, Alison M; East, Christine E; Teede, Helena J

2016-01-14

Health professionals need to be integrated more effectively in clinical research to ensure that research addresses clinical needs and provides practical solutions at the coal face of care. In light of limited evidence on how best to achieve this, evaluation of strategies to introduce, adapt and sustain evidence-based practices across different populations and settings is required. This project aims to address this gap through the co-design, development, implementation, evaluation, refinement and ultimately scale-up of a clinical research engagement and leadership capacity building program in a clinical setting with little to no co-ordinated approach to clinical research engagement and education. The protocol is based on principles of research capacity building and on a six-step framework, which have previously led to successful implementation and long-term sustainability. A mixed methods study design will be used. Methods will include: (1) a review of the literature about strategies that engage health professionals in research through capacity building and/or education in research methods; (2) a review of existing local research education and support elements; (3) a needs assessment in the local clinical setting, including an online cross-sectional survey and semi-structured interviews; (4) co-design and development of an educational and support program; (5) implementation of the program in the clinical environment; and (6) pre- and post-implementation evaluation and ultimately program scale-up. The evaluation focuses on research activity and knowledge, attitudes and preferences about clinical research, evidence-based practice and leadership and post implementation, about their satisfaction with the program. The investigators will evaluate the feasibility and effect of the program according to capacity building measures and will revise where appropriate prior to scale-up. It is anticipated that this clinical research engagement and leadership capacity building program will enable and enhance clinically relevant research to be led and conducted by health professionals in the health setting. This approach will also encourage identification of areas of clinical uncertainty and need that can be addressed through clinical research within the health setting.

Steps towards incorporating heterogeneities into program theory: A case study of a data-driven approach.

PubMed

Sridharan, Sanjeev; Jones, Bobby; Caudill, Barry; Nakaima, April

2016-10-01

This paper describes a framework that can help refine program theory through data explorations and stakeholder dialogue. The framework incorporates the following steps: a recognition that program implementation might need to be multi-phased for a number of interventions, the need to take stock of program theory, the application of pattern recognition methods to help identify heterogeneous program mechanisms, and stakeholder dialogue to refine the program. As part of the data exploration, a method known as developmental trajectories is implemented to learn about heterogeneous trajectories of outcomes in longitudinal evaluations. This method identifies trajectory clusters and also can estimate different treatment impacts for the various groups. The framework is highlighted with data collected in an evaluation of an alcohol risk-reduction program delivered in a college fraternity setting. The framework discussed in the paper is informed by a realist focus on "what works for whom under what contexts." The utility of the framework in contributing to a dialogue on heterogeneous mechanism and subsequent implementation is described. The connection of the ideas in paper to a 'learning through principled discovery' approach is also described. Copyright © 2016. Published by Elsevier Ltd.

Clinical peer review program self-evaluation for US hospitals.

PubMed

Edwards, Marc T

2010-01-01

Prior research has shown wide variation in clinical peer review program structure, process, governance, and perceived effectiveness. This study sought to validate the utility of a Peer Review Program Self-Evaluation Tool as a potential guide to physician and hospital leaders seeking greater program value. Data from 330 hospitals show that the total score from the self-evaluation tool is strongly associated with perceived quality impact. Organizational culture also plays a significant role. When controlling for these factors, there was no evidence of benefit from a multispecialty review process. Physicians do not generally use reliable methods to measure clinical performance. A high rate of change since 2007 has not produced much improvement. The Peer Review Program Self-Evaluation Tool reliably differentiates hospitals along a continuum of perceived program performance. The full potential of peer review as a process to improve the quality and safety of care has yet to be realized.

Development of test methods for textile composites

NASA Technical Reports Server (NTRS)

Masters, John E.; Ifju, Peter G.; Fedro, Mark J.

1993-01-01

NASA's Advanced Composite Technology (ACT) Program was initiated in 1990 with the purpose of developing less costly composite aircraft structures. A number of innovative materials and processes were evaluated as a part of this effort. Chief among them are composite materials reinforced with textile preforms. These new forms of composite materials bring with them potential testing problems. Methods currently in practice were developed over the years for composite materials made from prepreg tape or simple 2-D woven fabrics. A wide variety of 2-D and 3-D braided, woven, stitched, and knit preforms were suggested for application in the ACT program. The applicability of existing test methods to the wide range of emerging materials bears investigation. The overriding concern is that the values measured are accurate representations of the true material response. The ultimate objective of this work is to establish a set of test methods to evaluate the textile composites developed for the ACT Program.

Evaluation Program on the Implementation of Industrial Apprenticeship (Prakerin) in Electrical Engineering

NASA Astrophysics Data System (ADS)

Maulana, I.; Sumarto; Nurafiati, P.; Puspita, R. H.

2018-02-01

This research aims to find out the evaluation program of the Industrial apprenticeship (Prakerin) in electrical engineering. This research includes on four variables of CIPP. (1). Context (a). programme planning (b). design. (2). Input (a). readiness of students (b). performance of vocational education teachers (c). Facilities and infrastructure, (3). process (a). performance students (b). performance mentors, (4). Product (a). readiness of student work. This research is a type of program evaluation research with Stake model approach. Data collection methods used are questionnaires with closed questions and frequently asked questions.

Process evaluation of the Living Green, Healthy and Thrifty (LiGHT) web-based child obesity management program: combining health promotion with ecology and economy.

PubMed

Jogova, Maria; Song, Joshua Eun-Soo; Campbell, Audrey Clare; Warbuton, Darren; Warshawski, Tom; Chanoine, Jean-Pierre

2013-04-01

To conduct a process evaluation of the Living Green, Healthy and Thrifty (LiGHT) program, a novel virtual child obesity management program that combines health promotion with ecology and economy (Phase 1). We carried out a mixed methods process evaluation involving qualitative and quantitative data collection in 3 phases: among 3 child-parent units, (group 1) that informed program development; 9 child-parent units (group 2) that tested the draft program and further aided program refinement; and 17 child-parent units (group 3) for a 4-week pilot of the program. In the program pilot, we assessed participants' knowledge and readiness to change pre- and postintervention and explored perceptions of the program. Participants generally felt that the online format for program delivery was convenient and accessible, the content was practical, and the integration of health-environment-economy was well received. Many parents also appreciated the involvement of the family. However, the lack of visual appeal and overabundance of text was identified as a challenge, and children/youth in particular requested assurance that their personal information (e.g. weight) was not seen by their parents. The online method of program delivery holds the unique challenge of requiring special efforts to create a sense of personal connection and community. The presence of a "Way-finder" to assist participants and discussion boards/forums are potential solutions. The LiGHT online weight management program offers an accessible, convenient weight management resource that children and families appreciate for its availability, broader educational scope, and practicality. Outcome evaluation of LiGHT will be carried out in Phase 2 of the project. Copyright © 2013 Canadian Diabetes Association. Published by Elsevier Inc. All rights reserved.

A program needs-driven approach to selecting dietary assessment methods for decision-making in food fortification programs.

PubMed

Coates, Jennifer; Colaiezzi, Brooke; Fiedler, John L; Wirth, James; Lividini, Keith; Rogers, Beatrice

2012-09-01

Dietary assessment data are essential for designing, monitoring, and evaluating food fortification and other food-based nutrition programs. Planners and managers must understand the validity, usefulness, and cost tradeoffs of employing alternative dietary assessment methods, but little guidance exists. To identify and apply criteria to assess the tradeoffs of using alternative dietary methods for meeting fortification programming needs. Twenty-five semistructured expert interviews were conducted and literature was reviewed for information on the validity, usefulness, and cost of using 24-hour recalls, Food Frequency Questionnaires/Fortification Rapid Assessment Tool (FFQ/FRAT), Food Balance Sheets (FBS), and Household Consumption and Expenditures Surveys (HCES) for program stage-specific information needs. Criteria were developed and applied to construct relative rankings of the four methods. Needs assessment: HCES offers the greatest suitability at the lowest cost for estimating the risk of inadequate intakes, but relative to 24-hour recall compromises validity. HCES should be used to identify vehicles and to estimate coverage and likely impact due to its low cost and moderate-to-high validity. Baseline assessment: 24-hour recall should be applied using a representative sample. Monitoring: A simple, low-cost FFQ can be used to monitor coverage. Impact evaluation: 24-hour recall should be used to assess changes in nutrient intakes. FBS have low validity relative to other methods for all programmatic purposes. Each dietary assessment method has strengths and weaknesses that vary by context and purpose. Method selection must be driven by the program's data needs, the suitability of the methods for the purpose, and a clear understanding of the tradeoffs involved.

Repeatability of riparian vegetation sampling methods: how useful are these techniques for broad-scale, long-term monitoring?

Treesearch

Marc C. Coles-Ritchie; Richard C. Henderson; Eric K. Archer; Caroline Kennedy; Jeffrey L. Kershner

2004-01-01

Tests were conducted to evaluate variability among observers for riparian vegetation data collection methods and data reduction techniques. The methods are used as part of a largescale monitoring program designed to detect changes in riparian resource conditions on Federal lands. Methods were evaluated using agreement matrices, the Bray-Curtis dissimilarity metric, the...

Leveraging Administrative Data for Program Evaluations: A Method for Linking Data Sets Without Unique Identifiers.

PubMed

Lorden, Andrea L; Radcliff, Tiffany A; Jiang, Luohua; Horel, Scott A; Smith, Matthew L; Lorig, Kate; Howell, Benjamin L; Whitelaw, Nancy; Ory, Marcia

2016-06-01

In community-based wellness programs, Social Security Numbers (SSNs) are rarely collected to encourage participation and protect participant privacy. One measure of program effectiveness includes changes in health care utilization. For the 65 and over population, health care utilization is captured in Medicare administrative claims data. Therefore, methods as described in this article for linking participant information to administrative data are useful for program evaluations where unique identifiers such as SSN are not available. Following fuzzy matching methodologies, participant information from the National Study of the Chronic Disease Self-Management Program was linked to Medicare administrative data. Linking variables included participant name, date of birth, gender, address, and ZIP code. Seventy-eight percent of participants were linked to their Medicare claims data. Linking program participant information to Medicare administrative data where unique identifiers are not available provides researchers with the ability to leverage claims data to better understand program effects. © The Author(s) 2014.

Pilot Evaluation of a Home Visit Parent Training Program in Disadvantaged Families

ERIC Educational Resources Information Center

Leung, Cynthia; Tsang, Sandra; Heung, Kitty

2013-01-01

Objectives: The study reported the pilot evaluation of the Healthy Start Home Visit Program for disadvantaged Chinese parents with preschool children, delivered by trained parent assistants. Home visiting was used to make services more accessible to disadvantaged families. Method: The participants included 21 parent-child dyads. Outcome measures…

Computer-Assisted, Programmed Text, and Lecture Modes of Instruction in Three Medical Training Courses: Comparative Evaluation. Final Report.

ERIC Educational Resources Information Center

Deignan, Gerard M.; And Others

This report contains a comparative analysis of the differential effectiveness of computer-assisted instruction (CAI), programmed instructional text (PIT), and lecture methods of instruction in three medical courses--Medical Laboratory, Radiology, and Dental. The summative evaluation includes (1) multiple regression analyses conducted to predict…

Evaluating the Effects of Scripted Distributed Pair Programming on Student Performance and Participation

ERIC Educational Resources Information Center

Tsompanoudi, Despina; Satratzemi, Maya; Xinogalos, Stelios

2016-01-01

The results presented in this paper contribute to research on two different areas of teaching methods: distributed pair programming (DPP) and computer-supported collaborative learning (CSCL). An evaluation study of a DPP system that supports collaboration scripts was conducted over one semester of a computer science course. Seventy-four students…

The Administration of Programs for Educable Retarded Children in Small School Systems.

ERIC Educational Resources Information Center

Erdman, Robert L.; And Others

Administrative concerns of programs for educable mentally retarded children in small school systems are discussed beginning with preliminary steps of program promotion and development of program rationale. The selection of children is explored in areas of screening methods, individual evaluations, implications of student characteristics, and…

WAVAI Curriculum Guide for Instruction in Wisconsin High Schools. 1975 Revision. Bulletin No. 5011.

ERIC Educational Resources Information Center

Wisconsin State Dept. of Public Instruction, Madison. Bureau for Career and Manpower Development.

This guide is designed to aid the development of vocational education programs in agriculture in local school districts. Major program objectives, good learning conditions, program evaluation, suggested teaching methods, and Wisconsin's philosophy and procedures in developing occupational experience programs in vocational agriculture/agribusiness…

Workplace Education Partnerships: Compromises, Promises, and Practices.

ERIC Educational Resources Information Center

Hellman, Linda

A Pima County (Arizona) adult workplace education program in English as a Second Language (ESL) is outlined. Salient elements in program design, student recruitment and program marketing, student assessment, instructional methods, and program evaluation are noted, and the topical content of the ESL curriculum is charted. Tasks/competencies and…

A CONTROLLED EVALUATION OF A HIGH SCHOOL BIOMEDICAL PIPELINE PROGRAM: DESIGN AND METHODS

PubMed Central

Winkleby, Marilyn A.; Ned, Judith; Ahn, David; Koehler, Alana; Fagliano, Kathleen; Crump, Casey

2013-01-01

Given limited funding for school-based science education, non-school-based programs have been developed at colleges and universities to increase the number of students entering science- and health-related careers and address critical workforce needs. However, few evaluations of such programs have been conducted. We report the design and methods of a controlled trial to evaluate the Stanford Medical Youth Science Program’s Summer Residential Program (SRP), a 25-year-old university-based biomedical pipeline program. This 5-year matched cohort study uses an annual survey to assess educational and career outcomes among four cohorts of students who participate in the SRP and a matched comparison group of applicants who were not chosen to participate in the SRP. Matching on sociodemographic and academic background allows control for potential confounding. This design enables the testing of whether the SRP has an independent effect on educational- and career-related outcomes above and beyond the effects of other factors such as gender, ethnicity, socioeconomic background, and pre-intervention academic preparation. The results will help determine which curriculum components contribute most to successful outcomes and which students benefit most. After 4 years of follow-up, the results demonstrate high response rates from SRP participants and the comparison group with completion rates near 90%, similar response rates by gender and ethnicity, and little attrition with each additional year of follow-up. This design and methods can potentially be replicated to evaluate and improve other biomedical pipeline programs, which are increasingly important for equipping more students for science- and health-related careers. PMID:24563603

Canadian Pacific Railway Investigation of Safety-Related Occurrences Protocol considered helpful by both labor and management.

DOT National Transportation Integrated Search

2006-09-01

The Federal Railroad Administration (FRA) Human Factors Research and Development (R&D) Program sponsored an Alternative Safety Measures Program designed to explore alternative methods for evaluating whether safety programs improve safety outcomes and...

The fuel tax compliance unit : an evaluation and analysis of results.

DOT National Transportation Integrated Search

2004-01-01

Kentucky utilized TEA-21 federal funds to create an innovative pilot program to identify the best practices and methods for auditing taxpayers of transportation related taxes. This program involved a four-year experimental program called the Fuel Tax...

Using a Non-Equivalent Groups Quasi Experimental Design to Reduce Internal Validity Threats to Claims Made by Math and Science K-12 Teacher Recruitment Programs

NASA Astrophysics Data System (ADS)

Moin, Laura

2009-10-01

The American Recovery and Reinvestment Act national policy established in 2009 calls for ``meaningful data'' that demonstrate educational improvements, including the recruitment of high-quality teachers. The scant data available and the low credibility of many K-12 math/science teacher recruitment program evaluations remain the major barriers for the identification of effective recruitment strategies. Our study presents a methodology to better evaluate the impact of recruitment programs on increasing participants' interest in teaching careers. The research capitalizes on the use of several control groups and presents a non-equivalent groups quasi-experimental evaluation design that produces program effect claims with higher internal validity than claims generated by current program evaluations. With this method that compares responses to a teaching career interest question from undergraduates all along a continuum from just attending an information session to participating (or not) in the recruitment program, we were able to compare the effect of the program in increasing participants' interest in teaching careers versus the evolution of the same interest but in the absence of the program. We were also able to make suggestions for program improvement and further research. While our findings may not apply to other K-12 math/science teacher recruitment programs, we believe that our evaluation methodology does and will contribute to conduct stronger program evaluations. In so doing, our evaluation procedure may inform recruitment program designers and policy makers.

Process evaluation of the project P.A.T.H.S. (secondary 2 program): findings based on the co-walker scheme.

PubMed

Shek, Daniel T L; Tam, Suet-yan

2009-01-01

To understand the implementation quality of the Tier 1 Program (Secondary 2 Curriculum) of the P.A.T.H.S. Project, process evaluation was carried out by co-walkers through classroom observation of 195 units in 131 schools. Results showed that the overall level of program adherence was generally high with an average of 84.55%, and different factors of the implementation process were evaluated as positive. Quality of program implementation and achievement of program objectives were predicted by students' participation and involvement, strategies to enhance students' motivation, opportunity for reflection, time management, and class preparation. Success in program implementation was predicted by students' participation and involvement, classroom control, interactive delivery method, strategies to enhance students' motivation, opportunity for reflection, and lesson preparation.

Program Planners’ Perspectives of Promotora Roles, Recruitment, and Selection

PubMed Central

Koskan, Alexis; Hilfinger Messias, DeAnne K.; Friedman, Daniela B.; Brandt, Heather M.; Walsemann, Katrina M.

2013-01-01

Objective Program planners work with promotoras (the Spanish term for female community health workers) to reduce health disparities among underserved populations. Based on the Role-Outcomes Linkage Evaluation Model for Community Health Workers (ROLES) conceptual model, we explored how program planners conceptualized the promotora role and the approaches and strategies they used to recruit, select, and sustain promotoras. Design We conducted semi-structured, in-depth interviews with a purposive convenience sample of 24 program planners, program coordinators, promotora recruiters, research principal investigators, and other individuals who worked closely with promotoras on United States-based health programs for Hispanic women (ages 18 and older). Results Planners conceptualized the promotora role based on their personal experiences and their understanding of the underlying philosophical tenets of the promotora approach. Recruitment and selection methods reflected planners’ conceptualizations and experiences of promotoras as paid staff or volunteers. Participants described a variety of program planning and implementation methods. They focused on sustainability of the programs, the intended health behavior changes or activities, and the individual promotoras. Conclusion To strengthen health programs employing the promotora delivery model, job descriptions should delineate role expectations and boundaries and better guide promotora evaluations. We suggest including additional components such as information on funding sources, program type and delivery, and sustainability outcomes to enhance the ROLES conceptual model. The expanded model can be used to guide program planners in the planning, implementing, and evaluating of promotora health programs. PMID:23039847

A semi-automated tool for treatment plan-quality evaluation and clinical trial quality assurance

NASA Astrophysics Data System (ADS)

Wang, Jiazhou; Chen, Wenzhou; Studenski, Matthew; Cui, Yunfeng; Lee, Andrew J.; Xiao, Ying

2013-07-01

The goal of this work is to develop a plan-quality evaluation program for clinical routine and multi-institutional clinical trials so that the overall evaluation efficiency is improved. In multi-institutional clinical trials evaluating the plan quality is a time-consuming and labor-intensive process. In this note, we present a semi-automated plan-quality evaluation program which combines MIMVista, Java/MATLAB, and extensible markup language (XML). More specifically, MIMVista is used for data visualization; Java and its powerful function library are implemented for calculating dosimetry parameters; and to improve the clarity of the index definitions, XML is applied. The accuracy and the efficiency of the program were evaluated by comparing the results of the program with the manually recorded results in two RTOG trials. A slight difference of about 0.2% in volume or 0.6 Gy in dose between the semi-automated program and manual recording was observed. According to the criteria of indices, there are minimal differences between the two methods. The evaluation time is reduced from 10-20 min to 2 min by applying the semi-automated plan-quality evaluation program.

Solar thermal storage applications program

NASA Astrophysics Data System (ADS)

Peila, W. C.

1982-12-01

The efforts of the Storage Applications Program are reviewed. The program concentrated on the investigation of storage media and evaluation of storage methods. Extensive effort was given to experimental and analytical investigations of nitrate salts. Two tasks are the preliminary design of a 1200 MW/sub th/ system and the design, construction, operation, and evaluation of a subsystem research experiment, which utilized the same design. Some preliminary conclusions drawn from the subsystem research experiment are given.

Using Elite Athletes to Promote Drug Abstinence: Evaluation of a Single-Session School-Based Drug Use Prevention Program Delivered by Junior Hockey Players

ERIC Educational Resources Information Center

Wong, Jennifer

2016-01-01

School-based substance use prevention programs are a common method to approaching drug use in youths. Project SOS is a single-session drug prevention program developed by police officers and delivered by elite junior hockey players to students in grades 6 and 7. The current study evaluates the effects of Project SOS at achieving its objectives of…

Enhancing pediatric residents’ scholar role: the development of a Scholarly Activity Guidance and Evaluation program

PubMed Central

Pound, Catherine M.; Moreau, Katherine A.; Ward, Natalie; Eady, Kaylee; Writer, Hilary

2015-01-01

Background Research training is essential to the development of well-rounded physicians. Although many pediatric residency programs require residents to complete a research project, it is often challenging to integrate research training into educational programs. Objective We aimed to develop an innovative research program for pediatric residents, called the Scholarly Activity Guidance and Evaluation (SAGE) program. Methods We developed a competency-based program which establishes benchmarks for pediatric residents, while providing ongoing academic mentorship. Results Feedback from residents and their research supervisors about the SAGE program has been positive. Preliminary evaluation data have shown that all final-year residents have met or exceeded program expectations. Conclusions By providing residents with this supportive environment, we hope to influence their academic career paths, increase their research productivity, promote evidence-based practice, and ultimately, positively impact health outcomes. PMID:26059213

Flipping one-shot library instruction: using Canvas and Pecha Kucha for peer teaching*†

PubMed Central

Carroll, Alexander J.; Tchangalova, Nedelina; Harrington, Eileen G.

2016-01-01

Objective This study sought to determine whether a flipped classroom that facilitated peer learning would improve undergraduate health sciences students' abilities to find, evaluate, and use appropriate evidence for research assignments. Methods Students completed online modules in a learning management system, with librarians facilitating subsequent student-directed, in-person sessions. Mixed methods assessment was used to evaluate program outcomes. Results Students learned information literacy concepts but did not consistently apply them in research assignments. Faculty interviews revealed strengthened partnerships between librarians and teaching faculty. Conclusion This pedagogy shows promise for implementing and evaluating a successful flipped information literacy program. PMID:27076799

Chapter 6: Residential Lighting Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Dimetrosky, Scott; Parkinson, Katie

Given new regulations, increased complexity in the market, and the general shift from CFLs to LEDs, this evaluation protocol was updated in 2017 to shift the focus of the protocols toward LEDs and away from CFLs and to resolve evaluation uncertainties affecting residential lighting incentive programs.

Problems in Evaluating Adult Literacy Programmes.

ERIC Educational Resources Information Center

Rajyalakshmi, C.

1980-01-01

The study of the Functional Literacy Program in India encountered problems of opposition to evaluation, inappropriate respondents, and difficulties in contacting participants, which point to changes that should be made in the evaluation methods. (SK)

Ohio's Abandoned Mine Lands Reclamation Program: a Study of Data Collection and Evaluation Techniques

NASA Technical Reports Server (NTRS)

Sperry, S. L.

1982-01-01

The planning process for a statewide reclamation plan of Ohio abandoned minelands in response to the Federal Surface Mining Control and Reclamation Act of 1977 included: (1) the development of a screening and ranking methodology; (2) the establishment of a statewide review of major watersheds affected by mining; (3) the development of an immediate action process; and (4) a prototypical study of a priority watershed demonstrating the data collection, analysis, display and evaluation to be used for the remaining state watersheds. Historical methods for satisfying map information analysis and evaluation, as well as current methodologies being used were discussed. Various computer mapping and analysis programs were examined for their usability in evaluating the priority reclamation sites. Hand methods were chosen over automated procedures; intuitive evaluation was the primary reason.

Studying the teaching of kindness: A conceptual model for evaluating kindness education programs in schools.

PubMed

Kaplan, Deanna M; deBlois, Madeleine; Dominguez, Violeta; Walsh, Michele E

2016-10-01

Recent research suggests that school-based kindness education programs may benefit the learning and social-emotional development of youth and may improve school climate and school safety outcomes. However, how and to what extent kindness education programming influences positive outcomes in schools is poorly understood, and such programs are difficult to evaluate in the absence of a conceptual model for studying their effectiveness. In partnership with Kind Campus, a widely adopted school-based kindness education program that uses a bottom-up program framework, a methodology called concept mapping was used to develop a conceptual model for evaluating school-based kindness education programs from the input of 123 middle school students and approximately 150 educators, school professionals, and academic scholars. From the basis of this model, recommendations for processes and outcomes that would be useful to assess in evaluations of kindness education programs are made, and areas where additional instrument development may be necessary are highlighted. The utility of the concept mapping method as an initial step in evaluating other grassroots or non-traditional educational programming is also discussed. Copyright © 2016 Elsevier Ltd. All rights reserved.

PAREMD: A parallel program for the evaluation of momentum space properties of atoms and molecules

NASA Astrophysics Data System (ADS)

Meena, Deep Raj; Gadre, Shridhar R.; Balanarayan, P.

2018-03-01

The present work describes a code for evaluating the electron momentum density (EMD), its moments and the associated Shannon information entropy for a multi-electron molecular system. The code works specifically for electronic wave functions obtained from traditional electronic structure packages such as GAMESS and GAUSSIAN. For the momentum space orbitals, the general expression for Gaussian basis sets in position space is analytically Fourier transformed to momentum space Gaussian basis functions. The molecular orbital coefficients of the wave function are taken as an input from the output file of the electronic structure calculation. The analytic expressions of EMD are evaluated over a fine grid and the accuracy of the code is verified by a normalization check and a numerical kinetic energy evaluation which is compared with the analytic kinetic energy given by the electronic structure package. Apart from electron momentum density, electron density in position space has also been integrated into this package. The program is written in C++ and is executed through a Shell script. It is also tuned for multicore machines with shared memory through OpenMP. The program has been tested for a variety of molecules and correlated methods such as CISD, Møller-Plesset second order (MP2) theory and density functional methods. For correlated methods, the PAREMD program uses natural spin orbitals as an input. The program has been benchmarked for a variety of Gaussian basis sets for different molecules showing a linear speedup on a parallel architecture.

Satellite Power System (SPS) financial/management scenarios

NASA Technical Reports Server (NTRS)

Vajk, J. P.

1978-01-01

The possible benefits of a Satellite Power System (SPS) program, both domestically and internationally, justify detailed and imaginative investigation of the issues involved in financing and managing such a large-scale program. In this study, ten possible methods of financing a SPS program are identified ranging from pure government agency to private corporations. The following were analyzed and evaluated: (1) capital requirements for SPS; (2) ownership and control; (3) management principles; (4) organizational forms for SPS; (5) criteria for evaluation; (6) detailed description and preliminary evaluation of alternatives; (7) phased approaches; and (8) comparative evaluation. Key issues and observations and recommendations for further study are also presented.

Deductive Evaluation: Formal Code Analysis With Low User Burden

NASA Technical Reports Server (NTRS)

Di Vito, Ben. L

2016-01-01

We describe a framework for symbolically evaluating iterative C code using a deductive approach that automatically discovers and proves program properties. Although verification is not performed, the method can infer detailed program behavior. Software engineering work flows could be enhanced by this type of analysis. Floyd-Hoare verification principles are applied to synthesize loop invariants, using a library of iteration-specific deductive knowledge. When needed, theorem proving is interleaved with evaluation and performed on the fly. Evaluation results take the form of inferred expressions and type constraints for values of program variables. An implementation using PVS (Prototype Verification System) is presented along with results for sample C functions.

Did we get our money's worth? Bridging economic and behavioral measures of program success in adolescent drug prevention.

PubMed

Griffith, Kevin N; Scheier, Lawrence M

2013-11-08

The recent U.S. Congressional mandate for creating drug-free learning environments in elementary and secondary schools stipulates that education reform rely on accountability, parental and community involvement, local decision making, and use of evidence-based drug prevention programs. By necessity, this charge has been paralleled by increased interest in demonstrating that drug prevention programs net tangible benefits to society. One pressing concern is precisely how to integrate traditional scientific methods of program evaluation with economic measures of "cost efficiency". The languages and methods of each respective discipline don't necessarily converge on how to establish the true benefits of drug prevention. This article serves as a primer for conducting economic analyses of school-based drug prevention programs. The article provides the reader with a foundation in the relevant principles, methodologies, and benefits related to conducting economic analysis. Discussion revolves around how economists value the potential costs and benefits, both financial and personal, from implementing school-based drug prevention programs targeting youth. Application of heterogeneous costing methods coupled with widely divergent program evaluation findings influences the feasibility of these techniques and may hinder utilization of these practices. Determination of cost-efficiency should undoubtedly become one of several markers of program success and contribute to the ongoing debate over health policy.

The Careers in Health and Medical Professions Program (CHAMPS): An Impact Study of a University-Based STEM+H Outreach Program

NASA Astrophysics Data System (ADS)

Wallace, Eric W.; Perry, Justin C.; Ferguson, Robert L.; Jackson, Debbie K.

2015-08-01

The present study investigated the impact of a Science, Technology, Engineering, Mathematics and Health (STEM+H) university-based pipeline program, the Careers in Health and Medical Professions Program, over the course of two summers among predominantly African-American high school students recruited from urban school districts ( N = 155). Based on a mixed methods approach, results indicated that youth made significant gains in both academic and career knowledge. Furthermore, youth generally rated the program's sessions favorably, but also rated sessions with varying levels of satisfaction. The limitations and implications for program delivery and evaluation methods among pipeline programs are discussed.

New England Instructional Television Research Center (NETREC).

ERIC Educational Resources Information Center

Friedlander, Bernard Z.; Wetstone, Harriet S.

Projects of the New England Instructional Television Research Center (NITREC) are summarized in a collection of papers. Objectives, rationale, and program of NETREC are defined, along with methods of formative evaluation during production. Seven videotest research projects cover methods of evaluating communicative effectiveness of primary-grade…

Methods of assessing structural integrity for space shuttle vehicles

NASA Technical Reports Server (NTRS)

Anderson, R. E.; Stuckenberg, F. H.

1971-01-01

A detailed description and evaluation of nondestructive evaluation (NDE) methods are given which have application to space shuttle vehicles. Appropriate NDE design data is presented in twelve specifications in an appendix. Recommendations for NDE development work for the space shuttle program are presented.

Pharmacists’ Perception of Synchronous Versus Asynchronous Distance Learning for Continuing Education Programs

PubMed Central

2014-01-01

Objective. To evaluate and compare pharmacists’ satisfaction with the content and learning environment of a continuing education program series offered as either synchronous or asynchronous webinars. Methods. An 8-lecture series of online presentations on the topic of new drug therapies was offered to pharmacists in synchronous and asynchronous webinar formats. Participants completed a 50-question online survey at the end of the program series to evaluate their perceptions of the distance learning experience. Results. Eighty-two participants completed the survey instrument (41 participants from the live webinar series and 41 participants from the asynchronous webinar series.) Responses indicated that while both groups were satisfied with the program content, the asynchronous group showed greater satisfaction with many aspects of the learning environment. Conclusion. The synchronous and asynchronous webinar participants responded positively regarding the quality of the programming and the method of delivery, but asynchronous participants rated their experience more positively overall. PMID:24558276

Program evaluation of an Integrated Basic Science Medical Curriculum in Shiraz Medical School, Using CIPP Evaluation Model

PubMed Central

ROOHOLAMINI, AZADEH; AMINI, MITRA; BAZRAFKAN, LEILA; DEHGHANI, MOHAMMAD REZA; ESMAEILZADEH, ZOHREH; NABEIEI, PARISA; REZAEE, RITA; KOJURI, JAVAD

2017-01-01

Introduction: In recent years curriculum reform and integration was done in many medical schools. The integrated curriculum is a popular concept all over the world. In Shiraz medical school, the reform was initiated by stablishing the horizontal basic science integration model and Early Clinical Exposure (ECE) for undergraduate medical education. The purpose of this study was to provide the required data for the program evaluation of this curriculum for undergraduate medical students, using CIPP program evaluation model. Methods: This study is an analytic descriptive and triangulation mixed method study which was carried out in Shiraz Medical School in 2012, based on the views of professors of basic sciences courses and first and second year medical students. The study evaluated the quality of the relationship between basic sciences and clinical courses and the method of presenting such courses based on the Context, Input, Process and Product (CIPP) model. The tools for collecting data, both quantitatively and qualitatively, were some questionnaires, content analysis of portfolios, semi- structured interview and brain storming sessions. For quantitative data analysis, SPSS software, version 14, was used. Results: In the context evaluation by modified DREEM questionnaire, 77.75%of the students believed that this educational system encourages them to actively participate in classes. Course schedule and atmosphere of class were reported suitable by 87.81% and 83.86% of students. In input domain that was measured by a researcher made questionnaire, the facilities for education were acceptable except for shortage of cadavers. In process evaluation, the quality of integrated modules presentation and Early Clinical Exposure (ECE) was good from the students’ viewpoint. In product evaluation, students’ brain storming, students’ portfolio and semi-structured interview with faculties were done, showing some positive aspects of integration and some areas that need improvement. Conclusion: The main advantage of assessing an educational program based on CIPP evaluation model is that the context, input, process and product of the program are viewed and evaluated systematically. This will help the educational authorities to make proper decisions based on the weaknesses and strengths of the program on its continuation, cessation and revision. Based on the results of this study, the integrated basic sciences course for undergraduate medical students in Shiraz Medical School is at a desirable level. However, attempts to improve or reform some sections and continual evaluation of the program and its accreditation seem to be necessary. PMID:28761888

HYDROLOGIC EVALUATION OF LANDFILL PERFORMANCE (HELP) MODEL - USER'S GUIDE FOR VERSION 3

EPA Science Inventory

This report documents the solution methods and process descriptions used in the Version 3 of the HELP model. Program documentation including program options, system and operating requirements, file structures, program structure and variable descriptions are provided in a separat...

SITE TECHNOLOGY PROFILES, TENTH EDITION, VOLUME I - DEMONSTRATION PROGRAM

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its thirteenth year, is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine us...

Program Performance Inventory: Six Juvenile Offender Programs.

ERIC Educational Resources Information Center

Thomalla, Terri Groff; Dougherty, Victoria J.

This report describes the performance of 6 Connecticut juvenile justice alternative sanction programs in 14 qualitative areas: community reintegration; outcomes and evaluation; assessment methods; risk factors; escalation of criminal activity; family involvement; community involvement; work ethic and vocational training; education and life skills;…

Finally! Distance Education for Library Support Staff.

ERIC Educational Resources Information Center

Rau, Kirk

1998-01-01

Discusses distance undergraduate education programs for paraprofessional librarians and suggests how to evaluate institutions: accreditation, academic program, communication, curriculum, and delivery methods. Highlights the AS degree in Library and Information Technology distance-education program offered by the University of Maine at Augusta.…

Using Qualitative Methods to Evaluate a Family Behavioral Intervention for Type 1 Diabetes

PubMed Central

Herbert, Linda Jones; Sweenie, Rachel; Kelly, Katherine Patterson; Holmes, Clarissa; Streisand, Randi

2013-01-01

Introduction The objectives of this study were to qualitatively evaluate a dyadic adolescent-parent type 1 diabetes (T1D) program developed to prevent deterioration in diabetes care among adolescents with T1D and provide recommendations for program refinement. Method Thirteen adolescent-parent dyads who participated in the larger RCT, the TeamWork Project, were interviewed regarding their perceptions of their participation in the program and current T1D challenges. Interviews were transcribed and coded to establish broad themes. Results Adolescents and parents thought the TeamWork Project sessions were helpful and taught them new information. Five themes catalog findings from the qualitative interviews: TeamWork content, TeamWork structure, transition of responsibility, current and future challenges, and future intervention considerations. Discussion Addressing T1D challenges as a parent-adolescent dyad via a behavioral clinic program is helpful to families during adolescence. Findings highlight the utility of qualitative evaluation to tailor interventions for the unique challenges related to pediatric chronic illness. PMID:24269281

Challenges Facing Evidence-Based Prevention: Incorporating an Abductive Theory of Method.

PubMed

Mason, W Alex; Cogua-Lopez, Jasney; Fleming, Charles B; Scheier, Lawrence M

2018-06-01

Current systems used to determine whether prevention programs are "evidence-based" rely on the logic of deductive reasoning. This reliance has fostered implementation of strategies with explicitly stated evaluation criteria used to gauge program validity and suitability for dissemination. Frequently, investigators resort to the randomized controlled trial (RCT) combined with null hypothesis significance testing (NHST) as a means to rule out competing hypotheses and determine whether an intervention works. The RCT design has achieved success across numerous disciplines but is not without limitations. We outline several issues that question allegiance to the RCT, NHST, and the hypothetico-deductive method of scientific inquiry. We also discuss three challenges to the status of program evaluation including reproducibility, generalizability, and credibility of findings. As an alternative, we posit that extending current program evaluation criteria with principles drawn from an abductive theory of method (ATOM) can strengthen our ability to address these challenges and advance studies of drug prevention. Abductive reasoning involves working from observed phenomena to the generation of alternative explanations for the phenomena and comparing the alternatives to select the best possible explanation. We conclude that an ATOM can help increase the influence and impact of evidence-based prevention for population benefit.

Cultural and Linguistic Adaptation of a Healthy Diet Text Message Intervention for Hispanic Adults Living in the United States.

PubMed

Cameron, Linda D; Durazo, Arturo; Ramírez, A Susana; Corona, Roberto; Ultreras, Mayra; Piva, Sonia

2017-03-01

Hispanics represent a critical target for culturally adapted diet interventions. In this formative research, we translated HealthyYouTXT, an mHealth program developed by the U.S. National Cancer Institute, into HealthyYouTXT en Español, a linguistically and culturally appropriate version for Spanish speakers in the United States. We report a three-stage, mixed-methods process through which we culturally adapted the text messages, evaluated their acceptability, and revised the program based on the findings. In Stage 1, we conducted initial translations and adaptations of the text libraries using an iterative, principle-guided process. In Stage 2, we used mixed methods including focus groups and surveys with 109 Hispanic adults to evaluate the acceptability and cultural appropriateness of the program. We used survey data to evaluate whether self-determination theory (SDT) factors (used to develop HealthyYouTXT) of autonomous motivation, controlled motivation, and amotivation and Hispanic cultural beliefs about familism, fatalism, and destiny predict program interest and its perceived efficacy. Mixed-methods analyses revealed substantial interest in HealthyYouTXT, with most participants desiring to use it and viewing it as highly efficacious. Both cultural beliefs (i.e., beliefs in destiny and, for men, high familism) and SDT motivations (i.e., autonomy) predicted HealthyYouTXT evaluations, suggesting utility in emphasizing them in messages. Higher destiny beliefs predicted lower interest, suggesting that they could impede program use. In Stage 3, we implemented the mixed-methods findings to finalize HealthyYouTXT en Español. The emergent linguistic principles and multistage, multimethods process can be applied in health communication adaptations.

A quantitative approach to assessing the efficacy of occupant protection programs: A case study from Montana.

PubMed

Manlove, Kezia; Stanley, Laura; Peck, Alyssa

2015-10-01

Quantitative evaluation of vehicle occupant protection programs is critical for ensuring efficient government resource allocation, but few methods exist for conducting evaluation across multiple programs simultaneously. Here we present an analysis of occupant protection efficacy in the state of Montana. This approach relies on seat belt compliance rates as measured by the National Occupant Protection Usage Survey (NOPUS). A hierarchical logistic regression model is used to estimate the impacts of four Montana Department of Transportation (MDT)-funded occupant protection programs used in the state of Montana, following adjustment for a suite of potential confounders. Activity from two programs, Buckle Up coalitions and media campaigns, are associated with increased seat belt use in Montana, whereas the impact of another program, Selective Traffic Enforcement, is potentially masked by other program activity. A final program, Driver's Education, is not associated with any shift in seat belt use. This method allows for a preliminary quantitative estimation of program impacts without requiring states to obtain any new seat belt use data. This approach provides states a preliminary look at program impacts, and a means for carefully planning future program allocation and investigation. Copyright © 2015 Elsevier Ltd. All rights reserved.

Adaptation of Different Computerized Methods of Distance Learning to an External PharmD Degree Program.

ERIC Educational Resources Information Center

Kaplan, Irene Petzinger; And Others

1996-01-01

A distance education course in drug information, scholarly literature evaluation, and biostatistics, offered as part of a doctoral pharmacy program, incorporates teleconferencing and online critiquing of pharmacy journal literature. Structure of the class, software and hardware, student performance, and student response to the teaching methods are…

Evaluation of a Teaching Assistant Program for Third-Year Pharmacy Students.

PubMed

Bradley, Courtney L; Khanova, Julia; Scolaro, Kelly L

2016-11-25

Objectives. To determine if a teaching assistant (TA) program for third-year pharmacy students (PY3s) improves confidence in teaching abilities. Additionally, 3 assessment methods (faculty, student, and TA self-evaluations) were compared for similarities and correlations. Methods. An application and interview process was used to select 21 pharmacy students to serve as TAs for the Pharmaceutical Care Laboratory course for 2 semesters. Participants' self-perceived confidence in teaching abilities was assessed at the start, midpoint, and conclusion of the program. The relationships between the scores were analyzed using 3 assessment methods. Results. All 21 TAs agreed to participate in the study and completed the 2 teaching semesters. The TAs confidence in overall teaching abilities increased significantly (80.7 vs 91.4, p <0.001). There was a significant difference between the three assessment scores in the fall ( p =0.027) and spring ( p <0.001) semesters. However, no correlation was found among the assessment scores. Conclusions. The TA program was effective in improving confidence in teaching abilities. The lack of correlation among the assessment methods highlights the importance of various forms of feedback.

An interactive program for pharmacokinetic modeling.

PubMed

Lu, D R; Mao, F

1993-05-01

A computer program, PharmK, was developed for pharmacokinetic modeling of experimental data. The program was written in C computer language based on the high-level user-interface Macintosh operating system. The intention was to provide a user-friendly tool for users of Macintosh computers. An interactive algorithm based on the exponential stripping method is used for the initial parameter estimation. Nonlinear pharmacokinetic model fitting is based on the maximum likelihood estimation method and is performed by the Levenberg-Marquardt method based on chi 2 criterion. Several methods are available to aid the evaluation of the fitting results. Pharmacokinetic data sets have been examined with the PharmK program, and the results are comparable with those obtained with other programs that are currently available for IBM PC-compatible and other types of computers.

Preliminary demonstration of a robust controller design method

NASA Technical Reports Server (NTRS)

Anderson, L. R.

1980-01-01

Alternative computational procedures for obtaining a feedback control law which yields a control signal based on measurable quantitites are evaluated. The three methods evaluated are: (1) the standard linear quadratic regulator design model; (2) minimization of the norm of the feedback matrix, k via nonlinear programming subject to the constraint that the closed loop eigenvalues be in a specified domain in the complex plane; and (3) maximize the angles between the closed loop eigenvectors in combination with minimizing the norm of K also via the constrained nonlinear programming. The third or robust design method was chosen to yield a closed loop system whose eigenvalues are insensitive to small changes in the A and B matrices. The relationship between orthogonality of closed loop eigenvectors and the sensitivity of closed loop eigenvalues is described. Computer programs are described.

Employment and the Risk of Domestic Abuse among Low-Income Women

ERIC Educational Resources Information Center

Gibson-Davis, Christina M.; Magnuson, Katherine; Gennetian, Lisa A.; Duncan, Greg J.

2005-01-01

This paper uses data from 2 randomized evaluations of welfare-to-work programs--the Minnesota Family Investment Program and the National Evaluation of Welfare-to-Work Strategies--to estimate the effect of employment on domestic abuse among low-income single mothers. Unique to our analysis is the application of a 2-stage least squares method, in…

Teaching Teachers through Distance Methods: An Evaluation of a Sri Lankan Programme. Education Division Document, No. 40.

ERIC Educational Resources Information Center

Dock, Alan W.; And Others

This report describes the development process of a distance education program in Sri Lanka run by the Institute of Distance Education for nongraduate teachers inservice, and presents results of the program's evaluation. Two teacher education courses--an elementary education course and a combined science/mathematics course--were offered. The…

Quasi-experimental evaluation without regression analysis.

PubMed

Rohrer, James E

2009-01-01

Evaluators of public health programs in field settings cannot always randomize subjects into experimental or control groups. By default, they may choose to employ the weakest study design available: the pretest, posttest approach without a comparison group. This essay argues that natural experiments involving comparison groups are within reach of public health program managers. Methods for analyzing natural experiments are discussed.

Consumer Evaluation of Educational Programs: Using Questionnaires Completed by Alumni.

ERIC Educational Resources Information Center

Copeland, H. L.; Hewson, M.; Weiker, G.

This paper describes a method used to evaluate 48 medical residency programs at one institution. It presents the background rationale, logistics, and results obtained by surveying alumni, in the context of their current practice, for relevant data on the value of their training experience. Surveys were sent for 3 years to all alumni of Cleveland…

Evaluating the Impact of a Connecticut Program to Reduce Availability of Unhealthy Competitive Food in Schools

ERIC Educational Resources Information Center

Long, Michael W.; Henderson, Kathryn E.; Schwartz, Marlene B.

2010-01-01

Background: This article seeks to inform state and local school food policies by evaluating the impact of Connecticut's Healthy Food Certification (HFC), a program which provides monetary incentives to school districts that choose to implement state nutrition standards for all foods sold to students outside reimbursable school meals. Methods: Food…

Developing a Stakeholder-Driven Anticipated Timeline of Impact for Evaluation of Social Programs

ERIC Educational Resources Information Center

Sridharan, Sanjeev; Campbell, Bernadette; Zinzow, Heidi

2006-01-01

The authors present a stakeholder-driven method, the earliest anticipated timeline of impact, which is designed to assess stakeholder expectations for the earliest time frame in which social programs are likely to affect outcomes. The utility of the anticipated timeline of impact is illustrated using an example from an evaluation of a…

Mixed-Methods Evaluation of a Healthy Exercise, Eating, and Lifestyle Program for Primary Schools

ERIC Educational Resources Information Center

Cochrane, Thomas; Davey, Rachel C.

2017-01-01

BAckground: Reversing decline in physical fitness and increase in excess body weight in school children are considered major public health challenges. We evaluated a proposed model to integrate a screening and healthy exercise, eating, and lifestyle program (HEELP) into primary schools in Canberra, Australia. Objectives were: (1) to establish body…

Evaluation of Multidisciplinary Tobacco Cessation Training Program in a Large Health Care System

ERIC Educational Resources Information Center

Chen, Timothy C.; Hamlett-Berry, Kim W.; Watanabe, Jonathan H.; Bounthavong, Mark; Zillich, Alan J.; Christofferson, Dana E.; Myers, Mark G.; Himstreet, Julianne E.; Belperio, Pamela S.; Hudmon, Karen Suchanek

2015-01-01

Background: Health care professionals can have a dramatic impact by assisting patients with tobacco cessation but most have limited training. Purpose: To evaluate the effectiveness of a 4-hour tobacco cessation training program. Methods: A team of multidisciplinary health care professionals created a veteran-specific tailored version of the Rx for…

An Evaluation of a Comprehensive Mentoring Program on Selected At-Risk Students with Specific School Problems

ERIC Educational Resources Information Center

Washington, Taren L.

2015-01-01

This study evaluated a comprehensive mentoring program on selected at-risk students with specific school problems (attendance, discipline referrals, and core area grades). The sample included youths in Grades 4-8 who differed on some characteristics including grade-level, ethnicity, and gender. For the purpose of this mixed methods study, the…

Sensitizing Children to the Social and Emotional Mechanisms Involved in Racism: A Program Evaluation

ERIC Educational Resources Information Center

Triliva, Sofia; Anagnostopoulou, Tanya; Vleioras, Georgios

2014-01-01

This paper describes and discusses the results of an intervention aiming to sensitize children to the social and emotional processes involved in racism. The intervention was applied and evaluated in 10 Greek elementary schools. The goals and the intervention methods of the program modules are briefly outlined and the results of the program…

APPLICATION OF THE SPERM CHROMATIN STRUCTURE ASSAY TO THE TEPLICE PROGRAM SEMEN STUDIES: A NEW METHOD FOR EVALUATING SPERM NUCLEAR CHROMATIN DAMAGE

EPA Science Inventory

ABSTRACT
A measure of sperm chromatin integrity was added to the routine semen end points evaluated in the Teplice Program male reproductive health studies. To address the hypothesis that exposure to periods of elevated air pollution may be associated with abnormalities in sp...

Pilot Evaluation of Parent and Child Enhancement Program for Disadvantaged Families

ERIC Educational Resources Information Center

Leung, Cynthia; Tsang, Sandra; Leung, Iona

2014-01-01

Objective: The present study was a pilot evaluation of the effectiveness of the Parent and Child Enhancement (PACE) program in enhancing child development for disadvantaged families with children aged 2 to 3. Method: Participants included 23 parent-child dyads who attended 2-hr sessions twice weekly for 20 weeks. Parents and children were assessed…

Evaluating the Effectiveness of Mental Health First Aid Program for Chinese People in Hong Kong

ERIC Educational Resources Information Center

Wong, Daniel F. K.; Lau, Ying; Kwok, Sylvia; Wong, Prudence; Tori, Christopher

2017-01-01

Purpose: Chinese people generally lack knowledge of mental illness. Such phenomenon may lead to a delay in seeking psychiatric treatments. This study evaluated the effectiveness of Mental Health First Aid (MHFA) program in improving mental health knowledge of the general public in Hong Kong. Methods: A quasi-experimental design was adopted whereby…

The Use of Workforce Assessment as a Component of Career and Technical Education Program Evaluation

ERIC Educational Resources Information Center

Bartlett, Kenneth R.; Schleif, Nicole L.; Bowen, Mauvalyn M.

2011-01-01

This research project examined the extent to which Career and Technical Education (CTE)-related programs use workforce needs assessment as a component of their evaluation activities. An employer perspective was used to develop a conceptual framework drawing on strategic human resource management theory. The extent and methods utilized for…

Evaluating the Effects of Child Savings Accounts Program Participation on Parental Well-Being

ERIC Educational Resources Information Center

Okech, David

2012-01-01

Objectives: Using baseline and second wave data, the study evaluated the impact of child savings accounts participation on parenting stress, personal mastery, and economic strain with N = 381 lower income parents who decided to join and those who did not join in a child development savings account program. Methods: Structural equation modeling for…

A Healthy Lifestyle Program for Latino Daughters and Mothers: The BOUNCE Overview and Process Evaluation

ERIC Educational Resources Information Center

Olvera, Norma N.; Knox, Brook; Scherer, Rhonda; Maldonado, Gabriela; Sharma, Shreela V.; Alastuey, Lisa; Bush, Jill A.

2008-01-01

Background: Few family-based healthy lifestyle programs for Latinos have been conducted, especially family programs targeting mother-daughter dyads. Purpose: To assess the acceptability and feasibility of the Behavior Opportunities Uniting Nutrition Counseling and Exercise (BOUNCE) program designed for Latino mother-daughter pairs. Methods: 92…

Adult Education for the Homeless: A Program in Jeopardy.

ERIC Educational Resources Information Center

Office of Vocational and Adult Education (ED), Washington, DC. Div. of Adult Education and Literacy.

During its 8-year history, the federal Adult Education for the Homeless Program (AEH) pioneered new methods of service to adults in need and benefited over 320,000 homeless adults and families. Despite an evaluation that documented program success, funding was rescinded from the 1995 federal budget and never reinstated. AEH programs developed the…

Comparison Studies of Reading Readiness Skills Acquisition by Different Methods: Formal Reading Readiness Program, Informal Reading Readiness Program, and a Kindergarten Science Program.

ERIC Educational Resources Information Center

Quorn, Kerry Charles; Yore, Larry Dean

1978-01-01

Evaluates the effectiveness of two dissimilar programs, SAPA and the First Talking Alphabet (FTA), upon the acquisition of reading readiness skills of kindergarten pupils. Compares the effects of four different programs (SAPA, FTA, an informal language development program, and a control) on the reading readiness of kindergarten pupils. (GA)

A study of commuter airplane design optimization

NASA Technical Reports Server (NTRS)

Keppel, B. V.; Eysink, H.; Hammer, J.; Hawley, K.; Meredith, P.; Roskam, J.

1978-01-01

The usability of the general aviation synthesis program (GASP) was enhanced by the development of separate computer subroutines which can be added as a package to this assembly of computerized design methods or used as a separate subroutine program to compute the dynamic longitudinal, lateral-directional stability characteristics for a given airplane. Currently available analysis methods were evaluated to ascertain those most appropriate for the design functions which the GASP computerized design program performs. Methods for providing proper constraint and/or analysis functions for GASP were developed as well as the appropriate subroutines.

Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods.

PubMed

Pommier, Jeanine; Guével, Marie-Renée; Jourdan, Didier

2010-01-28

Schools are key settings for health promotion (HP) but the development of suitable approaches for evaluating HP in schools is still a major topic of discussion. This article presents a research protocol of a program developed to evaluate HP. After reviewing HP evaluation issues, the various possible approaches are analyzed and the importance of a realistic evaluation framework and a mixed methods (MM) design are demonstrated. The design is based on a systemic approach to evaluation, taking into account the mechanisms, context and outcomes, as defined in realistic evaluation, adjusted to our own French context using an MM approach. The characteristics of the design are illustrated through the evaluation of a nationwide HP program in French primary schools designed to enhance children's social, emotional and physical health by improving teachers' HP practices and promoting a healthy school environment. An embedded MM design is used in which a qualitative data set plays a supportive, secondary role in a study based primarily on a different quantitative data set. The way the qualitative and quantitative approaches are combined through the entire evaluation framework is detailed. This study is a contribution towards the development of suitable approaches for evaluating HP programs in schools. The systemic approach of the evaluation carried out in this research is appropriate since it takes account of the limitations of traditional evaluation approaches and considers suggestions made by the HP research community.

Success in the TACP Training Program An Objective Method for Selecting Battlefield Airmen

DTIC Science & Technology

2009-12-23

rationale and evaluation of the physical training doctrine. J Strength Cond Res. 2009 Jul;23(4):1353-62. Interventions used to improve retention...week training programs on military physical performance. J Strength Cond Res. 2008 Mar;22(2):524-34. Muza SR. Military applications of hypoxic...1    Success in the TACP Training Program An Objective Method for Selecting Battlefield Airmen FINAL REPORT December 23, 2009

Evaluations of Sexual Assault Prevention Programs in Military Settings: A Synthesis of the Research Literature.

PubMed

Orchowski, Lindsay M; Berry-Cabán, Cristóbal S; Prisock, Kara; Borsari, Brian; Kazemi, Donna M

2018-03-01

The prevention of sexual assault (SA) in the U.S. military is a significant priority. This study applied the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to a literature search that identified research evaluating SA prevention programs conducted within military settings. Only six studies published between 2005 and 2016 met criteria for inclusion in the review. Studies demonstrated high heterogeneity in the: (1) conceptual framework of the prevention approach; (2) target population and timing of administration; (3) study recruitment methods; (4) methodological design; (5) method of delivery, program dosage and theory of change; and (6) outcome administration and efficacy. Scientific rigor according to the Oxford Center for Evidence-based Medicine was also variable. Several gaps in the research base were identified. Specifically, research evaluating SA prevention programs have only been conducted among U.S. Army and U.S. Navy samples. Most studies did not examine whether program participation was associated with reductions in rates of sexual violence. Studies also lacked utilization of a long-term follow-up period. Additionally, studies did not reflect the types of SA prevention programs currently being implemented in military settings. Taken together, further research is needed to enhance the evidence base for SA prevention in the military, and to evaluate the effectiveness of the approaches currently being conducted with service members.

Monitoring and evaluating transition and sustainability of donor-funded programs: Reflections on the Avahan experience.

PubMed

Bennett, Sara; Ozawa, Sachiko; Rodriguez, Daniela; Paul, Amy; Singh, Kriti; Singh, Suneeta

2015-10-01

In low and middle-income countries, programs funded and implemented by international donors frequently transition to local funding and management, yet such processes are rarely evaluated. We reflect upon experience evaluating the transition of a large scale HIV/AIDS prevention program in India, known as Avahan, in order to draw lessons about transition evaluation approaches and implementation challenges. In terms of conceptualizing the transition theory, the evaluation team identified tensions between the idea of institutionalizing key features of the Avahan program, and ensuring program flexibility to promote sustainability. The transition was planned in three rounds allowing for adaptations to transition intervention and program design during the transition period. The assessment team found it important to track these changes in order to understand which strategies and contextual features supported transition. A mixed methods evaluation was employed, combining semi-structured surveys of transitioning entities (conducted pre and post transition), with longitudinal case studies. Qualitative data helped explain quantitative findings. Measures of transition readiness appeared robust, but we were uncertain of the robustness of institutionalization measures. Finally, challenges to the implementation of such an evaluation are discussed. Given the scarceness of transition evaluations, the lessons from this evaluation may have widespread relevance. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Framework and criteria for program evaluation in the Office of Conservation and Renewable Energy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

This study addresses the development of a framework and generic criteria for conducting program evaluation in the Office of Conservation and Renewable Energy. The evaluation process is intended to provide the Assistant Secretary with comprehensive and consistent evaluation data for management decisions regarding policy and strategy, crosscutting energy impacts and resource allocation and justification. The study defines evaluation objectives, identifies basic information requirements (criteria), and identifies a process for collecting evaluation results at the basic program level, integrating the results, and summarizing information upward through the CE organization to the Assistant Secretary. Methods are described by which initial criteria weremore » tested, analyzed, and refined for CE program applicability. General guidelines pertaining to evaluation and the Sunset Review requirements are examined and various types, designs, and models for evaluation are identified. Existing CE evaluation reports are reviewed and comments on their adequacy for meeting current needs are provided. An inventory and status survey of CE program evaluation activities is presented, as are issues, findings, and recommendations pertaining to CE evaluation and Sunset Review requirements. Also, sources of data for use in evaluation and the Sunset Review response are identified. An inventory of CE evaluation-related documents and reports is provided.« less

SITE TECHNOLOGY PROFILES, TENTH EDITION, VOLUME 3 - MEASUREMENT AND MONITORING PROGRAM

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its thirteenth year, is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine us...

SITE TECHNOLOGY PROFILES, TENTH EDITION, VOLUME 2 - EMERGING TECHNOLOGY PROGRAM

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its thirteenth year, is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine us...

SITE TECHNOLOGY PROFILES - 11TH EDITION - DEMONSTRATION PROGRAM, VOLUME 1

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its eleventh year is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine use o...

PREFACE: SPECIAL ISSUE OF ATMOSPHERIC ENVIRONMENT ON FINDINGS FROM EPA'S PARTICULATE MATTER SUPERSITES PROGRAM

EPA Science Inventory

The Supersites Program is a methods development and evaluation, measurements, modeling, and data analysis program designed to provide key stakeholders in the science, regulatory, and policy communities with information to support primarily implementation of National Ambient Air Q...

Permanent and separable aerospace tubing/ fitting evaluation program, volume 1

NASA Technical Reports Server (NTRS)

Ursell, C. R.; Franz, C. R.

1975-01-01

The tube fitting evaluation program was conducted to develop a standard overall test program including methods, procedures, and equipment as well as recommended test sequences for qualifying fitting/tubing assemblies. The program consisted of testing the MS flareless (separable) fitting and utilizing the results as baseline data from which all other fittings will be evaluated. Five separable designs and five permanent designs were tested in three sizes with two types of tubing materials. The basic design requirements were 4,000 psi operating pressure within the temperature range of minus 65 F to plus 450 F while also being compatible with 21-6-9 and titanium 3Al-2.5V tubing alloys.

Cardiac Rehabilitation Online Pilot: Extending Reach of Cardiac Rehabilitation.

PubMed

Higgins, Rosemary O; Rogerson, Michelle; Murphy, Barbara M; Navaratnam, Hema; Butler, Michael V; Barker, Lauren; Turner, Alyna; Lefkovits, Jeffrey; Jackson, Alun C

While cardiac rehabilitation (CR) is recommended for all patients after an acute cardiac event, limitations exist in reach. The purpose of the current study was to develop and pilot a flexible online CR program based on self-management principles "Help Yourself Online." The program was designed as an alternative to group-based CR as well as to complement traditional CR. The program was based on existing self-management resources developed previously by the Heart Research Centre. Twenty-one patients admitted to Cabrini Health for an acute cardiac event were recruited to test the program. The program was evaluated using qualitative and quantitative methods. Quantitative results demonstrated that patients believed the program would assist them in their self-management. Qualitative evaluation, using focus group and interview methods with 15 patients, showed that patients perceived the online CR approach to be a useful instrument for self-management. Broader implications of the data include the acceptability of the intervention, timing of intervention delivery, and patients' desire for additional online community support.

Conceptual Soundness, Metric Development, Benchmarking, and Targeting for PATH Subprogram Evaluation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mosey. G.; Doris, E.; Coggeshall, C.

The objective of this study is to evaluate the conceptual soundness of the U.S. Department of Housing and Urban Development (HUD) Partnership for Advancing Technology in Housing (PATH) program's revised goals and establish and apply a framework to identify and recommend metrics that are the most useful for measuring PATH's progress. This report provides an evaluative review of PATH's revised goals, outlines a structured method for identifying and selecting metrics, proposes metrics and benchmarks for a sampling of individual PATH programs, and discusses other metrics that potentially could be developed that may add value to the evaluation process. The frameworkmore » and individual program metrics can be used for ongoing management improvement efforts and to inform broader program-level metrics for government reporting requirements.« less

Intervention mapping: a process for developing theory- and evidence-based health education programs.

PubMed

Bartholomew, L K; Parcel, G S; Kok, G

1998-10-01

The practice of health education involves three major program-planning activities: needs assessment, program development, and evaluation. Over the past 20 years, significant enhancements have been made to the conceptual base and practice of health education. Models that outline explicit procedures and detailed conceptualization of community assessment and evaluation have been developed. Other advancements include the application of theory to health education and promotion program development and implementation. However, there remains a need for more explicit specification of the processes by which one uses theory and empirical findings to develop interventions. This article presents the origins, purpose, and description of Intervention Mapping, a framework for health education intervention development. Intervention Mapping is composed of five steps: (1) creating a matrix of proximal program objectives, (2) selecting theory-based intervention methods and practical strategies, (3) designing and organizing a program, (4) specifying adoption and implementation plans, and (5) generating program evaluation plans.

Where We Now Stand

ERIC Educational Resources Information Center

Lumsden, Keith G.

1969-01-01

Noting the acceleration of scientific evaluation, the author focuses his attention on where we now stand in the evaluation of two methods of instruction: programed learning materials and television. (Editor)

Accelerated Training of Skilled Birth Attendants in a Marginalized Population on the Thai-Myanmar Border: A Multiple Methods Program Evaluation

PubMed Central

White, Adrienne Lynne; Min, Thaw Htwe; Gross, Mechthild M.; Kajeechiwa, Ladda; Thwin, May Myo; Hanboonkunupakarn, Borimas; Than, Hla Hla; Zin, Thet Wai; Rijken, Marcus J.; Hoogenboom, Gabie; McGready, Rose

2016-01-01

Background To evaluate a skilled birth attendant (SBA) training program in a neglected population on the Thai-Myanmar border, we used multiple methods to show that refugee and migrant health workers can be given effective training in their own environment to become SBAs and teachers of SBAs. The loss of SBAs through resettlement to third countries necessitated urgent training of available workers to meet local needs. Methods and Findings All results were obtained from student records of theory grades and clinical log books. Qualitative evaluation of both the SBA and teacher programs was obtained using semi-structured interviews with supervisors and teachers. We also reviewed perinatal indicators over an eight-year period, starting prior to the first training program until after the graduation of the fourth cohort of SBAs. Results Four SBA training programs scheduled between 2009 and 2015 resulted in 79/88 (90%) of students successfully completing a training program of 250 theory hours and 625 supervised clinical hours. All 79 students were able to: achieve pass grades on theory examination (median 80%, range [70–89]); obtain the required clinical experience within twelve months; achieve clinical competence to provide safe care during childbirth. In 2010–2011, five experienced SBAs completed a train-the-trainer (TOT) program and went on to facilitate further training programs. Perinatal indicators within Shoklo Malaria Research Unit (SMRU), such as place of birth, maternal and newborn outcomes, showed no significant differences before and after introduction of training or following graduate deployment in the local maternity units. Confidence, competence and teamwork emerged from qualitative evaluation by senior SBAs working with and supervising students in the clinics. Conclusions We demonstrate that in resource-limited settings or in marginalized populations, it is possible to accelerate training of skilled birth attendants to provide safe maternity care. Education needs to be tailored to local needs to ensure evidence-based care of women and their families. PMID:27711144

Evaluation Methods Sourcebook.

ERIC Educational Resources Information Center

Love, Arnold J., Ed.

The chapters commissioned for this book describe key aspects of evaluation methodology as they are practiced in a Canadian context, providing representative illustrations of recent developments in evaluation methodology as it is currently applied. The following chapters are included: (1) "Program Evaluation with Limited Fiscal and Human…

DYNSYL: a general-purpose dynamic simulator for chemical processes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Patterson, G.K.; Rozsa, R.B.

1978-09-05

Lawrence Livermore Laboratory is conducting a safeguards program for the Nuclear Regulatory Commission. The goal of the Material Control Project of this program is to evaluate material control and accounting (MCA) methods in plants that handle special nuclear material (SNM). To this end we designed and implemented the dynamic chemical plant simulation program DYNSYL. This program can be used to generate process data or to provide estimates of process performance; it simulates both steady-state and dynamic behavior. The MCA methods that may have to be evaluated range from sophisticated on-line material trackers such as Kalman filter estimators, to relatively simplemore » material balance procedures. This report describes the overall structure of DYNSYL and includes some example problems. The code is still in the experimental stage and revision is continuing.« less

Estimating the Counterfactual Impact of Conservation Programs on Land Cover Outcomes: The Role of Matching and Panel Regression Techniques.

PubMed

Jones, Kelly W; Lewis, David J

2015-01-01

Deforestation and conversion of native habitats continues to be the leading driver of biodiversity and ecosystem service loss. A number of conservation policies and programs are implemented--from protected areas to payments for ecosystem services (PES)--to deter these losses. Currently, empirical evidence on whether these approaches stop or slow land cover change is lacking, but there is increasing interest in conducting rigorous, counterfactual impact evaluations, especially for many new conservation approaches, such as PES and REDD, which emphasize additionality. In addition, several new, globally available and free high-resolution remote sensing datasets have increased the ease of carrying out an impact evaluation on land cover change outcomes. While the number of conservation evaluations utilizing 'matching' to construct a valid control group is increasing, the majority of these studies use simple differences in means or linear cross-sectional regression to estimate the impact of the conservation program using this matched sample, with relatively few utilizing fixed effects panel methods--an alternative estimation method that relies on temporal variation in the data. In this paper we compare the advantages and limitations of (1) matching to construct the control group combined with differences in means and cross-sectional regression, which control for observable forms of bias in program evaluation, to (2) fixed effects panel methods, which control for observable and time-invariant unobservable forms of bias, with and without matching to create the control group. We then use these four approaches to estimate forest cover outcomes for two conservation programs: a PES program in Northeastern Ecuador and strict protected areas in European Russia. In the Russia case we find statistically significant differences across estimators--due to the presence of unobservable bias--that lead to differences in conclusions about effectiveness. The Ecuador case illustrates that if time-invariant unobservables are not present, matching combined with differences in means or cross-sectional regression leads to similar estimates of program effectiveness as matching combined with fixed effects panel regression. These results highlight the importance of considering observable and unobservable forms of bias and the methodological assumptions across estimators when designing an impact evaluation of conservation programs.

Quasi-experimental study designs series-paper 13: realizing the full potential of quasi-experiments for health research.

PubMed

Rockers, Peter C; Tugwell, Peter; Røttingen, John-Arne; Bärnighausen, Till

2017-09-01

Although the number of quasi-experiments conducted by health researchers has increased in recent years, there clearly remains unrealized potential for using these methods for causal evaluation of health policies and programs globally. This article proposes five prescriptions for capturing the full value of quasi-experiments for health research. First, new funding opportunities targeting proposals that use quasi-experimental methods should be made available to a broad pool of health researchers. Second, administrative data from health programs, often amenable to quasi-experimental analysis, should be made more accessible to researchers. Third, training in quasi-experimental methods should be integrated into existing health science graduate programs to increase global capacity to use these methods. Fourth, clear guidelines for primary research and synthesis of evidence from quasi-experiments should be developed. Fifth, strategic investments should be made to continue to develop new innovations in quasi-experimental methodologies. Tremendous opportunities exist to expand the use of quasi-experimental methods to increase our understanding of which health programs and policies work and which do not. Health researchers should continue to expand their commitment to rigorous causal evaluation with quasi-experimental methods, and international institutions should increase their support for these efforts. Copyright © 2017 Elsevier Inc. All rights reserved.

EVALUATION OF POSSIBLE BIASES IN THE U.S. EPA'S METHOD 101A-MERCURY EMISSIONS FROM STATIONARY SOURCES

EPA Science Inventory

The U.S. Environmental Protection Agency (EPA), Research Triangle Park, North Carolina, has a program to evaluate and standardize source testing methods for hazardous pollutants in support of current and future air quality regulations. ccasionally, questions arise concerning an e...

Project evaluation and selection using fuzzy Delphi method and zero - one goal programming

NASA Astrophysics Data System (ADS)

Alias, Suriana; Adna, Nofarziah; Arsad, Roslah; Soid, Siti Khuzaimah; Ali, Zaileha Md

2014-12-01

Project evaluation and selection is a factor affecting the impotence of board director in which is trying to maximize all the possible goals. Assessment of the problem occurred in organization plan is the first phase for decision making process. The company needs a group of expert to evaluate the problems. The Fuzzy Delphi Method (FDM) is a systematic procedure to evoke the group's opinion in order to get the best result to evaluate the project performance. This paper proposes an evaluation and selection of the best alternative project based on combination of FDM and Zero - One Goal Programming (ZOGP) formulation. ZOGP is used to solve the multi-criteria decision making for final decision part by using optimization software LINDO 6.1. An empirical example on an ongoing decision making project in Johor, Malaysia is implemented for case study.

Evaluation of a national quality use of medicines service in Australia: an evolving model.

PubMed

Beilby, Justin; Wutzke, Sonia E; Bowman, Jenny; Mackson, Judith M; Weekes, Lynn M

2006-04-01

To describe the first phase of a global evaluation framework for the National Prescribing Service (NPS), with a focus on services for health professionals, and in particular highlight the lessons learnt from evaluation around the establishment and implementation of this national program. The agreed evaluation framework used mixed methods focused around a series of evaluation questions, aimed at measuring the overall effect of this new organization as well as the individual programs within it. The evaluation questions were determined a priori and were based on the objectives established by the organization in its first year of operation. A detailed analyses has been completed of: the process, scope and reach of program delivery using both quantitative and qualitative measures; changes in attitudes and knowledge measured through key informant interviews and surveys of professional groups and consumers; and changes in prescribing behaviour and savings to the Pharmaceutical Benefits Scheme (PBS) through analysis of prescription data. The evaluation period for this report was mid-1998 to mid-2004. The NPS has successfully implemented a complex, multi-faceted program across Australia. From 1998 to 2004, in addition to print material provided to all general practitioners (GPs) and pharmacists, 90% of all GPs have been actively involved in one or more educational activity; 116 of 120 divisions of general practice have coordinated local NPS programs; and 9% of pharmacists have actively participated in at least one educational activity. Sixty per cent of GPs and pharmacists rated the printed educational materials as good or very good. In the last three years, the NPS activities have generated savings in the range of $121-163 million to the PBS, owing to changes in prescribing practices. The national evaluation framework has informed program delivery and ongoing design and development. Continued refinement of existing evaluation methods and further exploration of new techniques will remain a priority for the organization.

Nuclear Data Activities in Support of the DOE Nuclear Criticality Safety Program

NASA Astrophysics Data System (ADS)

Westfall, R. M.; McKnight, R. D.

2005-05-01

The DOE Nuclear Criticality Safety Program (NCSP) provides the technical infrastructure maintenance for those technologies applied in the evaluation and performance of safe fissionable-material operations in the DOE complex. These technologies include an Analytical Methods element for neutron transport as well as the development of sensitivity/uncertainty methods, the performance of Critical Experiments, evaluation and qualification of experiments as Benchmarks, and a comprehensive Nuclear Data program coordinated by the NCSP Nuclear Data Advisory Group (NDAG). The NDAG gathers and evaluates differential and integral nuclear data, identifies deficiencies, and recommends priorities on meeting DOE criticality safety needs to the NCSP Criticality Safety Support Group (CSSG). Then the NDAG identifies the required resources and unique capabilities for meeting these needs, not only for performing measurements but also for data evaluation with nuclear model codes as well as for data processing for criticality safety applications. The NDAG coordinates effort with the leadership of the National Nuclear Data Center, the Cross Section Evaluation Working Group (CSEWG), and the Working Party on International Evaluation Cooperation (WPEC) of the OECD/NEA Nuclear Science Committee. The overall objective is to expedite the issuance of new data and methods to the DOE criticality safety user. This paper describes these activities in detail, with examples based upon special studies being performed in support of criticality safety for a variety of DOE operations.

Evaluation of Nosocomial Infection Control Programs in health services 1

PubMed Central

Menegueti, Mayra Gonçalves; Canini, Silvia Rita Marin da Silva; Bellissimo-Rodrigues, Fernando; Laus, Ana Maria

2015-01-01

OBJECTIVES: to evaluate the Nosocomial Infection Control Programs in hospital institutions regarding structure and process indicators. METHOD: this is a descriptive, exploratory and quantitative study conducted in 2013. The study population comprised 13 Nosocomial Infection Control Programs of health services in a Brazilian city of the state of São Paulo. Public domain instruments available in the Manual of Evaluation Indicators of Nosocomial Infection Control Practices were used. RESULTS: The indicators with the highest average compliance were "Evaluation of the Structure of the Nosocomial Infection Control Programs" (75%) and "Evaluation of the Epidemiological Surveillance System of Nosocomial Infection" (82%) and those with the lowest mean compliance scores were "Evaluation of Operational Guidelines" (58.97%) and "Evaluation of Activities of Control and Prevention of Nosocomial Infection" (60.29%). CONCLUSION: The use of indicators identified that, despite having produced knowledge about prevention and control of nosocomial infections, there is still a large gap between the practice and the recommendations. PMID:25806637

An Evaluation of an Elementary Science Methods Course with Respect to Preservice Teacher's Pedagogical Development

ERIC Educational Resources Information Center

Lee, Carole Kwan-Ping

2012-01-01

The science methods course is a requirement for the Bachelor of Science degree in elementary education licensure program in a mid-west state university in the U.S.A. In one semester, the author decided to evaluate the effectiveness of the science methods course in pedagogical content knowledge areas such as theory, planning and implementation.…

The Computer in Performance and Instruction: Or, How to Tell the True Color of a Chameleon.

ERIC Educational Resources Information Center

Davis, Richard

1979-01-01

Discusses such potential uses for the computer in employee training programs as management support for training and performance, training project control, improved design and development methods, training program implementation and delivery, and program evaluation, revision, and maintenance. (JEG)

SITE TECHNOLOGY PROFILES - 11TH EDITION, EMERGING TECHNOLOGY PROGRAM, VOLUME 2

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its eleventh year is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine use o...

SITE TECHNOLOGY PROFILES - 11TH EDITION, MEASUREMENT AND MONITORING PROGRAM, VOLUME 3

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its eleventh year is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine use o...

AN ORGANIZATION GUIDE TO POLLUTION PREVENTION

EPA Science Inventory

This Pollution Prevention (P2) Guide provides information to help organizations get P2 programs started on t re-evaluate existing P2 programs. It presents an alternative method for working on P2 projects and four approaches to implementing a P2 program in an organization. The int...

Evaluation of Learning Environments for Object-Oriented Programming: Measuring Cognitive Load with a Novel Measurement Technique

ERIC Educational Resources Information Center

Uysal, Murat Pasa

2016-01-01

Various methods and tools have been proposed to overcome the learning obstacles for Object-Oriented Programming (OOP). However, it remains difficult especially for novice learners. The problem may be not only adopting an instructional method, but also an Integrated Development Environment (IDE). Learners employ IDEs as a means to solve programming…

VET Program Completion Rates: An Evaluation of the Current Method. Occasional Paper

ERIC Educational Resources Information Center

National Centre for Vocational Education Research (NCVER), 2016

2016-01-01

This work asks one simple question: "how reliable is the method used by the National Centre for Vocational Education Research (NCVER) to estimate projected rates of VET program completion?" In other words, how well do early projections align with actual completion rates some years later? Completion rates are simple to calculate with a…

Parent Support for Breakfast after the Bell Programs in a State with Very Low Breakfast Participation

ERIC Educational Resources Information Center

Haines, Amanda; Spruance, Lori Andersen

2018-01-01

Purpose/Objectives: The study aim was to evaluate parent support for breakfast after the bell programs (BABPs). Methods: Data were collected through an online survey from parents (n=488) of school-aged children enrolled in public schools in Utah. Data were analyzed using generalized estimating equation (GEE) regression methods. Results: Parents…

PERT and CPM: A Comparison with Implications for Education.

ERIC Educational Resources Information Center

Ragan, Stephen W.

Two systematic ways of analyzing and planning the components of a program or project, both used extensively by industry and government, are discussed in this paper. The methods are Program Evaluation and Review Technique (PERT) Networks and Critical Path Method (CPM) Arrow Diagrams. The purposes of this paper are (1) to explore the need for…

The Development of Online Tutorial Program Design Using Problem-Based Learning in Open Distance Learning System

ERIC Educational Resources Information Center

Said, Asnah; Syarif, Edy

2016-01-01

This research aimed to evaluate of online tutorial program design by applying problem-based learning Research Methods currently implemented in the system of Open Distance Learning (ODL). The students must take a Research Methods course to prepare themselves for academic writing projects. Problem-based learning basically emphasizes the process of…

Interviews with College Students: Evaluating Computer Programming Environments for Introductory Courses

ERIC Educational Resources Information Center

Uysal, Murat Pasa

2014-01-01

Different methods, strategies, or tools have been proposed for teaching Object Oriented Programming (OOP). However, it is still difficult to introduce OOP to novice learners. The problem may be not only adopting a method or language, but also use of an appropriate integrated development environment (IDE). Therefore, the focus should be on the…

Evaluation of accountability measurements

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cacic, C.G.

The New Brunswick Laboratory (NBL) is programmatically responsible to the U.S. Department of Energy (DOE) Office of Safeguards and Security (OSS) for providing independent review and evaluation of accountability measurement technology in DOE nuclear facilities. This function is addressed in part through the NBL Safegaurds Measurement Evaluation (SME) Program. The SME Program utilizes both on-site review of measurement methods along with material-specific measurement evaluation studies to provide information concerning the adequacy of subject accountability measurements. This paper reviews SME Program activities for the 1986-87 time period, with emphasis on noted improvements in measurement capabilities. Continued evolution of the SME Programmore » to respond to changing safeguards concerns is discussed.« less

Evaluating International Research Ethics Capacity Development: An Empirical Approach

PubMed Central

Ali, Joseph; Kass, Nancy E.; Sewankambo, Nelson K.; White, Tara D.; Hyder, Adnan A.

2014-01-01

The US National Institutes of health, Fogarty International Center (NIH-FIC) has, for the past 13 years, been a leading funder of international research ethics education for resource-limited settings. Nearly half of the NIH-FIC funding in this area has gone to training programs that train individuals from sub-Saharan Africa. Identifying the impact of training investments, as well as the potential predictors of post-training success, can support curricular decision-making, help establish funding priorities, and recognize the ultimate outcomes of trainees and training programs. Comprehensive evaluation frameworks and targeted evaluation tools for bioethics training programs generally, and for international research ethics programs in particular, are largely absent from published literature. This paper shares an original conceptual framework, data collection tool, and detailed methods for evaluating the inputs, processes, outputs, and outcomes of research ethics training programs serving individuals in resource-limited settings. This paper is part of a collection of papers analyzing the Fogarty International Center’s International Research Ethics Education and Curriculum Development program. PMID:24782071

Epidemiologic programs for computers and calculators. A microcomputer program for multiple logistic regression by unconditional and conditional maximum likelihood methods.

PubMed

Campos-Filho, N; Franco, E L

1989-02-01

A frequent procedure in matched case-control studies is to report results from the multivariate unmatched analyses if they do not differ substantially from the ones obtained after conditioning on the matching variables. Although conceptually simple, this rule requires that an extensive series of logistic regression models be evaluated by both the conditional and unconditional maximum likelihood methods. Most computer programs for logistic regression employ only one maximum likelihood method, which requires that the analyses be performed in separate steps. This paper describes a Pascal microcomputer (IBM PC) program that performs multiple logistic regression by both maximum likelihood estimation methods, which obviates the need for switching between programs to obtain relative risk estimates from both matched and unmatched analyses. The program calculates most standard statistics and allows factoring of categorical or continuous variables by two distinct methods of contrast. A built-in, descriptive statistics option allows the user to inspect the distribution of cases and controls across categories of any given variable.

A Study of Informal Out-of-School Programs with Migrant Children.

ERIC Educational Resources Information Center

Dik, David

The purposes of this study were to determine the educational impact of an out-of-school informal migrant program, and to discover materials, techniques, methods, and guidelines for similar programs with migrant children. The Orange County, New York 4-H program was evaluated weekly and a record of successful projects, staff training procedures, and…

Curriculum Development Based on the Big Picture Assessment of the Mechanical Engineering Program

ERIC Educational Resources Information Center

Sabri, Mohd Anas Mohd; Khamis, Nor Kamaliana; Tahir, Mohd Faizal Mat; Wahid, Zaliha; Kamal, Ahmad; Ihsan, Ariffin Mohd; Sulong, Abu Bakar; Abdullah, Shahrum

2013-01-01

One of the major concerns of the Engineering Accreditation Council (EAC) is the need for an effective monitoring and evaluation of program outcome domains that can be associated with courses taught under the Mechanical Engineering program. However, an effective monitoring method that can determine the results of each program outcome using Bloom's…

A Meta-Analytic Review of Components Associated with Parent Training Program Effectiveness

ERIC Educational Resources Information Center

Kaminski, Jennifer Wyatt; Valle, Linda Anne; Filene, Jill H.; Boyle, Cynthia L.

2008-01-01

This component analysis used meta-analytic techniques to synthesize the results of 77 published evaluations of parent training programs (i.e., programs that included the active acquisition of parenting skills) to enhance behavior and adjustment in children aged 0-7. Characteristics of program content and delivery method were used to predict effect…

Combining Primary Prevention and Risk Reduction Approaches in Sexual Assault Protection Programming

ERIC Educational Resources Information Center

Menning, Chadwick; Holtzman, Mellisa

2015-01-01

Objective: The object of this study is to extend prior evaluations of Elemental, a sexual assault protection program that combines primary prevention and risk reduction strategies within a single program. Participants and Methods: During 2012 and 2013, program group and control group students completed pretest, posttest, and 6-week and 6-month…

Rigorous Program Evaluations on a Budget: How Low-Cost Randomized Controlled Trials Are Possible in Many Areas of Social Policy

ERIC Educational Resources Information Center

Coalition for Evidence-Based Policy, 2012

2012-01-01

The increasing ability of social policy researchers to conduct randomized controlled trials (RCTs) at low cost could revolutionize the field of performance-based government. RCTs are widely judged to be the most credible method of evaluating whether a social program is effective, overcoming the demonstrated inability of other, more common methods…

Lights, Camera, Action: Advancing Learning, Research, and Program Evaluation through Video Production in Educational Leadership Preparation

ERIC Educational Resources Information Center

Friend, Jennifer; Militello, Matthew

2015-01-01

This article analyzes specific uses of digital video production in the field of educational leadership preparation, advancing a three-part framework that includes the use of video in (a) teaching and learning, (b) research methods, and (c) program evaluation and service to the profession. The first category within the framework examines videos…

Evaluation of the chronic disease self-management program in a Chinese population.

PubMed

Siu, Andrew M H; Chan, Chetwyn C H; Poon, Peter K K; Chui, Dominic Y Y; Chan, Sam C C

2007-01-01

This study evaluated the 6-week Chronic Disease Self-Management Program (CDSMP) in Hong Kong. A total of 148 subjects with chronic illness were recruited. Subjects were matched on duration of illness and gender, and then randomly allocated to experimental and comparison groups. The experimental group participated in the CDSMP, while the comparison group joined a Tai-Chi interest class in a mass-activity format. Subjects completed evaluation questionnaires before beginning their program and 1 week following the program. Analysis of covariance showed that the CDSMP participants demonstrated significantly higher self-efficacy in managing their illness, used more cognitive methods to manage pain and symptoms, and felt more energetic than the subjects in the comparison group. The CDSMP participants also demonstrated changes in their profile of coping strategies, having a tendency to adopt the cognitive methods of diverting attention, reinterpreting pain, ignoring sensations, and making positive self-statements. The short-term evaluation results showed that the CDSMP primarily increased the self-efficacy, exercise behavior, and application of cognitive coping strategies of the participants. The effect of the CDSMP in a Chinese population is similar to that found in studies in Western cultures, and the CDSMP could be applied effectively in a Chinese population.

Normal loads program for aerodynamic lifting surface theory. [evaluation of spanwise and chordwise loading distributions

NASA Technical Reports Server (NTRS)

Medan, R. T.; Ray, K. S.

1974-01-01

A description of and users manual are presented for a U.S.A. FORTRAN 4 computer program which evaluates spanwise and chordwise loading distributions, lift coefficient, pitching moment coefficient, and other stability derivatives for thin wings in linearized, steady, subsonic flow. The program is based on a kernel function method lifting surface theory and is applicable to a large class of planforms including asymmetrical ones and ones with mixed straight and curved edges.

A Wideband Fast Multipole Method for the two-dimensional complex Helmholtz equation

NASA Astrophysics Data System (ADS)

Cho, Min Hyung; Cai, Wei

2010-12-01

A Wideband Fast Multipole Method (FMM) for the 2D Helmholtz equation is presented. It can evaluate the interactions between N particles governed by the fundamental solution of 2D complex Helmholtz equation in a fast manner for a wide range of complex wave number k, which was not easy with the original FMM due to the instability of the diagonalized conversion operator. This paper includes the description of theoretical backgrounds, the FMM algorithm, software structures, and some test runs. Program summaryProgram title: 2D-WFMM Catalogue identifier: AEHI_v1_0 Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEHI_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 4636 No. of bytes in distributed program, including test data, etc.: 82 582 Distribution format: tar.gz Programming language: C Computer: Any Operating system: Any operating system with gcc version 4.2 or newer Has the code been vectorized or parallelized?: Multi-core processors with shared memory RAM: Depending on the number of particles N and the wave number k Classification: 4.8, 4.12 External routines: OpenMP ( http://openmp.org/wp/) Nature of problem: Evaluate interaction between N particles governed by the fundamental solution of 2D Helmholtz equation with complex k. Solution method: Multilevel Fast Multipole Algorithm in a hierarchical quad-tree structure with cutoff level which combines low frequency method and high frequency method. Running time: Depending on the number of particles N, wave number k, and number of cores in CPU. CPU time increases as N log N.

Using program evaluation to support knowledge translation in an interprofessional primary care team: a case study.

PubMed

Donnelly, Catherine; Shulha, Lyn; Klinger, Don; Letts, Lori

2016-10-06

Evaluation is a fundamental component in building quality primary care and is ideally situated to support individual, team and organizational learning by offering an accessible form of participatory inquiry. The evaluation literature has begun to recognize the unique features of KT evaluations and has described attributes to consider when evaluating KT activities. While both disciplines have focused on the evaluation of KT activities neither has explored the role of evaluation in KT. The purpose of the paper is to examine how participation in program evaluation can support KT in a primary care setting. A mixed methods case study design was used, where evaluation was conceptualized as a change process and intervention. A Memory Clinic at an interprofessional primary care clinic was the setting in which the study was conducted. An evaluation framework, Pathways of Influence provided the theoretical foundation to understand how program evaluation can facilitate the translation of knowledge at the level of the individual, inter-personal (Memory Clinic team) and the organization. Data collection included questionnaires, interviews, evaluation log and document analysis. Questionnaires and interviews were administered both before and after the evaluation: Pattern matching was used to analyze the data based on predetermined propositions. Individuals gained program knowledge that resulted in changes to both individual and program practices. One of the key themes was the importance clinicians placed on local, program based knowledge. The evaluation had less influence on the broader health organization. Program evaluation facilitated individual, team and organizational learning. The use of evaluation to support KT is ideally suited to a primary care setting by offering relevant and applicable knowledge to primary care team members while being sensitive to local context.

Did We Get Our Money’s Worth? Bridging Economic and Behavioral Measures of Program Success in Adolescent Drug Prevention

PubMed Central

Griffith, Kevin N.; Scheier, Lawrence M.

2013-01-01

The recent U.S. Congressional mandate for creating drug-free learning environments in elementary and secondary schools stipulates that education reform rely on accountability, parental and community involvement, local decision making, and use of evidence-based drug prevention programs. By necessity, this charge has been paralleled by increased interest in demonstrating that drug prevention programs net tangible benefits to society. One pressing concern is precisely how to integrate traditional scientific methods of program evaluation with economic measures of “cost efficiency”. The languages and methods of each respective discipline don’t necessarily converge on how to establish the true benefits of drug prevention. This article serves as a primer for conducting economic analyses of school-based drug prevention programs. The article provides the reader with a foundation in the relevant principles, methodologies, and benefits related to conducting economic analysis. Discussion revolves around how economists value the potential costs and benefits, both financial and personal, from implementing school-based drug prevention programs targeting youth. Application of heterogeneous costing methods coupled with widely divergent program evaluation findings influences the feasibility of these techniques and may hinder utilization of these practices. Determination of cost-efficiency should undoubtedly become one of several markers of program success and contribute to the ongoing debate over health policy. PMID:24217178

Test Program for Evaluation of Variable Frequency Power Conditioners

DOT National Transportation Integrated Search

1973-08-01

A test program is outlined for variable frequency power conditioners for 3-phase induction motors in vehicle propulsion applications. The Power Conditioner Unit (PCU) performance characteristics are discussed in some detail. Measurement methods, reco...

Generalizable Skills in Individualized Vocational Program Planning for Special Needs Students.

ERIC Educational Resources Information Center

Stodden, Robert A.; Boone, Rosalie

1986-01-01

The authors address planning issues and suggest methods of interagency cooperation, integration of generalizable skills in individualized educational planning/programming, developing goals and objectives, and using student assessment and evaluation information. (CT)

Testing by artificial intelligence: computational alternatives to the determination of mutagenicity.

PubMed

Klopman, G; Rosenkranz, H S

1992-08-01

In order to develop methods for evaluating the predictive performance of computer-driven structure-activity methods (SAR) as well as to determine the limits of predictivity, we investigated the behavior of two Salmonella mutagenicity data bases: (a) a subset from the Genetox Program and (b) one from the U.S. National Toxicology Program (NTP). For molecules common to the two data bases, the experimental concordance was 76% when "marginals" were included and 81% when they were excluded. Three SAR methods were evaluated: CASE, MULTICASE and CASE/Graph Indices (CASE/GI). The programs "learned" the Genetox data base and used it to predict NTP molecules that were not present in the Genetox compilation. The concordances were 72, 80 and 47% respectively. Obviously, the MULTICASE version is superior and approaches the 85% interlaboratory variability observed for the Salmonella mutagenicity assays when the latter was carried out under carefully controlled conditions.

Portfolio as a Teaching Method: A Capstone Project to Promote Recognition of Professional Growth

ERIC Educational Resources Information Center

Wolffe, Robert; Crowe, Helja Antola; Evens, Wayne; McConnaughay, Kelly

2013-01-01

A reflective portfolio as a capstone assignment was selected to accomplish recognition by teachers completing a science, technology, mathematics, engineering master's program for elementary teachers about their professional and personal changes and to provide program evaluators additional qualitative data regarding attainment of program goals. As…

A Field Study Program in Analytical Chemistry for College Seniors.

ERIC Educational Resources Information Center

Langhus, D. L.; Flinchbaugh, D. A.

1986-01-01

Describes an elective field study program at Moravian College (Pennsylvania) in which seniors in analytical chemistry obtain first-hand experience at Bethlehem Steel Corporation. Discusses the program's planning phase, some method development projects done by students, experiences received in laboratory operations, and the evaluation of student…

Alumni Perspectives on Career Preparation during a Postdoctoral Training Program: A Qualitative Study

ERIC Educational Resources Information Center

Faupel-Badger, Jessica M.; Raue, Kimberley; Nelson, David E.; Tsakraklides, Sophia

2015-01-01

Published evaluations of career preparation of alumni from long-standing postdoctoral fellowship programs in the biomedical sciences are limited and often focus on quantitative analysis of data from extant publicly available sources. Qualitative methods provide the opportunity to gather robust information about specific program elements from…

Evaluation of Teacher Preparation Programs: Purposes, Methods, and Policy Options

ERIC Educational Resources Information Center

Feuer, Michael J.; Floden, Robert E.; Chudowsky, Naomi; Ahn, Judie

2013-01-01

Teacher preparation programs (TPPs) are where prospective teachers gain a foundation of knowledge about pedagogy and subject matter, as well as early exposure to practical classroom experience. Although competence in teaching, as in all professions, is shaped significantly by on-the-job experiences and continuous learning, the programs that…

Outcomes Assessment in Accredited Health Information Management Programs

ERIC Educational Resources Information Center

Bennett, Dorine

2010-01-01

The purpose of this study was to determine the use and perceived usefulness of outcomes assessment methods in health information management programs. Additional characteristics of the outcomes assessment practices were recognized. The findings were evaluated for significant differences in results based on age of the program, type of institution,…

Comparative Methods of Program Planning.

ERIC Educational Resources Information Center

Palinchak, Robert S.

All planning techniques and models concern themselves with four basic tasks: (1) the determination of the objectives or purpose of the activity, (2) the development of programs likely to attain the objectives, (3) the organization of experiences into a coherent, meaningful unit, and (4) the evaluation of the extent to which programs meet the…

Evaluating complex health financing interventions: using mixed methods to inform further implementation of a novel PBI intervention in rural Malawi.

PubMed

McMahon, Shannon A; Brenner, Stephan; Lohmann, Julia; Makwero, Christopher; Torbica, Aleksandra; Mathanga, Don P; Muula, Adamson S; De Allegri, Manuela

2016-08-19

Gaps remain in understanding how performance-based incentive (PBI) programs affect quality of care and service quantity, whether programs are cost effective and how programs could be tailored to meet client and provider needs while remaining operationally viable. In 2014, Malawi's Ministry of Health launched the Service Delivery Integration-PBI (SSDI-PBI) program. The program is unique in that no portion of performance bonuses are paid to individual health workers, and it shifts responsibility for infrastructure and equipment procurement from facility staff to implementing partners. This protocol outlines an approach that analyzes processes and outcomes, considers expected and unexpected consequences of the program and frames the program's outputs relative to its costs. Findings from this evaluation will inform the intended future scale-up of PBI in Malawi. This study employs a prospective controlled before-and-after triangulation design to assess effects of the PBI program by analyzing quantitative and qualitative data from intervention and control facilities. Guided by a theoretical framework, the evaluation consists of four main components: service provision, health worker motivation, implementation processes and costing. Quality and access outcomes are assessed along four dimensions: (1) structural elements (related to equipment, drugs, staff); (2) process elements (providers' compliance with standards); (3) outputs (service utilization); (4) experiential elements (experiences of service delivery). The costing component includes costs related to start-up, ongoing management, and the cost of incentives themselves. The cost analysis considers costs incurred within the Ministry of Health, funders, and the implementing agency. The evaluation relies on primary data (including interviews and surveys) and secondary data (including costing and health management information system data). Through the lens of a PBI program, we illustrate how complex interventions can be evaluated via not only primary, mixed-methods data collection, but also through a wealth of secondary data from program implementers (including monitoring, evaluation and financial data), and the health system (including service utilization and service readiness data). We also highlight the importance of crafting a theory and using theory to inform the nature of data collected. Finally, we highlight the need to be responsive to stakeholders in order to enhance a study's relevance.

Evaluation of the Capillary Blood Glucose Self-monitoring Program

PubMed Central

Augusto, Mariana Cristina; Nitsche, Maria José Trevizani; Parada, Cristina Maria Garcia de Lima; Zanetti, Maria Lúcia; Carvalhaes, Maria Antonieta de Barros Leite

2014-01-01

OBJECTIVE: to evaluate the structure, process and results of the Capillary Blood Glucose Self-monitoring Program in a Brazilian city. METHOD: epidemiological, cross-sectional study. The methodological framework of Donabedian was used to construct indicators of structure, process and outcome. A random sample (n = 288) of users enrolled and 96 health professionals who worked in the program was studied. Two questionnaires were used that were constructed for this study, one for professionals and one for users, both containing data for the evaluation of structure, process and outcome. Anthropometric measures and laboratory results were collected by consulting the patients' health records. The analysis involved descriptive statistics. RESULTS: most of the professionals were not qualified to work in the program and were not knowledgeable about the set of criteria for patient registration. None of the patients received complete and correct orientations about the program and the percentage with skills to perform conducts autonomously was 10%. As regards the result indicators, 86.4% of the patients and 81.3% of the professionals evaluated the program positively. CONCLUSION: the evaluation indicators designed revealed that one of the main objectives of the program, self-care skills, has not been achieved. PMID:25493676

The Practice of Health Program Evaluation.

PubMed

Lewis, Sarah R

2017-11-01

The Practice of Health Program Evaluation provides an overview of the evaluation process for public health programs while diving deeper to address select advanced concepts and techniques. The book unfolds evaluation as a three-phased process consisting of identification of evaluation questions, data collection and analysis, and dissemination of results and recommendations. The text covers research design, sampling methods, as well as quantitative and qualitative approaches. Types of evaluation are also discussed, including economic assessment and systems research as relative newcomers. Aspects critical to conducting a successful evaluation regardless of type or research design are emphasized, such as stakeholder engagement, validity and reliability, and adoption of sound recommendations. The book encourages evaluators to document their approach by developing an evaluation plan, a data analysis plan, and a dissemination plan, in order to help build consensus throughout the process. The evaluative text offers a good bird's-eye view of the evaluation process, while offering guidance for evaluation experts on how to navigate political waters and advocate for their findings to help affect change.

Development of a theory- and evidence-based intervention to enhance implementation of physical therapy guidelines for the management of low back pain

PubMed Central

2014-01-01

Background Systematic planning could improve the generally moderate effectiveness of interventions to enhance adherence to clinical practice guidelines. The aim of our study was to demonstrate how the process of Intervention Mapping was used to develop an intervention to address the lack of adherence to the national CPG for low back pain by Dutch physical therapists. Methods We systematically developed a program to improve adherence to the Dutch physical therapy guidelines for low back pain. Based on multi-method formative research, we formulated program and change objectives. Selected theory-based methods of change and practical applications were combined into an intervention program. Implementation and evaluation plans were developed. Results Formative research revealed influential determinants for physical therapists and practice quality managers. Self-regulation was appropriate because both the physical therapists and the practice managers needed to monitor current practice and make and implement plans for change. The program stimulated interaction between practice levels by emphasizing collective goal setting. It combined practical applications, such as knowledge transfer and discussion-and-feedback, based on theory-based methods, such as consciousness raising and active learning. The implementation plan incorporated the wider environment. The evaluation plan included an effect and process evaluation. Conclusions Intervention Mapping is a useful framework for formative data in program planning in the field of clinical guideline implementation. However, a decision aid to select determinants of guideline adherence identified in the formative research to analyse the problem may increase the efficiency of the application of the Intervention Mapping process. PMID:24428945

Materials development and evaluation for the ceramic helical expander

DOE Office of Scientific and Technical Information (OSTI.GOV)

Landingham, R.L.; Taylor, R.W.

The supporting role of the materials program for the ceramic helical expander program is described. The materials problems for this rotory expander in an extremely severe environment-a direct coal-fired Brayton topping cycle is defined. Readily available materials and methods for possible solution to these material problems as well as initiating some longer-range studies to improve reliability were evaluated. A preliminary screening of materials in hot coal-fired environments to select candidate materials and coating was made. More detailed evaluations of these candidate materials-reaction-bonded silicon nitride (RBSN) and Si--Al--O--N (Sialon) system- and coatings-chemical-vapor-deposited silicon nitride (CVD-Si/sub 3/N/sub 4/) and CVD-Sialon need tomore » be performed. Termination of the helical expander program abruptly stopped the materials program during this evaluation.« less

Materials development and evaluation for the ceramic helical expander

DOE Office of Scientific and Technical Information (OSTI.GOV)

Landingham, R.L.; Taylor, R.W.

The supporting role of the materials program for the ceramic helical expander program is described. The materials problems for this rotory expander in an extremely severe environment - a direct coal-fired Brayton topping cycle is defined. Readily available materials and methods are evaluated for possible solution to these material problems as well as initiating some longer-range studies to improve reliability. A preliminary screening of materials in hot coal-fired environments to select candidate materials and coating, was made, but there is a need to perform more detailed evaluations of these candidate materials-reaction-bonded silicon nitride (RBSN) and Si--Al--O--N (Sialon) system- and coatings-chemical-vapor-depositedmore » silicon nitride (CVD-Si/sub 3/N/sub 4/) and CVD-Sialon. Termination of the helical expander program abruptly stopped the materials program during this evaluation.« less

Effectiveness of the surviving the Teens® suicide prevention and depression awareness program: an impact evaluation utilizing a comparison group.

PubMed

Strunk, Catherine M; King, Keith A; Vidourek, Rebecca A; Sorter, Michael T

2014-12-01

Youth suicide is a serious public health issue in the United States. It is currently the third leading cause of death for youth aged 10 to 19. School-based prevention programs may be an effective method of educating youth and enhancing their help-seeking. Most school-based suicide prevention programs have not been rigorously evaluated for their effectiveness. This evaluation employs a comparison group to measure whether program group participants differed significantly from comparison group participants on pretest-posttest measures while assessing the immediate impact of the Surviving the Teens® Suicide Prevention and Depression Awareness Program. Findings indicate several positive outcomes in program group students' suicide and depression knowledge, attitudes, confidence, and behavioral intentions compared with the comparison group. Suicide prevention specialists and prevention planners may benefit from study findings. © 2014 Society for Public Health Education.

Qualitative Analysis of Written Reflections during a Teaching Certificate Program

PubMed Central

Castleberry, Ashley N.; Payakachat, Nalin; Ashby, Sarah; Nolen, Amanda; Carle, Martha; Neill, Kathryn K.

2016-01-01

Objective. To evaluate the success of a teaching certificate program by qualitatively evaluating the content and extent of participants’ reflections. Methods. Two investigators independently identified themes within midpoint and final reflection essays across six program years. Each essay was evaluated to determine the extent of reflection in prompted teaching-related topic areas (strengths, weaknesses, assessment, feedback). Results. Twenty-eight themes were identified within 132 essays. Common themes encompassed content delivery, student assessment, personal successes, and challenges encountered. Deep reflection was exhibited, with 48% of essays achieving the highest level of critical reflection. Extent of reflection trended higher from midpoint to final essays, with significant increases in the strengths and feedback areas. Conclusion. The teaching certificate program fostered critical reflection and self-reported positive behavior change in teaching, thus providing a high-quality professional development opportunity. Such programs should strongly consider emphasizing critical reflection through required reflective exercises at multiple points within program curricula. PMID:26941436

Analysis of Academic and Non-Academic Outcomes from a Bottom-up Comprehensive School Reform in the Absence of Student Level Data through Simulation Methods: A Mixed Methods Case Study

ERIC Educational Resources Information Center

Sondergeld, Toni A.

2009-01-01

This dissertation examines the efficacy of a bottom-up comprehensive school reform (CSR) program by evaluating its impact on student achievement, attendance, and behavior outcomes through an explanatory mixed methods design. The CSR program (Gear Up) was implemented in an urban junior high school over the course of seven years allowing for…

Usability of a Patient Education and Motivation Tool Using Heuristic Evaluation

PubMed Central

Arora, Mohit; Dai, Liwei; Price, Kathleen; Vizer, Lisa; Sears, Andrew

2009-01-01

Background Computer-mediated educational applications can provide a self-paced, interactive environment to deliver educational content to individuals about their health condition. These programs have been used to deliver health-related information about a variety of topics, including breast cancer screening, asthma management, and injury prevention. We have designed the Patient Education and Motivation Tool (PEMT), an interactive computer-based educational program based on behavioral, cognitive, and humanistic learning theories. The tool is designed to educate users and has three key components: screening, learning, and evaluation. Objective The objective of this tutorial is to illustrate a heuristic evaluation using a computer-based patient education program (PEMT) as a case study. The aims were to improve the usability of PEMT through heuristic evaluation of the interface; to report the results of these usability evaluations; to make changes based on the findings of the usability experts; and to describe the benefits and limitations of applying usability evaluations to PEMT. Methods PEMT was evaluated by three usability experts using Nielsen’s usability heuristics while reviewing the interface to produce a list of heuristic violations with severity ratings. The violations were sorted by heuristic and ordered from most to least severe within each heuristic. Results A total of 127 violations were identified with a median severity of 3 (range 0 to 4 with 0 = no problem to 4 = catastrophic problem). Results showed 13 violations for visibility (median severity = 2), 38 violations for match between system and real world (median severity = 2), 6 violations for user control and freedom (median severity = 3), 34 violations for consistency and standards (median severity = 2), 11 violations for error severity (median severity = 3), 1 violation for recognition and control (median severity = 3), 7 violations for flexibility and efficiency (median severity = 2), 9 violations for aesthetic and minimalist design (median severity = 2), 4 violations for help users recognize, diagnose, and recover from errors (median severity = 3), and 4 violations for help and documentation (median severity = 4). Conclusion We describe the heuristic evaluation method employed to assess the usability of PEMT, a method which uncovers heuristic violations in the interface design in a quick and efficient manner. Bringing together usability experts and health professionals to evaluate a computer-mediated patient education program can help to identify problems in a timely manner. This makes this method particularly well suited to the iterative design process when developing other computer-mediated health education programs. Heuristic evaluations provided a means to assess the user interface of PEMT. PMID:19897458

Integrate Evaluation into the Planning Process.

ERIC Educational Resources Information Center

Camp, William

1985-01-01

In an attempt to correct for limitations in the Program Evaluation and Review Technique-Critical Path Method (PERT-CPM), the Graphical Evaluation and Review Technique (GERT) has been developed. This management tool allows for evaluation during the facilities' development process. Two figures and two references are provided. (DCS)

Methodology of ETV Formative Evaluation.

ERIC Educational Resources Information Center

Duby, Aliza

Formative evaluation is discussed as it relates to educational television (ETV). Formative evaluation, used to improve a project during its development, is especially valuable in the production of television programs. Methods useful in the formative evaluation of ETV are: (1) try-out studies; (2) learner verification and revision; (3)…

Adding a Time-Series Design Element to the Success Case Method to Improve Methodological Rigor: An Application for Nonprofit Program Evaluation

ERIC Educational Resources Information Center

Coryn, Chris L. S.; Schroter, Daniela C.; Hanssen, Carl E.

2009-01-01

Brinkerhoff's Success Case Method (SCM) was developed with the specific purpose of assessing the impact of organizational interventions (e.g., training and coaching) on business goals by analyzing extreme groups using case study techniques and storytelling. As an efficient and cost-effective method of evaluative inquiry, SCM is attractive in other…

EVALUATION OF A MEASUREMENT METHOD FOR FOREST VEGETATION IN A LARGE-SCALE ECOLOGICAL SURVEY

EPA Science Inventory

We evaluate a field method for determining species richness and canopy cover of vascular plants for the Forest Health Monitoring Program (FHM), an ecological survey of U.S. forests. Measurements are taken within 12 1-m2 quadrats on 1/15 ha plots in FHM. Species richness and cover...

A Strengths-Based Group Intervention for Women Who Experienced Child Sexual Abuse

ERIC Educational Resources Information Center

Walker-Williams, Hayley J.; Fouché, Ansie

2017-01-01

Purpose: This study evaluated the benefits of a ''survivor to thriver'' strengths-based group intervention program to facilitate posttraumatic growth in women survivors of child sexual abuse. Method: A quasi-experimental, one group, pretest, posttest, time-delay design was employed using qualitative methods to evaluate the benefits of the…

DEVELOPMENT OF CRITERIA AND METHODS FOR EVALUATING TRAINER AIRCRAFT EFFECTIVENESS.

ERIC Educational Resources Information Center

KUSEWITT, J.B.

THE PURPOSE OF THIS STUDY WAS TO DEVELOP A METHOD FOR DETERMINING OBJECTIVE MEASURES OF TRAINER AIRCRAFT EFFECTIVENESS TO EVALUATE PROGRAM ALTERNATIVES FOR TRAINING PILOTS FOR FLEET FIGHTER AND ATTACK-TYPE AIRCRAFT. THE TRAINING SYLLABUS WAS BASED ON AVERAGE STUDENT ABILITY. THE BASIC PROBLEM WAS TO ESTABLISH QUANTITATIVE TIME-DIFFICULTY…

Evaluating Behaviorally Oriented Aviation Maintenance Resource Management (MRM) Training and Programs: Methods, Results, and Conclusions

NASA Technical Reports Server (NTRS)

Taylor, James C.; Thomas, Robert L., III

2003-01-01

Assessment of the impact of Aviation Resource Management Programs on aviation culture and performance has compelled a considerable body of research (Taylor & Robertson, 1995; Taylor, 1998; Taylor & Patankar, 2001). In recent years new methods have been applied to the problem of maintenance error precipitated by factors such as the need for self-assessment of communication and trust. The present study - 2002 -- is an extension of that past work. This research project was designed as the conclusion of a larger effort to help understand, evaluate and validate the impact of Maintenance Resource Management (MRM) training programs, and other MRM interventions on participant attitudes, opinions, behaviors, and ultimately on enhanced safety performance. It includes research and development of evaluation methodology as well as examination of psychological constructs and correlates of maintainer performance. In particular, during 2002, three issues were addressed. First, the evaluation of two (independent & different) MRM programs for changing behaviors was undertaken. In one case we were able to further apply the approach to measuring written communication developed during 2001 (Taylor, 2002; Taylor & Thomas, 2003). Second, the MRM/TOQ surveys were made available for completion on the internet. The responses from these on-line surveys were automatically linked to a results calculator (like the one developed and described in Taylor, 2002) to aid industry users in analyzing and evaluating their local survey data on the internet. Third, the main trends and themes from our research about MRM programs over the past dozen years were reviewed.

Evaluating Training Programs for Primary Care Providers in Child/Adolescent Mental Health in Canada: A Systematic Review

PubMed Central

Espinet, Stacey; Naqvi, Reza; Lingard, Lorelei; Steele, Margaret

2018-01-01

Introduction The need for child/adolescent mental health care in Canada is growing. Primary care can play a key role in filling this gap, yet most providers feel they do not have adequate training. This paper reviews the Canadian literature on capacity building programs in child and adolescent psychiatry for primary care providers, to examine how these programs are being implemented and evaluated to contribute to evidence-based initiatives. Methods A systematic literature review of peer-reviewed published articles of capacity building initiatives in child/adolescent mental health care for primary care practitioners that have been implemented in Canada. Results Sixteen articles were identified that met inclusion criteria. Analysis revealed that capacity building initiatives in Canada are varied but rigorous evaluation methodology is lacking. Primary care providers welcome efforts to increase mental health care capacity and were satisfied with the implementation of most programs. Discussion Objective conclusions regarding the effectiveness of these programs to increase mental health care capacity is challenging given the evaluation methodology of these studies. Conclusion Rigorous evaluation methods are needed to make evidence-based decisions on ways forward to be able to build child/adolescent mental health care capacity in primary care. Outcome measures need to move beyond self-report to more objective measures, and should expand the measurement of patient outcomes to ensure that these initiative are indeed leading to improved care for families. PMID:29662521

A Mixed Methods Evaluation of a 12-Week Insurance-Sponsored Weight Management Program Incorporating Cognitive-Behavioral Counseling

ERIC Educational Resources Information Center

Abildso, Christiaan; Zizzi, Sam; Gilleland, Diana; Thomas, James; Bonner, Daniel

2010-01-01

Physical activity is critical in healthy weight loss, yet there is still much to be learned about psychosocial mechanisms of physical activity behavior change in weight loss. A sequential mixed methods approach was used to assess the physical and psychosocial impact of a 12-week cognitive-behavioral weight management program and explore factors…

Culturally Grounded Prevention for Minority Youth Populations: A Systematic Review of the Literature

PubMed Central

Lauricella, Michela; Valdez, Jessica K.; Okamoto, Scott K.; Helm, Susana; Zaremba, Colleen

2016-01-01

Contemporary prevention science has focused on the application of cultural adaptations of evidence-based prevention programs for minority youth populations. Far less is known about culturally grounded methods that are intended to organically develop prevention programs within specific populations and communities. This article systematically reviews recent literature on culturally grounded interventions used to prevent health disparities in ethnic minority youth populations. In this review, we assessed 31 peer-reviewed articles published in 2003 or later that fit inclusionary criteria pertaining to the development and evaluation of culturally grounded prevention programs. The evaluated studies indicated different approaches toward cultural grounding, as well as specific populations, geographic regions, and health issues that have been targeted. Specifically, the findings indicated that most of the studies focused on the development and evaluation of culturally grounded HIV/STI and substance abuse prevention programs for Mexican American, African American, and American Indian/Alaska Native youth residing in the South or Southwestern U.S. These studies largely relied on community-based participatory or qualitative research methods to develop programs from the “ground up.” This review has implications for the development of future culturally grounded and culturally adapted prevention programs targeting underserved minority youth populations and geographic regions. Specifically, it identifies populations and regions where culturally grounded prevention efforts are underdeveloped or non-existent, providing some scientific direction for the future development of these types of programs. PMID:26733384

Culturally Grounded Prevention for Minority Youth Populations: A Systematic Review of the Literature.

PubMed

Lauricella, Michela; Valdez, Jessica K; Okamoto, Scott K; Helm, Susana; Zaremba, Colleen

2016-02-01

Contemporary prevention science has focused on the application of cultural adaptations of evidence-based prevention programs for minority youth populations. Far less is known about culturally grounded methods that are intended to organically develop prevention programs within specific populations and communities. This article systematically reviews recent literature on culturally grounded interventions used to prevent health disparities in ethnic minority youth populations. In this review, we assessed 31 peer-reviewed articles published in 2003 or later that fit inclusionary criteria pertaining to the development and evaluation of culturally grounded prevention programs. The evaluated studies indicated different approaches toward cultural grounding, as well as specific populations, geographic regions, and health issues that have been targeted. Specifically, the findings indicated that most of the studies focused on the development and evaluation of culturally grounded HIV/STI and substance abuse prevention programs for Mexican-American, African American, and American Indian/Alaska Native youth residing in the South or Southwestern US. These studies largely relied on community-based participatory or qualitative research methods to develop programs from the "ground up." This review has implications for the development of future culturally grounded and culturally adapted prevention programs targeting underserved minority youth populations and geographic regions. Specifically, it identifies populations and regions where culturally grounded prevention efforts are underdeveloped or non-existent, providing some scientific direction for the future development of these types of programs.

Zero-G Workstation Design

NASA Technical Reports Server (NTRS)

Gundersen, R. T.; Bond, R. L.

1976-01-01

Zero-g workstations were designed throughout manned spaceflight, based on different criteria and requirements for different programs. The history of design of these workstations is presented along with a thorough evaluation of selected Skylab workstations (the best zero-g experience available on the subject). The results were applied to on-going and future programs, with special emphasis on the correlation of neutral body posture in zero-g to workstation design. Where selected samples of shuttle orbiter workstations are shown as currently designed and compared to experience gained during prior programs in terms of man machine interface design, the evaluations were done in a generic sense to show the methods of applying evaluative techniques.

Evaluating the impact of geologic heritage on Earth science literacy: Adapting best practices from pedagogy and interpretation (Invited)

NASA Astrophysics Data System (ADS)

Semken, S. C.

2013-12-01

How might we authentically and practically evaluate the effects of a geologic heritage place or program on public Earth science literacy? This pedagogical form of evaluation is distinct from the evaluation of a place for its geological importance, heritage value, economic or cultural impact, and so on. Best evaluation practices from the realms of formal education, informal education, and interpretation start with a coherent set of evaluable learning outcomes, ideally recapitulated in one or more 'big ideas' that capture the essential attributes of the place or program. Learning outcomes may be classified as cognitive, affective, or psychomotor. Cognitive learning outcomes in a geoheritage context are the Earth-science concepts a visitor or student would be expected to uncover through on-site or virtual exploration of the stratigraphy, structure, landforms, and processes in a place. The Earth Science Literacy Principles (ESLP), and similar literacy documents relating to atmosphere, oceans, and climate; offer a template for mapping localized concepts onto more global ones. Quantitative instruments to evaluate understanding of the ESLP are in development, and the ESLP also map directly onto measures used in formal educational assessment, notably the Next Generation Science Standards in the USA. Nongeological place meanings (a component of sense of place) may suggest other cognitive outcomes. Affective learning outcomes for visitors and students in geoheritage sites are less readily defined, but may include place attachment (also a component of sense of place), attitudes, and interest. Multiple quantitative and qualitative methods of evaluating these outcomes exist. Psychomotor learning outcomes are even muddier, but accessibility (defined by statutes) offers a potential starting point. In practice, evaluation may be conducted synchronously or asynchronously with visitors' or students' interaction with the geoheritage place or program. Evaluation programs are typically constrained by access and practicality. Synchronous methods include observation, semi-structured interviews, rapid prototyping and surveys. Asynchronous methods include interviews, surveys, and tracking. Evaluation tools may require content or instrument validation for the specific context in which they are used. Illustrative examples from evaluation of public engagement with geoheritage places (National Parks) in the Southwest USA will be offered.

Using health promotion guidelines for persons with disabilities to develop and evaluate a physical activity program for individuals with multiple sclerosis: A feasibility study.

PubMed

Dixon-Ibarra, Alicia; Nery-Hurwit, Mara; Driver, Simon; MacDonald, Megan

2017-04-01

The Health Education for Persons with Multiple Sclerosis (HEMS) program was developed in response to the need for interventions aimed at increasing physical activity for individuals with Multiple Sclerosis (MS). It was developed and evaluated using Drum and colleagues (2009) guidelines for implementing health promotion programs for individuals with disabilities. The purpose of this feasibility study is to describe the development, implementation, and evaluation of the HEMS program. Thirteen individuals with MS completed the 8-week health education program. A mixed method approach for evaluation was implemented (i.e., survey and focus groups). Process and resource feasibility demonstrated that over half of the participants attended at least 80% of the weekly sessions. Focus group data provided valuable feedback for future iterations of the program including critiques on the delivery, content, and group support provided. Outcome evaluation showed increases in self-efficacy (survey), improvements in theoretical constructs (focus groups), and increased physical activity (focus groups). Results show that health promotion programs for persons with MS can improve physical activity and related constructs. Next steps will be to revise, implement, and reevaluate the HEMS program in a larger randomized control trial. Published by Elsevier Ltd.

Growing Community: The Impact of the Stephanie Alexander Kitchen Garden Program on the Social and Learning Environment in Primary Schools

ERIC Educational Resources Information Center

Block, Karen; Gibbs, Lisa; Staiger, Petra K.; Gold, Lisa; Johnson, Britt; Macfarlane, Susie; Long, Caroline; Townsend, Mardie

2012-01-01

This article presents results from a mixed-method evaluation of a structured cooking and gardening program in Australian primary schools, focusing on program impacts on the social and learning environment of the school. In particular, we address the Stephanie Alexander Kitchen Garden Program objective of providing a pleasurable experience that has…

The Effects of a Contingency Contracting Program on the Nocturnal Enuresis of Three Children

ERIC Educational Resources Information Center

Stover, Anthony C.; Dunlap, Glen; Neff, Bryon

2008-01-01

Objective: This study was conducted to evaluate the effects of a contingency contracting program to eliminate nocturnal enuresis of three children ages 5 to 12. Methods: The program was implemented by the children's primary caregivers, two of whom were foster parents and the third a case worker in the foster care system. The program was a package…

Adapting the Community of Inquiry Survey for an Online Graduate Program: Implications for Online Programs

ERIC Educational Resources Information Center

Kumar, Swapna; Ritzhaupt, Albert D.

2014-01-01

A cohort-based online professional doctorate program that consisted of both online coursework and research activities was designed using Garrison et al's community of inquiry (CoI) framework. The evaluation of the program proved a challenge because all existing CoI assessment methods in the past have dealt with online courses, not with online…

[Process evaluation in relation to effectiveness assessment: experiences with school-based programs].

PubMed

Ariza, Carles; Villalbí, Joan R; Sánchez-Martínez, Francesca; Nebot, Manel

2011-06-01

Evaluation of public health interventions usually focus on the quality of design and research methods, and less on the quality of the intervention or process evaluation. In process evaluation of school-based interventions, key issues are how completely the intervention is carried out and adherence to the protocol. In addition, exploration of intermediate variables, such as those that influence (and often predict) preventable behavior, is highly useful. This article describes the basic concepts in this topic, using examples of the effectiveness of some preventive interventions carried out in schools. The interventions discussed were mainly quasi-experimental studies, based on data from programs promoted by public health teams in the city of Barcelona. Data from process evaluation of preventive programs in secondary schools that underwent formal assessment of their effectiveness is provided. The examples are drawn from preventive programs of HIV infection or unprotected sexual intercourse (PRESSEC program) and drug consumption prevention (the PASE, PASE.bcn and x kpts programs). These examples show why the intervention process influences the impact of the programs and their results. Thorough planning of process evaluation is essential to obtain valid indicators that will identify, in the effectiveness evaluation of the intervention, the most efficacious strategies to obtain positive outcomes. Copyright © 2011 Sociedad Española de Salud Pública y Administración Sanitaria. Published by Elsevier Espana. All rights reserved.

Evaluating Active Parental Consent Procedures for School Programming: Addressing the Sensitive Topic of Suicide Prevention.

PubMed

Totura, Christine M Wienke; Kutash, Krista; Labouliere, Christa D; Karver, Marc S

2017-02-01

Suicide is the second leading cause of death for adolescents. Whereas school-based prevention programs are effective, obtaining active consent for youth participation in public health programming concerning sensitive topics is challenging. We explored several active consent procedures for improving participation rates. Five active consent methods (in-person, students taking forms home, mailing, mailing preceded by primers, mailing followed by reminder calls) were compared against passive consent procedures to evaluate recruitment success, as determined by participation (proportion who responded yes) and response (proportion who returned any response) rates. Participation acceptance rates ranged from 38 to 100% depending on consent method implemented. Compared with passive consent, active consent procedures were more variable in response and participation rates. In-person methods provided higher rates than less interpersonal methods, such as mailing or students taking consents home. Mailed primers before or reminder calls after consent forms were mailed increased response but not participation rates. Students taking consents home resulted in the lowest rates. Although passive consent produces the highest student participation, these methods are not always appropriate for programs addressing sensitive topics in schools. In-person active consent procedures may be the best option when prioritizing balance between parental awareness and successful student recruitment. © 2017, American School Health Association.

Model-based optimal design of experiments - semidefinite and nonlinear programming formulations

PubMed Central

Duarte, Belmiro P.M.; Wong, Weng Kee; Oliveira, Nuno M.C.

2015-01-01

We use mathematical programming tools, such as Semidefinite Programming (SDP) and Nonlinear Programming (NLP)-based formulations to find optimal designs for models used in chemistry and chemical engineering. In particular, we employ local design-based setups in linear models and a Bayesian setup in nonlinear models to find optimal designs. In the latter case, Gaussian Quadrature Formulas (GQFs) are used to evaluate the optimality criterion averaged over the prior distribution for the model parameters. Mathematical programming techniques are then applied to solve the optimization problems. Because such methods require the design space be discretized, we also evaluate the impact of the discretization scheme on the generated design. We demonstrate the techniques for finding D–, A– and E–optimal designs using design problems in biochemical engineering and show the method can also be directly applied to tackle additional issues, such as heteroscedasticity in the model. Our results show that the NLP formulation produces highly efficient D–optimal designs but is computationally less efficient than that required for the SDP formulation. The efficiencies of the generated designs from the two methods are generally very close and so we recommend the SDP formulation in practice. PMID:26949279

Model-based optimal design of experiments - semidefinite and nonlinear programming formulations.

PubMed

Duarte, Belmiro P M; Wong, Weng Kee; Oliveira, Nuno M C

2016-02-15

We use mathematical programming tools, such as Semidefinite Programming (SDP) and Nonlinear Programming (NLP)-based formulations to find optimal designs for models used in chemistry and chemical engineering. In particular, we employ local design-based setups in linear models and a Bayesian setup in nonlinear models to find optimal designs. In the latter case, Gaussian Quadrature Formulas (GQFs) are used to evaluate the optimality criterion averaged over the prior distribution for the model parameters. Mathematical programming techniques are then applied to solve the optimization problems. Because such methods require the design space be discretized, we also evaluate the impact of the discretization scheme on the generated design. We demonstrate the techniques for finding D -, A - and E -optimal designs using design problems in biochemical engineering and show the method can also be directly applied to tackle additional issues, such as heteroscedasticity in the model. Our results show that the NLP formulation produces highly efficient D -optimal designs but is computationally less efficient than that required for the SDP formulation. The efficiencies of the generated designs from the two methods are generally very close and so we recommend the SDP formulation in practice.

Educational Enrichment for Disadvantaged Inschool Neighborhood Youth Corps Enrollees During the Summer 1967. Evaluation of New York City Title I Educational Projects 1966-67.

ERIC Educational Resources Information Center

Williams, E. Belvin; Tannenbaum, Robert S.

A 1967 evaluation of New York City's Neighborhood Youth Corps (NYC) educational enrichment program presents the objectives and the methods of implementation of the Board of Education and the local community agencies which administered the project. Criticized are the diversity of the program objectives, the lack of provision for a specified…

A Review and Evaluation of the Langley Research Center's Scientific and Technical Information Program. Results of Phase VI-The Technical Report: A Survey and Analysis.

ERIC Educational Resources Information Center

McCullough, Robert A.; And Others

This report presents the results of a review and evaluation of the Langley Research Center's scientific and technical information program, which examined technical reports from various institutions and organizations to determine the organization of reports, the language used to convey information, and the methods used to present information.…

The Politics of Speaking: An Approach to Evaluating Bilingual-Bicultural Schools. Bilingual Education Paper Series, Vol. 1, No. 6.

ERIC Educational Resources Information Center

Erickson, Frederick

A method of evaluating bilingual-bicultural education programs that has a sociolinguistic basis uses samples of the language spoken by a number of bilingual program students as they go through their school day. A random sample of the child's speech would be continuously recorded for an hour, with a bilingual observer taking running notes on where…

Five Steps for Improving Evaluation Reports by Using Different Data Analysis Methods.

ERIC Educational Resources Information Center

Thompson, Bruce

Although methodological integrity is not the sole determinant of the value of a program evaluation, decision-makers do have a right, at a minimum, to be able to expect competent work from evaluators. This paper explores five areas where evaluators might improve methodological practices. First, evaluation reports should reflect the limited…

A Comparison of Formative and Summative Evaluation.

ERIC Educational Resources Information Center

Belenski, Mary Jo

Formative and summative evaluations in education are compared, and appropriate uses of these methods in program evaluation are discussed. The main purpose of formative evaluation is to determine a level of mastery of a learning task, along with discovering any part of the task that was not mastered. In other words, formative evaluation focuses the…

Emphasizing Public Health Within a Health Information Exchange: An Evaluation of the District of Columbia’s Health Information Exchange Program

PubMed Central

Goldwater, Jason; Jardim, Juliette; Khan, Tasnuva; Chan, Karen

2014-01-01

Introduction: Clinovations Government Solutions (CGS) was contracted in 2013 to conduct a mixed-methods evaluation of the District of Columbia (D.C.) Health Information Exchange (HIE) program as part of their Cooperative Agreement Grant funded by the Office of the National Coordinator in 2010. The evaluation was to focus on the progress of the HIE, how many providers and hospitals were participating in the program, and what benefits were being realized through the use of the HIE. During the course of the evaluation, the CGS team found that the use of the HIE to support public health reporting was one of its core elements. Background: The D.C. HIE is one of 56 HIE that were funded out of the Cooperative Agreement program. The HIE program was managed by the District of Columbia Department of Health Care Finance (DHCF), which also manages the District of Columbia Medicaid Program. The program was initially designed to accomplish the following: developing state-level directories and enabling technical services for HIE within and across states; ensuring an effective model for governance and accountability; coordinating an integrated approach with Medicaid and public health; and developing or updating privacy and security requirements for HIE within and across state borders. As the evaluation progressed, the CGS team discovered that the relationship between the DHCF and the District of Columbia Department of Health (DOH) had become a cornerstone of the D.C. HIE program. Methods: The CGS team used a mixed-methods approach for the evaluation, including a review of documents developed by the DHCF in its HIE program, including its original application. We also conducted 10 key informant interviews and moderated two small-group discussions using a semistructured protocol; and we developed a survey that measured the use, satisfaction, and future sustainability of the HIE for over 200 providers within the District of Columbia. Findings: While the evaluation focused on the D.C. HIE program in its entirety, the results indicated the value of utilizing the HIE for public health reporting to enhance the surveillance activities of the DOH. Specifically, the DHCF and DOH collaboration resulted in using the HIE to electronically capture and report immunization data; and in requiring electronic lab reporting and results as part of the Meaningful Use Requirement—which can assist in detecting HIV/AIDS and providing better care for the district’s high population of individuals with HIV/AIDS. Electronic lab reporting and electronic prescribing within the HIE can assist the DOH and providers in identifying specific diseases, such as tuberculosis and viral hepatitis, before they affect a significant part of the population. Discussion: Given the severe health disparities in the district, the ability of the D.C. HIE program to collect public health information on affected populations will be instrumental in better understanding and identifying methods of supporting these populations through improved surveillance and identification of the appropriate treatments. The D.C. HIE program is uniquely positioned to support these populations due to the partnership of DHCF with the D.C. DOH. Conclusion and Next Steps: The District of Columbia has made significant strides in expanding its public health infrastructure and activities. Three key areas of growth were identified that have the potential to transform the District of Columbia’s public health approach: establishing sufficient feedback loops, collection of environmental data, integration, and interoperability. PMID:25848617

Laboratory and field evaluations of non-bituminous hot melt pavement marker adhesive Georgia test deck : two year evaluation (fall 2006 product submissions).

DOT National Transportation Integrated Search

2008-10-01

Purpose: : The purpose of the National Transportation Product Evaluation Program (NTPEP) is to provide a cost-effective method of evaluation for materials of common interest among all participating NTPEP member departments. NTPEP reports allow member...

A Holistic Approach to Evaluating Vocational Education: Traditional Chinese Physicians (TCP) Model.

ERIC Educational Resources Information Center

Lee, Lung-Sheng; Chang, Liang-Te

Conventional approaches to evaluating vocational education have often been criticized for failing to deal holistically with the institution or program being evaluated. Integrated quantitative and qualitative evaluation methods have documented benefits; therefore, it would be useful to consider possibility of developing a model for evaluating…

Evaluation of Educational Administration: A Decade Review of Research (2001-2010)

ERIC Educational Resources Information Center

Parylo, Oksana

2012-01-01

This sequential mixed methods study analyzed how program evaluation was used to assess educational administration and examined thematic trends in educational evaluation published over 10 years (2001-2010). First, qualitative content analysis examined the articles in eight peer-reviewed evaluation journals. This analysis revealed that numerous…

Demand generation activities and modern contraceptive use in urban areas of four countries: a longitudinal evaluation

PubMed Central

Speizer, Ilene S; Corroon, Meghan; Calhoun, Lisa; Lance, Peter; Montana, Livia; Nanda, Priya; Guilkey, David

2014-01-01

ABSTRACT Family planning is crucial for preventing unintended pregnancies and for improving maternal and child health and well-being. In urban areas where there are large inequities in family planning use, particularly among the urban poor, programs are needed to increase access to and use of contraception among those most in need. This paper presents the midterm evaluation findings of the Urban Reproductive Health Initiative (Urban RH Initiative) programs, funded by the Bill & Melinda Gates Foundation, that are being implemented in 4 countries: India (Uttar Pradesh), Kenya, Nigeria, and Senegal. Between 2010 and 2013, the Measurement, Learning & Evaluation (MLE) project collected baseline and 2-year longitudinal follow-up data from women in target study cities to examine the role of demand generation activities undertaken as part of the Urban RH Initiative programs. Evaluation results demonstrate that, in each country where it was measured, outreach by community health or family planning workers as well as local radio programs were significantly associated with increased use of modern contraceptive methods. In addition, in India and Nigeria, television programs had a significant effect on modern contraceptive use, and in Kenya and Nigeria, the program slogans and materials that were blanketed across the cities (eg, leaflets/brochures distributed at health clinics and the program logo placed on all forms of materials, from market umbrellas to health facility signs and television programs) were also significantly associated with modern method use. Our results show that targeted, multilevel demand generation activities can make an important contribution to increasing modern contraceptive use in urban areas and could impact Millennium Development Goals for improved maternal and child health and access to reproductive health for all. PMID:25611476

Programs to improve adolescent sexual and reproductive health in the US: a review of the evidence.

PubMed

Manlove, Jennifer; Fish, Heather; Moore, Kristin Anderson

2015-01-01

US adolescents have high rates of teen pregnancy, childbearing, and sexually transmitted infections (STIs), highlighting the need to identify and implement effective programs that will help improve teen sexual and reproductive health. This review identified 103 random-assignment evaluations of 85 programs that incorporated intent-to-treat analyses and assessed impacts on pregnancy, childbearing, STIs, and their key determinants - sexual activity, number of sexual partners, condom use, and other contraceptive use - among teens. This review describes the evidence base for five broad program approaches, including abstinence education, comprehensive sex education, clinic-based programs, youth development programs, and parent-youth relationship programs. We also describe programs with impacts on key outcomes, including pregnancy/childbearing, STIs, and those that found impacts on both sexual activity and contraceptive use. Our review identified 52 effective programs: 38 with consistent impacts on reproductive health outcomes, and 14 with mixed findings (across subpopulations, follow-ups, or multiple measures of a single outcome). We found that a variety of program approaches produced impacts on sexual and reproductive health outcomes. Parent-youth relationship programs and clinic-based program evaluations more frequently showed impacts than other program approaches, although we also identified a number of abstinence-education, comprehensive sex education, and youth-development programs with impacts on sexual and reproductive health outcomes. Overall, we identified nine program evaluations with impacts on teen pregnancies or births, five with impacts on reducing STIs, and 15 with impacts on both delaying/reducing sexual activity and increasing contraceptive use (including condom use). Future efforts should conduct replications of existing program evaluations, identify implementation components linked to impacts, rigorously evaluate programs that appear promising, and expand the evidence base on programs that impact hormonal and long-acting contraceptive method use.

Process evaluation of two home-based bimanual training programs in children with unilateral cerebral palsy (the COAD-study): protocol for a mixed methods study.

PubMed

Beckers, Laura; van der Burg, Jan; Janssen-Potten, Yvonne; Rameckers, Eugène; Aarts, Pauline; Smeets, Rob

2018-04-24

As part of the COAD-study two home-based bimanual training programs for young children with unilateral Cerebral Palsy (uCP) have been developed, both consisting of a preparation phase and a home-based training phase. Parents are coached to use either an explicit or implicit motor learning approach while teaching bimanual activities to their child. A process evaluation of these complex interventions is crucial in order to draw accurate conclusions and provide recommendations for implementation in clinical practice and further research. The aim of the process evaluation is to systematically assess fidelity of the home-based training programs, to examine the mechanisms that contribute to their effects on child-related and parent-related outcomes, and to explore the influence of contextual factors. A mixed methods embedded design is used that emerges from a pragmatism paradigm. The qualitative strand involves a generic qualitative approach. The process evaluation components fidelity (quality), dose delivered (completeness), dose received (exposure and satisfaction), recruitment and context will be investigated. Data collection includes registration of attendance of therapists and remedial educationalists to a course regarding the home-based training programs; a questionnaire to evaluate this course by the instructor; a report form concerning the preparation phase to be completed by the therapist; registration and video analyses of the home-based training; interviews with parents and questionnaires to be filled out by the therapist and remedial educationalist regarding the process of training; and focus groups with therapists and remedial educationalists as well as registration of drop-out rates and reasons, to evaluate the overall home-based training programs. Inductive thematic analysis will be used to analyse qualitative data. Qualitative and quantitative findings are merged through meta-inference. So far, effects of home-based training programs in paediatric rehabilitation have been studied without an extensive process evaluation. The findings of this process evaluation will have implications for clinical practice and further research regarding development and application of home-based bimanual training programs, executed by parents and aimed at improving activity performance and participation of children with uCP.

Preparation for an online asynchronous university doctoral course. Lessons learned.

PubMed

Milstead, J A; Nelson, R

1998-01-01

This article addresses the development of the initial course in the first completely online doctoral program in nursing. Synchronous and asynchronous methods of distance education were assessed. Planning focused at the university, school, and course levels. University planning involved the technical infrastructure, registration, student services, and library services. School planning examined administrative commitment and faculty commitment and willingness. Course planning focused on marketing, precourse information, time frame, modular design, planned interaction, and professor availability and support. Implementation issues centered on getting students connected, learning the software, changing instructional methods, and managing chats. Traditional methods of evaluating student learning and course evaluation were supplemented with the development of qualitative and quantitative tools to gather data for making administrative decisions. The Dean and faculty agreed that the internet was an effective method of delivering content in the initial Health Policy course. The Dean and faculty agreed to continue the PhD program online for one cohort and continue to evaluate student progress and faculty and student satisfaction.

Interactive program for analysis and design problems in advanced composites technology

NASA Technical Reports Server (NTRS)

Cruse, T. A.; Swedlow, J. L.

1971-01-01

During the past year an experimental program in the fracture of advanced fiber composites has been completed. The experimental program has given direction to additional experimental and theoretical work. A synthesis program for designing low weight multifastener joints in composites is proposed, based on extensive analytical background. A number of failed joints have been thoroughly analyzed to evaluate the failure hypothesis used in the synthesis procedure. Finally, a new solution is reported for isotropic and anisotropic laminates using the boundary-integral method. The solution method offers significant savings of computer core and time for important problems.

Evaluating the Effectiveness of Programs to Improve Educational Attainment of Unwed African American Teen Mothers: A Meta Analysis

ERIC Educational Resources Information Center

Baytop, Chanza M.

2006-01-01

A study implements meta-analytic methods to synthesize the findings and analyze the effects of interventions, including secondary teen pregnancy prevention programs, on educational achievement among unwed African American teen mothers. Results indicate that secondary teen pregnancy prevention programs and other interventions for adolescent mothers…

Evaluation of an In-Service Training Program for Child Welfare Practitioners

ERIC Educational Resources Information Center

Turcotte, Daniel; Lamonde, Genevieve; Beaudoin, Andre

2009-01-01

Objective: To test the effectiveness of an in-training program for practitioners in public child welfare organizations. Method: The sample consists of practitioners (N = 945) working in youth centers or in local community service centers. Data are collected through self-administered questionnaires prior to and after the program. Results: The data…

A Digital Program Informs Low-Income Caregivers of Preschool-Age Children about Family Meals

ERIC Educational Resources Information Center

Lohse, Barbara; Rifkin, Robin; Arnold, Kristen; Least, Christine

2012-01-01

Objective: To evaluate the digital program, "Mealtime is Family Time", as a means of educating caregivers of preschoolers on the importance of family meals within the division of feeding responsibility framework. Methods: Descriptive design using 2 approaches: focus group program review and discussion or self-report survey after independent…

An Abstinence Program's Impact on Cognitive Mediators and Sexual Initiation

ERIC Educational Resources Information Center

Weed, Stan E.; Ericksen, Irene H.; Lewis, Allen; Grant, Gale E.; Wibberly, Kathy H.

2008-01-01

Objectives: To evaluate the impact of an abstinence education program on sexual intercourse initiation and on possible cognitive mediators of sexual initiation for virgin seventh graders in suburban Virginia. Methods: Measures of sexual behavior and 6 mediating variables were compared at 3 time periods for program participants and a matched…

Do Premarital Education Programs Really Work? A Meta-Analytic Study

ERIC Educational Resources Information Center

Fawcett, Elizabeth B.; Hawkins, Alan J.; Blanchard, Victoria L.; Carroll, Jason S.

2010-01-01

Previous studies (J. S. Carroll & W. J. Doherty, 2003) have asserted that premarital education programs have a positive effect on program participants. Using meta-analytic methods of current best practices to look across the entire body of published and unpublished evaluation research on premarital education, we found a more complex pattern of…

Evaluating Active Parental Consent Procedures for School Programming: Addressing the Sensitive Topic of Suicide Prevention

ERIC Educational Resources Information Center

Totura, Christine M. Wienke; Kutash, Krista; Labouliere, Christa D.; Karver, Marc S.

2017-01-01

Background: Suicide is the second leading cause of death for adolescents. Whereas school-based prevention programs are effective, obtaining active consent for youth participation in public health programming concerning sensitive topics is challenging. We explored several active consent procedures for improving participation rates. Methods: Five…

Using Critical Appraisal to Inform Program Improvement

ERIC Educational Resources Information Center

Esterlund, Theresa; Krantz, Amanda; Sigmond, Catherine

2017-01-01

In 2016, the education department at the National Building Museum offered a series of Late Night programs in conjunction with the Summer Block Party installation, ICEBERGS. The museum wanted to evaluate the programs but had limited resources to do so. In the following article, we describe the use of critical appraisal, an economic method applied…

Evaluation of Parent and Child Enhancement (PACE) Program: Randomized Controlled Trial

ERIC Educational Resources Information Center

Leung, Cynthia; Tsang, Sandra; Lo, Cyrus

2017-01-01

Objective: This study examined the efficacy of the Parent and Child Enhancement (PACE) program on child learning, child behavior problems, and parental stress, using randomized controlled trial design, in social services centers. Methods: Eligibility criteria were (1) children aged 2 years at program commencement, (2) low-income, new immigrant, or…

Implementing a Project-Based Learning Model in a Pre-Service Leadership Program

ERIC Educational Resources Information Center

Albritton, Shelly; Stacks, Jamie

2016-01-01

This paper describes two instructors' efforts to more authentically engage students in a preservice leadership program's course called Program Planning and Evaluation by using a project-based learning approach. Markham, Larmer, and Ravitz (2003) describe project-based learning (PjBL) as "a systematic teaching method that engages students in…

Assessing the New Competencies for Resident Education: A Model from an Emergency Medicine Program.

ERIC Educational Resources Information Center

Reisdorff, Earl J.; Hayes, Oliver W.; Carlson, Dale J.; Walker, Gregory L.

2001-01-01

Based on the experience of Michigan State University's emergency medicine residency program, proposes a practical method for modifying an existing student evaluation format. The model provides a template other programs could use in assessing residents' acquisition of the knowledge, skills, and attitudes reflected in the six general competencies…

EXPERIMENTATION AND INNOVATION--A SEVEN-YEAR REPORT.

ERIC Educational Resources Information Center

DI LORENZO, LOUIS T.; SALTER, RUTH

REPORTED IN AN EVALUATION OF A PROGRAM OF STATE AID FOR EXPERIMENTAL PROGRAMS WAS AN OVERVIEW OF THE LEGISLATION AND ADMINISTRATION GOVERNING SUCH PROGRAMS. A PURPOSE OF THE FINANCIAL AID TO LOCAL SCHOOL DISTRICTS WILLING TO EXPERIMENT WITH INNOVATIONS TO IMPROVE INSTRUCTIONAL QUALITY WAS TO PROMOTE THE SCIENTIFIC METHOD IN ATTACKING TEACHING AND…

Developing medical educators – a mixed method evaluation of a teaching education program

PubMed Central

Roos, Marco; Kadmon, Martina; Kirschfink, Michael; Koch, Eginhard; Jünger, Jana; Strittmatter-Haubold, Veronika; Steiner, Thorsten

2014-01-01

Background It is well accepted that medical faculty teaching staff require an understanding of educational theory and pedagogical methods for effective medical teaching. The purpose of this study was to evaluate the effectiveness of a 5-day teaching education program. Methods An open prospective interventional study using quantitative and qualitative instruments was performed, covering all four levels of the Kirkpatrick model: Evaluation of 1) ‘Reaction’ on a professional and emotional level using standardized questionnaires; 2) ‘Learning’ applying a multiple choice test; 3) ‘Behavior’ by self-, peer-, and expert assessment of teaching sessions with semistructured interviews; and 4) ‘Results’ from student evaluations. Results Our data indicate the success of the educational intervention at all observed levels. 1) Reaction: The participants showed a high acceptance of the instructional content. 2) Learning: There was a significant increase in knowledge (P<0.001) as deduced from a pre-post multiple-choice questionnaire, which was retained at 6 months (P<0.001). 3) Behavior: Peer-, self-, and expert-assessment indicated a transfer of learning into teaching performance. Semistructured interviews reflected a higher level of professionalism in medical teaching by the participants. 4) Results: Teaching performance ratings improved in students’ evaluations. Conclusions Our results demonstrate the success of a 5-day education program in embedding knowledge and skills to improve performance of medical educators. This multimethodological approach, using both qualitative and quantitative measures, may serve as a model to evaluate effectiveness of comparable interventions in other settings. PMID:24679671

A Proposed Resident's Operative Case Tracking and Evaluation System.

PubMed

Sehli, Deema N; Esene, Ignatius N; Baeesa, Saleh S

2016-03-01

Neurosurgery program trainers are continuously searching for new methods to evaluate trainees' competency besides number of cases and training duration. Recently, efforts are made on the development of reliable methods to teach competency and valid methods to measure teaching efficacy. Herein, we propose the "Resident's Operative Case Tracking and Evaluation System" (ROCTES) for the assessment and monitoring of the resident's performance quality during each procedure. We developed a data-based website and smartphone application for neurosurgical attending physicians, residents, and resident review committees in our accredited neurosurgical institutions. ROCTES runs through five steps: Login (Resident), Case Entry, Login (Attending Physician), Case Approval and Evaluation, and Report. The Resident enters each case record under "Case Entry" field and can "save," "edit," or "submit" the case data to the Attending Physician. The latter from the attending physician login profile will be able to "approve and evaluate" the resident's "knowledge," "skills," and "attitude" ranking from 1 to 15 for that particular case; add his comments and then "save," "edit," or "submit" the data, which can be viewed by users as a "report." Program Directors can also "login" to monitor the resident's progress. The implementation of this communication tool should enable the filtering and retrieval of information needed for the better assessment and monitoring of residents' exposure to variety of cases in each training center. This proposed evaluation system will provide a transparent assessment for residency training programs and should convert trainees into competent neurosurgeons. Copyright © 2016 Elsevier Inc. All rights reserved.

When science and politics listen to each other: good prospects from a new school breakfast program in Peru.

PubMed

Jacoby, E R; Cueto, S; Pollitt, E

1998-04-01

This article provides an overview of a school breakfast program implemented in 1993 in the Peruvian Andes. The program, designed by the Instituto de Investigación Nutricional in Lima and supported by the government of Peru, constitutes a clear departure from previous school feeding programs, which were heavily politicized and poorly documented. From the program's inception, nutritionists, managers, and social scientists have collaborated to produce a sound nutritional design, efficient distribution mechanisms, and effective evaluation methods. During the program's first year, controlled evaluations conducted in several Andean regions documented improved dietary intake and a significant decline in the prevalence of anemia. An educational evaluation also found improved verbal skills, higher school attendance, and lower dropout rates among recipients of the school breakfast. The results have prompted the Peruvian government to continue supporting the program, thus setting a new standard for the effective management of social expenditure in the context of economic adjustment.

A Systematic Review of the Effects of Continuing Education Programs on Providing Clinical Community Pharmacy Services

PubMed Central

Marques dos Reis, Tiago; Guidoni, Camilo Molino; Girotto, Edmarlon; Guerra, Marisabelle Lima; de Oliveira Baldoni, André; Leira Pereira, Leonardo Régis

2016-01-01

Objective. To summarize the effects of media methods used in continuing education (CE) programs on providing clinical community pharmacy services and the methods used to evaluate the effectiveness of these programs. Methods. A systematic review was performed using Medline, SciELO, and Scopus databases. The timeline of the search was 1990 to 2013. Searches were conducted in English, Portuguese, and Spanish. Results. Nineteen articles of 3990 were included. Fourteen studies used only one media method, and the live method (n=11) was the most frequent (alone or in combination). Only two studies found that the CE program was ineffective or partially effective; these studies used only the live method. Most studies used nonrobust, nonvalidated, and nonstandardized methods to measure effectiveness. The majority of studies focused on the effect of the CE program on modifying the knowledge and skills of the pharmacists. One study assessed the CE program’s benefits to patients or clients. Conclusion. No evidence was obtained regarding which media methods are the most effective. Robust and validated methods, as well as assessment standardization, are required to clearly determine whether a particular media method is effective. PMID:27402991

77 FR 40358 - Meeting of the Scientific Advisory Committee on Alternative Toxicological Methods (SACATM)

Federal Register 2010, 2011, 2012, 2013, 2014

2012-07-09

... Alternative Toxicological Methods (SACATM) AGENCY: Division of the National Toxicology Program (DNTP.../live ). SACATM advises the Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM), the NTP Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM...

5 CFR 470.101 - Statutory authority.

Code of Federal Regulations, 2012 CFR

2012-01-01

... MANAGEMENT RESEARCH PROGRAMS AND DEMONSTRATIONS PROJECTS General Provisions § 470.101 Statutory authority. (a... programs to study improved methods and technologies in Federal personnel management; (2) Evaluate the... for the collection and public dissemination of information relating to personnel management research...

5 CFR 470.101 - Statutory authority.

Code of Federal Regulations, 2014 CFR

2014-01-01

... MANAGEMENT RESEARCH PROGRAMS AND DEMONSTRATIONS PROJECTS General Provisions § 470.101 Statutory authority. (a... programs to study improved methods and technologies in Federal personnel management; (2) Evaluate the... for the collection and public dissemination of information relating to personnel management research...

5 CFR 470.101 - Statutory authority.

Code of Federal Regulations, 2013 CFR

2013-01-01

... MANAGEMENT RESEARCH PROGRAMS AND DEMONSTRATIONS PROJECTS General Provisions § 470.101 Statutory authority. (a... programs to study improved methods and technologies in Federal personnel management; (2) Evaluate the... for the collection and public dissemination of information relating to personnel management research...

7 CFR 3403.10 - Proposal review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... Regulations of the Department of Agriculture (Continued) COOPERATIVE STATE RESEARCH, EDUCATION, AND EXTENSION SERVICE, DEPARTMENT OF AGRICULTURE SMALL BUSINESS INNOVATION RESEARCH GRANTS PROGRAM Submission and... in the “Method of Selection and Evaluation Criteria” section of the program solicitation. (d...

Two autowire versions for CDC-3200 and IBM-360

NASA Technical Reports Server (NTRS)

Billingsley, J. B.

1972-01-01

Microelectronics program was initiated to evaluate circuitry, packaging methods, and fabrication approaches necessary to produce completely procured logic system. Two autowire programs were developed for CDC-3200 and IBM-360 computers for use in designing logic systems.

Cooperative Learning--Part 3. Electronic Cooperative Quizzes.

ERIC Educational Resources Information Center

Jensen, Murray; Moore, Randy; Hatch, Jay

2002-01-01

Introduces the Electronic Cooperative Quiz (ECQ) program which aims to investigate the implementation of cooperative quizzes to the World Wide Web. Describes the creation and design of the program and evaluates the two different methods used with students. (YDS)