25 CFR 36.50 - Standard XVII-School program evaluation and needs assessment.

Code of Federal Regulations, 2013 CFR

2013-04-01

... each school, Agency or Area, as appropriate, a standardized needs assessment and evaluation instrument... 25 Indians 1 2013-04-01 2013-04-01 false Standard XVII-School program evaluation and needs assessment. 36.50 Section 36.50 Indians BUREAU OF INDIAN AFFAIRS, DEPARTMENT OF THE INTERIOR EDUCATION...

25 CFR 36.50 - Standard XVII-School program evaluation and needs assessment.

Code of Federal Regulations, 2012 CFR

2012-04-01

... each school, Agency or Area, as appropriate, a standardized needs assessment and evaluation instrument... 25 Indians 1 2012-04-01 2011-04-01 true Standard XVII-School program evaluation and needs assessment. 36.50 Section 36.50 Indians BUREAU OF INDIAN AFFAIRS, DEPARTMENT OF THE INTERIOR EDUCATION...

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed

Colantonio, A

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed Central

Colantonio, A.

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations. PMID:2728498

Implementation of the Career Education Incentive Act. First Interim Report on the Evaluability Assessment.

ERIC Educational Resources Information Center

Jung, Steven M.; And Others

Survey activities are reported which were designed to provide the foundation for a national evaluation of the effectiveness of programs assisted under the Career Education Incentive Act of 1977 (PL 95-207). The methodology described, called "program evaluability assessment," focuses on detailed analysis of program assumptions in order to…

Needs assessment and evaluation of a short course to improve faculties teaching skills at a former World Health Organization regional teacher training center

PubMed Central

KOJURI, JAVAD; AMINI, MITRA; KARIMIAN, ZAHRA; DEHGHANI, MOHAMMAD REZA; SABER, MAHBOOBEH; BAZRAFCAN, LEILA; EBRAHIMI, SEDIGHEH; REZAEE, RITA

2015-01-01

Introduction: In the design of educational programs, much attention has been paid to teaching methods, needs assessment, an important part of the development of educational programs, generally is neglected. Another important aspect in educational program design is assessing effectiveness. The aims of this study were to design a formal needs assessment program to define the core contents of a faculty development program, and to determine whether participation in the faculty development program reinforced new teaching skills. Methods: A teacher-training program was designed at Shiraz University of Medical Sciences to help medical instructors boost their teaching skills. Needs assessment was done with nominal group technique followed by a 5-point Likert scale questionnaire. The program, imparted in workshop format, covered effective teaching methods, feedback, assessing knowledge and time management. Instruction was in the form of lectures, group discussions, case simulations, video presentations and role-plays. The program was evaluated in several phases using data triangulation and multi-item assessments of overall program quality in three major dimensions: Kirkpatrick program evaluation model, evaluation of the educational environment and qualitative analysis with open-ended questions. All participants in the study belonged to the academic staff of Shiraz University of Medical Sciences (n=396). Results: Seven main categories were derived from nominal group techniques and questionnaires. After the program, participants rated the quality of the program highly. They felt that the educational intervention was appropriate and had a positive impact on their knowledge of effective teaching methods, feedback, knowledge assessment and time management. Assessment of the effectiveness of the program showed that participants reported significant improvements in their teaching abilities. Conclusions: Our faculty development program  have a significant positive effect on medical university teaching staff members’ competencies. Further research is needed to investigate whether the faculty development program actually results in improved teaching performance. PMID:25587548

Computer Aided Instruction in Teaching Program Evaluation.

ERIC Educational Resources Information Center

Dowell, David A.; Binette, Holly A. Lizotte

This paper reports the results of two semesters of experience using computer-assisted instruction (CAI) to teach topics in program evaluation to undergraduate and graduate psychology students at California State University, Long Beach. (The topics addressed are models of evaluation, evaluability assessment, needs assessment, experimental and…

Responding to the Crisis of Accountability: A Review of Program Assessment Methods.

ERIC Educational Resources Information Center

Haley, Eric; Jackson, DeForrest

The advertising program at the University of Tennessee, Knoxville (UTK) has at least 12 measures of program assessment, which serve as a basis for discussion rather than as a prescription for an effective assessment program. The program assessment methods are accrediting, internal program review, teaching evaluations, a university survey of…

Developing and Using a Logic Model for Evaluation and Assessment of University Student Affairs Programming: A Case Study

ERIC Educational Resources Information Center

Cooper, Jeff

2009-01-01

This dissertation addresses theory and practice of evaluation and assessment in university student affairs, by applying logic modeling/program theory to a case study. I intend to add knowledge to ongoing dialogue among evaluation scholars and practitioners on student affairs program planning and improvement as integral considerations that serve…

Military Emergency Medical Service System Assessment: Application of the National Park Service Needs Assessment and Program Audit to Objectively Evaluate the Military EMS System of Okinawa, Japan.

PubMed

Ross, Elliot M; Harper, Stephen A; Cunningham, Cord; Walrath, Benjamin D; DeMers, Gerard; Kharod, Chetan U

2017-03-01

As part of a Military Emergency Medical Services (EMS) system process improvement initiative, the authors sought to objectively evaluate the U.S. military EMS system for the island of Okinawa. They applied a program evaluation tool currently utilized by the U.S. National Park Service (NPS). A comprehensive needs assessment was conducted to evaluate the current Military EMS system in Okinawa, Japan. The NPS EMS Program Audit Worksheet was used to get an overall "score" of our assessment. After all the data had been collected, a joint committee of Military EMS physicians reviewed the findings and made formal recommendations. From 2011 to 2014, U.S. military EMS on Okinawa averaged 1,345 ± 137 patient transports annually. An advanced life support (ALS) provider would have been dispatched on 558 EMS runs (38%) based on chief complaint in 2014 had they been available. Over 36,000 man-hours were expended during this period to provide National Registry Emergency Medical Technician (EMT)-accredited instruction to certify 141 Navy Corpsman as EMT Basics. The NPS EMS Program Audit Worksheet was used and the program scored a total of 31, suggesting the program is well planned and operating within standards. This evaluation of the Military EMS system on Okinawa using the NPS program assessment and audit worksheet demonstrates the NPS evaluation instruments may offer a useful assessment tool for the evaluation of Military EMS systems. Reprint & Copyright © 2017 Association of Military Surgeons of the U.S.

Feedback Improvement in Automatic Program Evaluation Systems

ERIC Educational Resources Information Center

Skupas, Bronius

2010-01-01

Automatic program evaluation is a way to assess source program files. These techniques are used in learning management environments, programming exams and contest systems. However, use of automated program evaluation encounters problems: some evaluations are not clear for the students and the system messages do not show reasons for lost points.…

Texas Nutrition Education and Training Program. Final Evaluation Report for Federal Fiscal Year 1994.

ERIC Educational Resources Information Center

Ahmad, Mahassen

This report presents federally mandated evaluations and needs assessments conducted by the Texas Nutrition Education and Training (NET) program during Federal Fiscal Year 1994. General program performance of NET reflected an increase in the number of children in Texas and expansion in NET program activities. Needs assessment data collected from…

Residency Programs' Evaluations of the Competencies: Data Provided to the ACGME About Types of Assessments Used by Programs

PubMed Central

Holt, Kathleen D.; Miller, Rebecca S.; Nasca, Thomas J.

2010-01-01

Background In 1999, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project began to focus on resident performance in the 6 competencies of patient care, medical knowledge, professionalism, practice-based learning and improvement, interpersonal communication skills, and professionalism. Beginning in 2007, the ACGME began collecting information on how programs assess these competencies. This report provides information on the nature and extent of those assessments. Methods Using data collected by the ACGME for site visits, we use descriptive statistics and percentages to describe the number and type of methods and assessors accredited programs (n  =  4417) report using to assess the competencies. Observed differences among specialties, methodologies, and assessors are tested with analysis of variance procedures. Results Almost all (>97%) of programs report assessing all of the competencies and using multiple methods and multiple assessors. Similar assessment methods and evaluator types were consistently used across the 6 competencies. However, there were some differences in the use of patient and family as assessors: Primary care and ambulatory specialties used these to a greater extent than other specialties. Conclusion Residency programs are emphasizing the competencies in their evaluation of residents. Understanding the scope of evaluation methodologies that programs use in resident assessment is important for both the profession and the public, so that together we may monitor continuing improvement in US graduate medical education. PMID:22132294

Evaluating Evaluations: The Case of Parent Involvement Programs

ERIC Educational Resources Information Center

Mattingly, Doreen J.; Prislin, Radmila; McKenzie, Thomas L.; Rodriguez, James L.; Kayzar, Brenda

2002-01-01

This article analyzes 41 studies that evaluated K-12 parent involvement programs in order to assess claims that such programs are an effective means of improving student learning. It examines the characteristics of the parent involvement programs, as well as the research design, data, and analytical techniques used in program evaluation. Our…

Evaluating Educational Programs.

ERIC Educational Resources Information Center

Ball, Samuel

The activities of Educational Testing Service (ETS) in evaluating educational programs are described. Program evaluations are categorized as needs assessment, formative evaluation, or summative evaluation. Three classic efforts which illustrate the range of ETS' participation are the Pennsylvania Goals Study (1965), the Coleman Report--Equality of…

INTEGRATION OF SPATIAL DATA: METHODS EVALUATION WITH REGARD TO DATA ISSUES AND ASSESSMENT QUESTIONS

EPA Science Inventory

EPA's Regional Vulnerability Assessment (REVA) Program is developing and demonstrating approaches to assess current and future environmental vulnerabilities at a regional scale. An initial effort within this research program has been to develop and evaluate methods to synthesize ...

Design and Operation of the Transformed National Healthy Start Evaluation.

PubMed

Banks, Jamelle E; Dwyer, Maura; Hirai, Ashley; Ghandour, Reem M; Atrash, Hani K

2017-12-01

Purpose Improving pregnancy outcomes for women and children is one of the nation's top priorities. The Healthy Start (HS) program was created to address factors that contribute to high infant mortality rates (IMRs) and persistent disparities in IMRs. The program began in 1991 and was transformed in 2014 to apply lessons from emerging research, past evaluation findings, and expert recommendations. To understand the implementation and impact of the transformed program, there is a need for a robust and comprehensive evaluation. Description The national HS evaluation will include an implementation evaluation, which will describe program components that affect outcomes; a utilization evaluation, which will examine the characteristics of women and infants who did and did not utilize the program; and an outcome evaluation, which will assess the program's effectiveness with regard to producing expected outcomes among the target population. Data sources include the National HS Program Survey, a HS participant survey, and individual-level program data linked to vital records and the Pregnancy Risk Assessment Monitoring System (PRAMS) survey. Assessment Descriptive analyses will be used to examine differences in risk profiles between participants and non-participants, as well as to calculate penetration rates for high-risk women in respective service areas. Multivariable analyses will be used to determine the impact of the program on key outcomes and will explore variation by dose, type of services received, and grantee characteristics. Conclusion Evaluation findings are expected to inform program decisions and direction, including identification of effective program components that can be spread and scaled.

Assessment and Evaluation of Primary Prevention in Spinal Cord Injury

PubMed Central

2013-01-01

Although the incidence of spinal cord injury (SCI) is low, the consequences of this disabling condition are extremely significant for the individual, family, and the community. Sequelae occur in the physical, psychosocial, sexual, and financial arenas, making global prevention of SCI crucial. Understanding how to assess and evaluate primary prevention programs is an important competency for SCI professionals. Assessing a program’s success requires measuring processes, outcomes, and impact. Effective evaluation can lead future efforts for program design while ensuring accountability for the program itself. The intended impact of primary prevention programs for SCI is to decrease the number of individuals who sustain traumatic injury; many programs have process and outcome goals as well. An understanding of the basic types of evaluation, evaluation design, and the overall process of program evaluation is essential for ensuring that these programs are efficacious. All health care professionals have the opportunity to put prevention at the forefront of their practice. With the current paucity of available data, it is important that clinicians share their program design, their successes, and their failures so that all can benefit and future injury can be prevented. PMID:23678281

Failure Is Not an Option: A Root Cause Analysis of Failed Acquisition Programs

DTIC Science & Technology

2017-12-01

Kendall (2012, 2015). ..................................................................................4 Table 3. Description and Decision Authority...Assessment and Program Evaluation DAE Defense Acquisition Executive DAMIR Defense Acquisition Management Information Retrieval DARPA Defense Advanced...and evaluation OTA other transactions authority PARCA performance assessments and root cause analyses PBD program budget decisions PDR preliminary

Advanced On-the-Job Training System: Master Test Plan

DTIC Science & Technology

1990-05-01

synonymous with program evaluation and consists of a plan to evaluate AOTS with regard to assessment of the four crit’cal issues of system compliance...acceptance, performance and suitability. Within the MTP, these critical issues are assessed at subcomponent, component, and subsystem levels. 14. SUBJECT...Master Test Plan is synonymous with program evaluation and consists of a plan to evaluate AOTS with regard to assessment of the four critical issues

Meeting the Pepsi Challenge: Preparing Evaluation Programs for Scholastic Improvement.

ERIC Educational Resources Information Center

Hopkins, Mona S.

This document provides guidelines for preparing evaluation programs for scholastic improvement. The philosophy underlying assessment and accountability is discussed, with specific reference to the positive and negative aspects of assessment. The design of a curriculum evaluation model is presented, including goal identification, data gathering,…

The Value in Evaluating and Communicating Program Impact: The Ohio BR&E Program

ERIC Educational Resources Information Center

Daivs, Gregory

2012-01-01

Assessing program impact can provide useful program evaluation data. It also provides a basis for program development, marketing, and justification. This article discusses recent impact evaluation efforts and findings of a long-time Extension program; referred to as Business Retention and Expansion (BR&E). How such information can be…

Education Model Program on Water-Energy Research: A New STEM Graduate Program from Development through Evaluation

NASA Astrophysics Data System (ADS)

McCay, D.; Fiorenza, P.; Lautz, L.

2017-12-01

More than half of Ph.D. scientists and engineers find employment in non-academic sectors. Recognizing the range of career options for graduate degree holders and the need to align graduate education with the expectations of prospective employers, the National Science Foundation (NSF) created the NSF Research Traineeship (NRT) program. To date, over 100 NRT programs have been funded. As these programs are implemented, it is important to assess their progress, successes, and challenges. This presentation describes the ongoing evaluation of one NRT program, "Education Model Program on Water-Energy Research" (or EMPOWER) at Syracuse University. Through seminars, mini-grants, professional development activities, field courses, internship opportunities, and coursework, EMPOWER's goal is to equip students with the skills needed for the range of career options in water and energy. In collaboration with an external evaluator, EMPOWER is examining the fidelity of the program to proposed goals, providing feedback to inform project improvement (formative assessment) and assessing the effectiveness of achieving program goals (summative assessment). Using a convergent parallel mixed method design, qualitative and quantitative data were collected to develop a full assessment of the first year of the program. Evaluation findings have resulted in several positive changes to the program. For example, EMPOWER students perceive themselves to have high technical skills, but the data show that the students do not believe that they have a strong professional network. Based on those findings, EMPOWER offered several professional development events focused on building one's professional network. Preliminary findings have enabled the EMPOWER leadership team to make informed decisions about the ways the program elements can be redesigned to better meet student needs, about how to the make the program more effective, and determine the program elements that may be sustained beyond the funding period. Evaluation of programs like EMPOWER provide essential information to support continual improvement of STEM graduate programs.

Learning from internships in gerontology and geriatrics: assessment and program evaluation.

PubMed

Karasik, Rona J

2009-01-01

Internships are an essential component of gerontological education. Harvesting the learning from internships, however, requires careful attention to assessing an intern's work. In addition to providing feedback to students, internship assessment can also yield data useful for academic program evaluation. Drawing on internship assessment data collected from undergraduate and graduate gerontology interns and their community preceptors over a period of seven semesters, this article explores (1) concerns regarding how to assess what interns are learning, (2) ways to provide students with additional opportunities for learning from their internships, and (3) how information from these student-learning outcomes may be used to evaluate the effectiveness of the overall academic program.

Moving Nursing Program Portfolio Assessment From Midterm to End of Program: Lessons Learned.

PubMed

Hickey, Kari; Rossetti, Jeanette; Oldenburg, Nancy; Abendroth, Maryann; Uhlken, Connie; Musker, Kathleen; Peters, Bradley; Paramore, Patricia

Portfolio assessment promotes a culture of evidence, evaluates program outcomes, and provides an opportunity to assess the acquisition of knowledge and skills that are not easily assessed by examinations and other traditional assessment methods in nursing curricula. The portfolio program of 1 Midwestern school of nursing recently moved portfolio assessment to the end of program. The process of this change including logistics, rubric development, and lessons learned is highlighted.

Title V Delinquency Prevention Program. Community Self-Evaluation Workbook.

ERIC Educational Resources Information Center

Caliber Associates, Fairfax, VA.

This workbook is designed to help communities and program administrators assess the success of their Title V delinquency prevention programs, but it may serve as an evaluation tool for other prevention efforts as well. It provides information and resource aids on program planning, conducting evaluations, tracking programs, describing activities,…

Assessment Inventories, Surveys, and Templates for Evaluating Educational Technology Regional and Local Assistance Programs. Phase IV of the Comprehensive Study of Educational Technology Programs Authorized from 1984-1992.

ERIC Educational Resources Information Center

Far West Lab. for Educational Research and Development, San Francisco, CA.

This report on Phase IV, of a four-phase study, provided for the development of evaluation templates and instruments to be used by the California Department of Education to facilitate systematic assessment of state funded educational technology programs and projects. These assessment documents comprised the major data collection sources for the…

The Discrepancy Evaluation Model: A Systematic Approach for the Evaluation of Career Planning and Placement Programs.

ERIC Educational Resources Information Center

Buttram, Joan L.; Covert, Robert W.

The Discrepancy Evaluation Model (DEM), developed in 1966 by Malcolm Provus, provides information for program assessment and program improvement. Under the DEM, evaluation is defined as the comparison of an actual performance to a desired standard. The DEM embodies five stages of evaluation based upon a program's natural development: program…

The Faculty Learning Outcome Assessment Framework

ERIC Educational Resources Information Center

Hurney, Carol A.; Brantmeier, Edward J.; Good, Megan R.; Harrison, Douglas; Meixner, Cara

2016-01-01

Assessment is a cyclical process within which educators construct outcomes, implement programs, assess constructs such as learning, evaluate results, and utilize results to craft stronger programs and services. Within educational and faculty development, assessment measures program impact on faculty, students, and/or institutional culture.…

Evaluating the Child Care Director: The Collaborative Professional Assessment Process.

ERIC Educational Resources Information Center

Freeman, Nancy K.; Brown, Mac H.

2000-01-01

Describes the Collaborative Professional Assessment Process (CPAP) to guide the evaluation of the director of early childhood programs. Examines the assumptions upon which the CPAP is based. Lists the management skills and leadership abilities of successful child care directors. Includes the Director Self-Evaluation form and a program evaluation…

Postadoption and Guardianship: An Evaluation of the Adoption Preservation, Assessment, and Linkage Program

ERIC Educational Resources Information Center

Liao, Minli; Testa, Mark

2016-01-01

Objectives: This study evaluated the effects of the Adoption Preservation, Assessment, and Linkage (APAL) postpermanency program. Method: A quasi-experimental, posttest-only design was used to estimate the program's effects on youth discharged from foster care to adoption or legal guardianship. A random sample was surveyed (female = 44.7%; African…

Multiple-Methods Needs Assessment of California 4-H Science Education Programming

ERIC Educational Resources Information Center

Worker, Steven M.; Schmitt-McQuitty, Lynn; Ambrose, Andrea; Brian, Kelley; Schoenfelder, Emily; Smith, Martin H.

2017-01-01

The California 4-H Science Leadership Team conducted a statewide assessment to evaluate the needs of county-based 4-H programs related to the key areas of the 4-H Science Initiative: program development and design, professional development, curricula, evaluation, partnerships, and fund development. The use of multiple qualitative data sources…

Evaluating the Quality of Competency Assessment in Pharmacy: A Framework for Workplace Learning.

PubMed

Shah, Shailly; McLaughlin, Jacqueline E; Eckel, Stephen F; Mangun, Jesica; Hawes, Emily

2016-01-19

Demonstration of achieved competencies is critical in the pharmacy workplace. The purpose of this study was to evaluate the quality of the competency assessment program for pharmacy residents at an academic medical center. The competency assessment program (CAP) survey is a validated, 48-item instrument that evaluates the quality of an assessment program based on 12 criteria, each measured by four questions on a scale of 0 to 100. The CAP was completed by residents ( n = 23) and preceptors ( n = 28) from the pharmacy residency program between 2010 and 2013. Results were analyzed using descriptive statistics, Cronbach's alpha, and non-parametric tests. Educational Consequences was the only quality criteria falling below the standard for "good quality." Participants that completed residency training elsewhere rated the Comparability (0.04) and Meaningfulness (0.01) of the assessment program higher than those that completed residency at the academic medical center. There were no significant differences between resident and preceptor scores. Overall, the quality of the assessment program was rated highly by residents and preceptors. The process described here provides a useful framework for understanding the quality of workplace learning assessments in pharmacy practice.

Intern evaluation strategies in family medicine residency education: what is-and is not-being done.

PubMed

Yates, Jennifer E

2013-06-01

Family medicine interns often have deficiencies that are not initially appreciated. By recognizing those growth opportunities early, programs may be able to better meet their interns' training needs. This study provides a needs assessment to ascertain what evaluation tools are being utilized by residency programs to assess their incoming interns. A questionnaire was sent to all US family medicine residency program coordinators (439 programs) via Survey Monkey© inquiring about whether intern evaluation is performed and, if so, what strategies are used. A mixed mode methodology was used: mailing with incentive, email prompts, and telephone calls. Of 439 programs, 220 (50%) responded to the survey. Most respondents (145, 66%) think intern evaluation is needed. However, only 79 (36%) programs are actually doing intern evaluations-only 14 (6.4%) extensively. Most programs are performing simulations (81, 45%) and assessing knowledge/comfort levels (79, 36%); less than one third are considering personality/learning styles, and almost no programs are evaluating skills such as typing (three, 1.4%) and math (one, 0.5%). Many programs use evaluations to guide future planning, help with early identification of challenging learners, and to match training to the residents' needs. Several programs expressed concern about how they would use the information once obtained. The majority of respondents agreed that a baseline intern evaluation is useful; few are actually doing it. This area is not well-described in the literature; residency programs could benefit from information sharing. The next step is to encourage interest in and implementation of such strategies.

The State of Evaluation in Internal Medicine Residency

PubMed Central

Holmboe, Eric; Beasley, Brent W.

2008-01-01

Background There are no nationwide data on the methods residency programs are using to assess trainee competence. The Accreditation Council for Graduate Medical Education (ACGME) has recommended tools that programs can use to evaluate their trainees. It is unknown if programs are adhering to these recommendations. Objective To describe evaluation methods used by our nation’s internal medicine residency programs and assess adherence to ACGME methodological recommendations for evaluation. Design Nationwide survey. Participants All internal medicine programs registered with the Association of Program Directors of Internal Medicine (APDIM). Measurements Descriptive statistics of programs and tools used to evaluate competence; compliance with ACGME recommended evaluative methods. Results The response rate was 70%. Programs were using an average of 4.2–6.0 tools to evaluate their trainees with heavy reliance on rating forms. Direct observation and practice and data-based tools were used much less frequently. Most programs were using at least 1 of the Accreditation Council for Graduate Medical Education (ACGME)’s “most desirable” methods of evaluation for all 6 measures of trainee competence. These programs had higher support staff to resident ratios than programs using less desirable evaluative methods. Conclusions Residency programs are using a large number and variety of tools for evaluating the competence of their trainees. Most are complying with ACGME recommended methods of evaluation especially if the support staff to resident ratio is high. PMID:18612734

Evaluation Strategies in Financial Education: Evaluation with Imperfect Instruments

ERIC Educational Resources Information Center

Robinson, Lauren; Dudensing, Rebekka; Granovsky, Nancy L.

2016-01-01

Program evaluation often suffers due to time constraints, imperfect instruments, incomplete data, and the need to report standardized metrics. This article about the evaluation process for the Wi$eUp financial education program showcases the difficulties inherent in evaluation and suggests best practices for assessing program effectiveness. We…

Program evaluation of FHWA pedestrian and bicycle safety activities.

DOT National Transportation Integrated Search

2011-03-01

"Introduction : FHWAs Office of Highway Safety (HSA) initiated a program evaluation by Booz Allen Hamilton to assess the overall effectiveness of the Agencys Pedestrian and Bicycle Safety Program. The evaluation covers pedestrian and bicycle sa...

Program Theory Evaluation: Logic Analysis

ERIC Educational Resources Information Center

Brousselle, Astrid; Champagne, Francois

2011-01-01

Program theory evaluation, which has grown in use over the past 10 years, assesses whether a program is designed in such a way that it can achieve its intended outcomes. This article describes a particular type of program theory evaluation--logic analysis--that allows us to test the plausibility of a program's theory using scientific knowledge.…

Critical evaluation of international health programs: Reframing global health and evaluation.

PubMed

Chi, Chunhuei; Tuepker, Anaïs; Schoon, Rebecca; Núñez Mondaca, Alicia

2018-04-01

Striking changes in the funding and implementation of international health programs in recent decades have stimulated debate about the role of communities in deciding which health programs to implement. An important yet neglected piece of that discussion is the need to change norms in program evaluation so that analysis of community ownership, beyond various degrees of "participation," is seen as central to strong evaluation practices. This article challenges mainstream evaluation practices and proposes a framework of Critical Evaluation with 3 levels: upstream evaluation assessing the "who" and "how" of programming decisions; midstream evaluation focusing on the "who" and "how" of selecting program objectives; and downstream evaluation, the focus of current mainstream evaluation, which assesses whether the program achieved its stated objectives. A vital tenet of our framework is that a community possesses the right to determine the path of its health development. A prerequisite of success, regardless of technical outcomes, is that programs must address communities' high priority concerns. Current participatory methods still seldom practice community ownership of program selection because they are vulnerable to funding agencies' predetermined priorities. In addition to critiquing evaluation practices and proposing an alternative framework, we acknowledge likely challenges and propose directions for future research. Copyright © 2018 John Wiley & Sons, Ltd.

A Model for Evaluating Programs for the Gifted under Non-Experimental Conditions.

ERIC Educational Resources Information Center

Carter, Kyle R.

1992-01-01

The article presents and illustrates use of an evaluation model for assessing programs for the gifted where tight experimental control is not possible. The model consists of four components: ex post factor designs including intact groups; comparative evaluation; strength of treatment; and multiple outcome assessment from flexible data sources. (DB)

Longitudinal Assessment of Comprehensive School Reform Program Implementation and Outcomes: First-Year Report

ERIC Educational Resources Information Center

Tushnet, Naida C., Flaherty, John, Jr., Smith, And

2004-01-01

The Longitudinal Assessment of Comprehensive School Reform Implementation and Outcomes (LACIO) responds to the No Child Left Behind Act's requirement for an evaluation of the federal Comprehensive School Reform (CSR) program. The legislation stipulates two broad goals for the evaluation: (1) to evaluate the implementation and outcomes achieved by…

Evaluability Assessment Thesis and Dissertation Studies in Graduate Professional Degree Programs

ERIC Educational Resources Information Center

Walser, Tamara M.; Trevisan, Michael S.

2016-01-01

Evaluability assessment (EA) has potential as a design option for thesis and dissertation studies, serving as a practical training experience for both technical and nontechnical evaluation skills. Based on a content review of a sample of EA theses and dissertations from graduate professional degree programs, the authors of this article found that…

Design and Implementation of Performance Metrics for Evaluation of Assessments Data

ERIC Educational Resources Information Center

Ahmed, Irfan; Bhatti, Arif

2016-01-01

Evocative evaluation of assessment data is essential to quantify the achievements at course and program levels. The objective of this paper is to design performance metrics and respective formulas to quantitatively evaluate the achievement of set objectives and expected outcomes at the course levels for program accreditation. Even though…

A novel resident-as-teacher training program to improve and evaluate obstetrics and gynecology resident teaching skills.

PubMed

Ricciotti, Hope A; Dodge, Laura E; Head, Julia; Atkins, K Meredith; Hacker, Michele R

2012-01-01

Residents play a significant role in teaching, but formal training, feedback, and evaluation are needed. Our aims were to assess resident teaching skills in the resident-as-teacher program, quantify correlations of faculty evaluations with resident self-evaluations, compare resident-as-teacher evaluations with clinical evaluations, and evaluate the resident-as-teacher program. The resident-as-teacher training program is a simulated, videotaped teaching encounter with a trained medical student and standardized teaching evaluation tool. Evaluations from the resident-as-teacher training program were compared to evaluations of resident teaching done by faculty, residents, and medical students from the clinical setting. Faculty evaluation of resident teaching skills in the resident-as-teacher program showed a mean total score of 4.5 ± 0.5 with statistically significant correlations between faculty assessment and resident self-evaluations (r = 0.47; p < 0.001). However, resident self-evaluation of teaching skill was lower than faculty evaluation (mean difference: 0.4; 95% CI 0.3-0.6). When compared to the clinical setting, resident-as-teacher evaluations were significantly correlated with faculty and resident evaluations, but not medical student evaluations. Evaluations from both the resident-as-teacher program and the clinical setting improved with duration of residency. The resident-as-teacher program provides a method to train, give feedback, and evaluate resident teaching.

Measuring Program Quality, Part 2: Addressing Potential Cultural Bias in a Rater Reliability Exam

ERIC Educational Resources Information Center

Richer, Amanda; Charmaraman, Linda; Ceder, Ineke

2018-01-01

Like instruments used in afterschool programs to assess children's social and emotional growth or to evaluate staff members' performance, instruments used to evaluate program quality should be free from bias. Practitioners and researchers alike want to know that assessment instruments, whatever their type or intent, treat all people fairly and do…

Creating an Information Literacy Badges Program in Blackboard: A Formative Program Evaluation

ERIC Educational Resources Information Center

Tunon, Johanna; Ramirez, Laura Lucio; Ryckman, Brian; Campbell, Loy; Mlinar, Courtney

2015-01-01

A formative program evaluation using Stufflebeam's (2010) Context, Input, Process, Product (CIPP) model was conducted to assess the use of digital badges for tracking basic library instructional skills across academic programs at Nova Southeastern University. Based on the evaluation of pilot library modules and Blackboard Learn's badges…

ORD Human Health Risk Assessment (HHRA) Research Program Mid-Cycle Progress Report and Review - 2010

EPA Pesticide Factsheets

The principal charge to the BOSC reviewers was to evaluate ORD’s HHRA Program from a program assessment framework relative to program relevance, structure, performance, quality, leadership, communication, and outcomes.

Faculty performance evaluation in accredited U.S. public health graduate schools and programs: a national study.

PubMed

Gimbel, Ronald W; Cruess, David F; Schor, Kenneth; Hooper, Tomoko I; Barbour, Galen L

2008-10-01

To provide baseline data on evaluation of faculty performance in U.S. schools and programs of public health. The authors administered an anonymous Internet-based questionnaire using PHP Surveyor. The invited sample consisted of individuals listed in the Council on Education for Public Health (CEPH) Directory of Accredited Schools and Programs of Public Health. The authors explored performance measures in teaching, research, and service, and assessed how faculty performance measures are used. A total of 64 individuals (60.4%) responded to the survey, with 26 (40.6%) reporting accreditation/reaccreditation by CEPH within the preceding 24 months. Although all schools and programs employ faculty performance evaluations, a significant difference exists between schools and programs in the use of results for merit pay increases and mentoring purposes. Thirty-one (48.4%) of the organizations published minimum performance expectations. Fifty-nine (92.2%) of the respondents counted number of publications, but only 22 (34.4%) formally evaluated their quality. Sixty-two (96.9%) evaluated teaching through student course evaluations, and only 29 (45.3%) engaged in peer assessment. Although aggregate results of teaching evaluation are available to faculty and administrators, this information is often unavailable to students and the public. Most schools and programs documented faculty service activities qualitatively but neither assessed it quantitatively nor evaluated its impact. This study provides insight into how schools and programs of public health evaluate faculty performance. Results suggest that although schools and programs do evaluate faculty performance on a basic level, many do not devote substantial attention to this process.

Evaluating programs that address ideological issues: ethical and practical considerations for practitioners and evaluators.

PubMed

Lieberman, Lisa D; Fagen, Michael C; Neiger, Brad L

2014-03-01

There are important practical and ethical considerations for organizations in conducting their own, or commissioning external, evaluations and for both practitioners and evaluators, when assessing programs built on strongly held ideological or philosophical approaches. Assessing whether programs "work" has strong political, financial, and/or moral implications, particularly when expending public dollars, and may challenge objectivity about a particular program or approach. Using a case study of the evaluation of a school-based abstinence-until-marriage program, this article discusses the challenges, lessons learned, and ethical responsibilities regarding decisions about evaluation, specifically associated with ideologically driven programs. Organizations should consider various stakeholders and views associated with their program to help identify potential pitfalls in evaluation. Once identified, the program or agency needs to carefully consider its answers to two key questions: Do they want the answer and are they willing to modify the program? Having decided to evaluate, the choice of evaluator is critical to assuring that ethical principles are maintained and potential skepticism or criticism of findings can be addressed appropriately. The relationship between program and evaluator, including agreements about ownership and eventual publication and/or promotion of data, should be addressed at the outset. Programs and organizations should consider, at the outset, their ethical responsibility when findings are not expected or desired. Ultimately, agencies, organizations, and programs have an ethical responsibility to use their data to provide health promotion programs, whether ideologically founded or not, that appropriately and effectively address the problems they seek to solve.

Family Assessment/Treatment/Evaluation Methods Integrated for Helping Teen Suicide Attempters/Families in Short Term Psychiatric Hospitalization Programs.

ERIC Educational Resources Information Center

Shepard, Suzanne

The assessment process can be integrated with treatment and evaluation for helping teenage suicide attempters and families in short term psychiatric hospitalization programs. The method is an extremely efficient way for the therapist to work within a given time constraint. During family assessment sufficient information can be gathered to…

Toward a Capacity Framework for Useful Student Learning Outcomes Assessment in College Foreign Language Programs

ERIC Educational Resources Information Center

Davis, John McEwan

2016-01-01

The educational impacts of mandated assessment in U.S. colleges is part of a growing research agenda focused on how methodologies of program evaluation best enable educators to improve teaching and learning. Accordingly, research has tried to identify the key aspects of evaluation/assessment "capacity" in college language departments…

Core Professionalism Education in Surgery: A Systematic Review.

PubMed

Sarıoğlu Büke, Akile; Karabilgin Öztürkçü, Özlem Sürel; Yılmaz, Yusuf; Sayek, İskender

2018-03-15

Professionalism education is one of the major elements of surgical residency education. To evaluate the studies on core professionalism education programs in surgical professionalism education. Systematic review. This systematic literature review was performed to analyze core professionalism programs for surgical residency education published in English with at least three of the following features: program developmental model/instructional design method, aims and competencies, methods of teaching, methods of assessment, and program evaluation model or method. A total of 27083 articles were retrieved using EBSCOHOST, PubMed, Science Direct, Web of Science, and manual search. Eight articles met the selection criteria. The instructional design method was presented in only one article, which described the Analysis, Design, Development, Implementation, and Evaluation model. Six articles were based on the Accreditation Council for Graduate Medical Education criterion, although there was significant variability in content. The most common teaching method was role modeling with scenario- and case-based learning. A wide range of assessment methods for evaluating professionalism education were reported. The Kirkpatrick model was reported in one article as a method for program evaluation. It is suggested that for a core surgical professionalism education program, developmental/instructional design model, aims and competencies, content, teaching methods, assessment methods, and program evaluation methods/models should be well defined, and the content should be comparable.

National Weatherization Assistance Program Evaluation: Assessment of Refrigerator Energy Use

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tonn, Bruce Edward; Goeltz, Rick

2015-03-01

This report assesses the energy consumption characteristics and performance of refrigerators that were monintored as a component of the Indoor Air Quality Study that itself was a component of the retrospective evaluation of the Department of Energy's Weatherization Assistance Program.

Iterative evaluation in a mobile counseling and testing program to reach people of color at risk for HIV--new strategies improve program acceptability, effectiveness, and evaluation capabilities.

PubMed

Spielberg, Freya; Kurth, Ann; Reidy, William; McKnight, Teka; Dikobe, Wame; Wilson, Charles

2011-06-01

This article highlights findings from an evaluation that explored the impact of mobile versus clinic-based testing, rapid versus central-lab based testing, incentives for testing, and the use of a computer counseling program to guide counseling and automate evaluation in a mobile program reaching people of color at risk for HIV. The program's results show that an increased focus on mobile outreach using rapid testing, incentives and health information technology tools may improve program acceptability, quality, productivity and timeliness of reports. This article describes program design decisions based on continuous quality assessment efforts. It also examines the impact of the Computer Assessment and Risk Reduction Education computer tool on HIV testing rates, staff perception of counseling quality, program productivity, and on the timeliness of evaluation reports. The article concludes with a discussion of implications for programmatic responses to the Centers for Disease Control and Prevention's HIV testing recommendations.

Evaluation Processes Used to Assess the Effectiveness of Vocational-Technical Programs.

ERIC Educational Resources Information Center

Bruhns, Arthur E.

Evaluation is quantitative or qualitative, the criteria determined by or given to the student. The criteria show how close he has come to the program's objectives and the ranking of individual performance. Vocational education programs susceptible to evaluation are listed and relevant evaluative techniques discussed. Graduate interviews concerning…

Evaluating Educational Programs. ETS R&D Scientific and Policy Contributions Series. ETS SPC-11-01. ETS Research Report No. RR-11-15

ERIC Educational Resources Information Center

Ball, Samuel

2011-01-01

Since its founding in 1947, ETS has conducted a significant and wide-ranging research program that has focused on, among other things, psychometric and statistical methodology; educational evaluation; performance assessment and scoring; large-scale assessment and evaluation; cognitive, developmental, personality, and social psychology; and…

Pilot Evaluation of an In-Store Nutrition Label Education Program.

PubMed

Dukeshire, Steven; Nicks, Emily; Ferguson, Jennifer

2014-12-01

To describe and provide recommendations for the implementation of an evaluation for an already existing, in-store Nutrition Label Education Program (NLEP). We describe the development and implementation of an evaluation consisting of a pre- and postsurvey and one month follow-up. The evaluation was designed to assess satisfaction with the NLEP as well as changes in participant nutrition label knowledge, confidence in using nutrition labels, and actual changes in nutrition label use. Nineteen participants took part in the pilot evaluation. The evaluation was successful in demonstrating high levels of satisfaction with the NLEP as well as positive changes in participant confidence and some increased knowledge in using nutrition labels. However, only 3 people participated in the follow-up, limiting the ability to assess behaviour change. Ideally, NLEPs should include ongoing evaluation that extends beyond just assessing participant satisfaction. Recommendations are provided for conducting such evaluations, including the importance of incorporating the evaluation into the program itself, using existing questionnaires when possible, and employing pre- and postsurveys as well as follow-up interviews to assess change.

Evaluation of the Brownfields Program

EPA Pesticide Factsheets

The Evaluation of 2003-2008 Brownfields Assessment, Revolving Loan Fund, and Cleanup Grants is the first national program evaluation of the outcomes, efficiencies, and economic benefits produced by Brownfields grants.

Improving Accreditor's Evaluation of Experiential Learning Programs.

ERIC Educational Resources Information Center

Keeton, Morris T.

1980-01-01

Principles of good practice in assessing experiential learning include better self-evaluation of the learning outcomes of experiential components, systematic program auditing, and training of external evaluators. (SK)

Standardized patients in audiology: a proposal for a new method of evaluating clinical competence.

PubMed

Dinsmore, Brooke Freeman; Bohnert, Carrie; Preminger, Jill E

2013-05-01

While accrediting organizations require AuD programs to provide evidence that their students are able to demonstrate knowledge and competencies in specific content areas, there are no generally accepted mechanisms for the assessment and the measurement of these proficiencies. We propose that AuD programs consider developing standardized patient (SP) cases in order to develop consistent summative assessment programs within and across universities. The purpose of this article is to provide a framework for establishing SP programs to evaluate competencies in AuD students by detailing the history of SP cases and their use, developing a rationale for this method of assessment, and outlining the steps for writing and implementing SP cases. Literature review. SPs have been used to assess clinical competence in medical students for over 50 yr. The prevalence of SP assessment in allied health professions (e.g., dentistry, psychology, pharmacy) has increased over the last two decades but has only gained a limited following in audiology. SP assessment has been implemented in medical education using the Objective Structured Clinical Examination, a multistation, timed exam that uses fictional cases to assess students' clinical abilities. To date, only one published report has been completed that evaluates the use of SPs to assess clinical abilities in audiology students. This article expands upon the work of English et al (2007) and their efforts to use SPs to evaluate counseling abilities. To this end, we describe the steps necessary to write a case, procedures to determine performance requirements, and the need to develop remediation plans. As an example, we include a case that we have developed in order to evaluate vestibular assessment and patient communication skills. Utilizing SP assessment in audiology education would provide useful means to evaluate competence in a uniform way. Future research is necessary to develop reliable and valid cases that may be implemented across programs. This article aims to serve as a call to audiology programs to begin developing and reporting these cases. Once these are established we can begin to use SP cases for summative assessment. American Academy of Audiology.

Environment, Safety and Health Self-Assessment Report Fiscal Year 2010

DOE Office of Scientific and Technical Information (OSTI.GOV)

Robinson, Scott

2011-03-23

The Lawrence Berkeley National Laboratory (LBNL) Environment, Safety, and Health (ES&H) Self-Assessment Program was established to ensure that Integrated Safety Management (ISM) is implemented institutionally and by all divisions. The ES&H Self-Assessment Program, managed by the Office of Contractor Assurance (OCA), provides for an internal evaluation of all ES&H programs and systems at LBNL. The primary objective of the program is to ensure that work is conducted safely and with minimal negative impact to workers, the public, and the environment. Self-assessment follows the five core functions and guiding principles of ISM. Self-assessment is the mechanism used to promote the continuousmore » improvement of the Laboratory's ES&H programs. The process is described in the Environment, Safety, and Health Assurance Plan (PUB-5344) and is composed of three types of self-assessments: Division ES&H Self-Assessment, ES&H Technical Assurance Program Assessment, and Division ES&H Peer Review. The Division ES&H Self-Assessment Manual (PUB-3105) provides the framework by which divisions conduct formal ES&H self-assessments to systematically identify program deficiencies. Issue-specific assessments are designed and implemented by the divisions and focus on areas of interest to division management. They may be conducted by teams and involve advance planning to ensure that appropriate resources are available. The ES&H Technical Assurance Program Manual (PUB-913E) provides the framework for systematic reviews of ES&H programs and processes. The ES&H Technical Assurance Program Assessment is designed to evaluate whether ES&H programs and processes are compliant with guiding regulations, are effective, and are properly implemented by LBNL divisions. The Division ES&H Peer Review Manual provides the framework by which division ISM systems are evaluated and improved. Peer Reviews are conducted by teams under the direction of senior division management and focus on higher-level management issues. Peer Review teams are selected on the basis of members knowledge and experience in the issues of interest to the division director. LBNL periodically requests in-depth independent assessments of selected ES&H programs. Such assessments augment LBNL's established assessment processes and provide an objective view of ES&H program effectiveness. Institutional Findings, Observations, and Noteworthy Practices identified during independent assessments are specifically intended to help LBNL identify opportunities for program improvement. This report includes the results of the Division ES&H Self-Assessment, ES&H Technical Assurance Program Assessment, and Division ES&H Peer Review, respectively.« less

Program Evaluation in Gifted Education. Essential Readings in Gifted Education Series

ERIC Educational Resources Information Center

Callahan, Carolyn M., Ed.; Reis, Sally M., Ed.

2004-01-01

The readings in this ready-reference report on specific program evaluations, offer critical guidance in the development and utilization of instruments for assessing gifted and talented programs, and are designed to stimulate the discussion of issues surrounding the evaluation of gifted programs. Key features include: (1) Carolyn M. Callahan's…

A Program Evaluation Tool for Dual Enrollment Transition Programs

ERIC Educational Resources Information Center

Grigal, Meg; Dwyre, Amy; Emmett, Joyce; Emmett, Richard

2012-01-01

This article describes the development and use of a program evaluation tool designed to support self-assessment of college-based dual enrollment transition programs serving students with intellectual disabilities between the ages of 18-21 in college settings. The authors describe the need for such an evaluation tool, outline the areas addressed by…

EVALUATION OF THE WEIGHT-BASED COLLECTION PROJECT IN FARMINGTON, MINNESOTA: A MITE PROGRAM EVALUATION

EPA Science Inventory

This project evaluates a test program of a totally automated weight-based refuse disposal rate system. his test program was conducted by the City of Farmington, Minnesota between 1991 and 1993. he intent of the program was to test a mechanism which would automatically assess a fe...

Some Measures of Evaluation and Effectiveness in Social Work Practice.

ERIC Educational Resources Information Center

Kapoor, J. M.

Measures of accountability and evaluation of social work program efforts are examined. Evaluation of program effort refers to an assessment of the amount and kinds of program activities considered necessary for the accomplishment of program goals within a particular stage of development. It refers not only to staff time, activity, and commitment,…

Formative Evaluation of the Canada Student Loans Program. Final Report

ERIC Educational Resources Information Center

Human Resources and Skills Development Canada, 2004

2004-01-01

The "Formative Evaluation of the Canada Student Loans Program" was undertaken to assess issues of program relevance, design and delivery and for the purposes of examining the early impacts of changes made to the program since 1998. The evaluation also reviewed the Performance Measurement Strategy contained in the July 2002 Results-Based…

Curriculum-Based Vocational Assessment of Students with Special Needs at the Middle School/Junior High School Levels: The Practical Arts Evaluation System (PAES).

ERIC Educational Resources Information Center

Swisher, Judy; Clark, Gary M.

1991-01-01

Describes the Practical Arts Evaluation System (PAES), a curriculum-based vocational assessment program for students with special needs at the middle school/junior high school level. Provides a rationale for curriculum-based assessment and occupational exploration at this level and describes benefits of the program. (Author/JOW)

Defense Logistics Agency Disposition Services Needs to Improve Demilitarization Program Self-Assessment Evaluations - Redacted

DTIC Science & Technology

2016-12-20

Program Self -Assessment Evaluations I N T E G R I T Y  E F F I C I E N C Y  A C C O U N T A B I L I T Y  E X C E L L E N C E Mission Our mission...Project No. D2016-D000RD-0057.000) │ i Results in Brief Defense Logistics Agency Disposition Services Needs to Improve Demilitarization Program Self ...mutilation contractor3 in Holbrook, New York, the two sites we visited. However, for self -assessment evaluations4 at three DLA Centralized

Planning an integrated agriculture and health program and designing its evaluation: Experience from Western Kenya.

PubMed

Cole, Donald C; Levin, Carol; Loechl, Cornelia; Thiele, Graham; Grant, Frederick; Girard, Aimee Webb; Sindi, Kirimi; Low, Jan

2016-06-01

Multi-sectoral programs that involve stakeholders in agriculture, nutrition and health care are essential for responding to nutrition problems such as vitamin A deficiency among pregnant and lactating women and their infants in many poor areas of lower income countries. Yet planning such multi-sectoral programs and designing appropriate evaluations, to respond to different disciplinary cultures of evidence, remain a challenge. We describe the context, program development process, and evaluation design of the Mama SASHA project (Sweetpotato Action for Security and Health in Africa) which promoted production and consumption of a bio-fortified, orange-fleshed sweetpotato (OFSP). In planning the program we drew upon information from needs assessments, stakeholder consultations, and a first round of the implementation evaluation of a pilot project. The multi-disciplinary team worked with partner organizations to develop a program theory of change and an impact pathway which identified aspects of the program that would be monitored and established evaluation methods. Responding to the growing demand for greater rigour in impact evaluations, we carried out quasi-experimental allocation by health facility catchment area, repeat village surveys for assessment of change in intervention and control areas, and longitudinal tracking of individual mother-child pairs. Mid-course corrections in program implementation were informed by program monitoring, regular feedback from implementers and partners' meetings. To assess economic efficiency and provide evidence for scaling we collected data on resources used and project expenses. Managing the multi-sectoral program and the mixed methods evaluation involved bargaining and trade-offs that were deemed essential to respond to the array of stakeholders, program funders and disciplines involved. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Planning an integrated agriculture and health program and designing its evaluation: Experience from Western Kenya

PubMed Central

Cole, Donald C.; Levin, Carol; Loechl, Cornelia; Thiele, Graham; Grant, Frederick; Girard, Aimee Webb; Sindi, Kirimi; Low, Jan

2016-01-01

Multi-sectoral programs that involve stakeholders in agriculture, nutrition and health care are essential for responding to nutrition problems such as vitamin A deficiency among pregnant and lactating women and their infants in many poor areas of lower income countries. Yet planning such multi-sectoral programs and designing appropriate evaluations, to respond to different disciplinary cultures of evidence, remain a challenge. We describe the context, program development process, and evaluation design of the Mama SASHA project (Sweetpotato Action for Security and Health in Africa) which promoted production and consumption of a bio-fortified, orange-fleshed sweetpotato (OFSP). In planning the program we drew upon information from needs assessments, stakeholder consultations, and a first round of the implementation evaluation of a pilot project. The multi-disciplinary team worked with partner organizations to develop a program theory of change and an impact pathway which identified aspects of the program that would be monitored and established evaluation methods. Responding to the growing demand for greater rigour in impact evaluations, we carried out quasi-experimental allocation by health facility catchment area, repeat village surveys for assessment of change in intervention and control areas, and longitudinal tracking of individual mother-child pairs. Mid-course corrections in program implementation were informed by program monitoring, regular feedback from implementers and partners’ meetings. To assess economic efficiency and provide evidence for scaling we collected data on resources used and project expenses. Managing the multi-sectoral program and the mixed methods evaluation involved bargaining and trade-offs that were deemed essential to respond to the array of stakeholders, program funders and disciplines involved. PMID:27003730

Program Evaluation at Foothill College.

ERIC Educational Resources Information Center

Lowe, Irel D.

Evaluation systems were designed for the assessment of the instructional and non-instructional programs at Foothill College (California) to determine whether the objectives of the programs were being met and to ascertain the direct and indirect costs of the programs. For the instructional programs, criteria for determining effectiveness, revenue…

The Do's and Don'ts in Regard to the Evaluation of Bilingual Programs.

ERIC Educational Resources Information Center

Rodriquez-Brown, Flora V.

The difficulties in evaluating bilingual education appear to have prevented success in all but a few evaluation attempts, but better and more meaningful evaluation is neccessary in order to identify the strengths and weaknesses of bilingual programs. Many bilingual programs are undergoing constant modification, adequate assessment instruments have…

Quantitative assessment of participant knowledge and evaluation of participant satisfaction in the CARES training program.

PubMed

Goodman, Melody S; Si, Xuemei; Stafford, Jewel D; Obasohan, Adesuwa; Mchunguzi, Cheryl

2012-01-01

The purpose of the Community Alliance for Research Empowering Social change (CARES) training program was to (1) train community members on evidence-based public health, (2) increase their scientific literacy, and (3) develop the infrastructure for community-based participatory research (CBPR). We assessed participant knowledge and evaluated participant satisfaction of the CARES training program to identify learning needs, obtain valuable feedback about the training, and ensure learning objectives were met through mutually beneficial CBPR approaches. A baseline assessment was administered before the first training session and a follow-up assessment and evaluation was administered after the final training session. At each training session a pretest was administered before the session and a posttest and evaluation were administered at the end of the session. After training session six, a mid-training evaluation was administered. We analyze results from quantitative questions on the assessments, pre- and post-tests, and evaluations. CARES fellows knowledge increased at follow-up (75% of questions were answered correctly on average) compared with baseline (38% of questions were answered correctly on average) assessment; post-test scores were higher than pre-test scores in 9 out of 11 sessions. Fellows enjoyed the training and rated all sessions well on the evaluations. The CARES fellows training program was successful in participant satisfaction and increasing community knowledge of public health, CBPR, and research methodology. Engaging and training community members in evidence-based public health research can develop an infrastructure for community-academic research partnerships.

Are You Making an Impact? Evaluating the Population Health Impact of Community Benefit Programs.

PubMed

Rains, Catherine M; Todd, Greta; Kozma, Nicole; Goodman, Melody S

The Patient Protection and Affordable Care Act includes a change to the IRS 990 Schedule H, requiring nonprofit hospitals to submit a community health needs assessment every 3 years. Such health care entities are challenged to evaluate the effectiveness of community benefit programs addressing the health needs identified. In an effort to determine the population health impact of community benefit programs in 1 hospital outreach department, researchers and staff conducted an impact evaluation to develop priority areas and overarching goals along with program- and department-level objectives. The longitudinal impact evaluation study design consists of retrospective and prospective secondary data analyses. As an urban pediatric hospital, St Louis Children's Hospital provides an array of community benefit programs to the surrounding community. Hospital staff and researchers came together to form an evaluation team. Data from program evaluation and administrative data for analysis were provided by hospital staff. Impact scores were calculated by scoring objectives as met or unmet and averaged across goals to create impact scores that measure how closely programs meet the overarching departmental mission and goals. Over the 4-year period, there is an increasing trend in program-specific impact scores across all programs except one, Healthy Kids Express Asthma, which had a slight decrease in year 4 only. Current work in measuring and assessing the population health impact of community benefit programs is mostly focused on quantifying dollars invested into community benefit work rather than measuring the quality and impact of services. This article provides a methodology for measuring population health impact of community benefit programs that can be used to evaluate the effort of hospitals in providing community benefit. This is particularly relevant in our changing health care climate, as hospitals are being asked to justify community benefit and make meaningful contributions to population health. The Patient Protection and Affordable Care Act includes a change to the IRS 990 Schedule H, requiring nonprofit hospitals to submit a community health needs assessment every 3 years, and requires evaluation of program effectiveness; yet, it does not require any quantification of the impact of community benefit programs. The IRS Schedule H 990 policies could be strengthened by requiring an impact evaluation such as outlined in this article. As hospitals are being asked to justify community benefit and make meaningful contributions to population health, impact evaluations can be utilized to demonstrate the cumulative community benefit of programs and assess population health impact of community benefit programs.

Remake/Remodel: Using ePortfolios and a System of Gates to Improve Student Assessment and Program Evaluation

ERIC Educational Resources Information Center

Lowenthal, Patrick; White, John W.; Cooley, Karen

2011-01-01

Electronic portfolios have become increasingly popular. The value of a portfolio, though, depends on how, when, and why students create, submit, and have their portfolios evaluated. In the following paper, we describe how we redesigned a program's assessment and evaluation plan around the use of electronic portfolios and a system of gates focusing…

Surgeons' attitude toward a competency-based training and assessment program: results of a multicenter survey.

PubMed

Hopmans, Cornelis J; den Hoed, Pieter T; Wallenburg, Iris; van der Laan, Lijkckle; van der Harst, Erwin; van der Elst, Maarten; Mannaerts, Guido H H; Dawson, Imro; van Lanschot, Jan J B; Ijzermans, Jan N M

2013-01-01

Currently, most surgical training programs are focused on the development and evaluation of professional competencies. Also in the Netherlands, competency-based training and assessment programs were introduced to restructure postgraduate medical training. The current surgical residency program is based on the Canadian Medical Education Directives for Specialists (CanMEDS) competencies and uses assessment tools to evaluate residents' competence progression. In this study, we examined the attitude of surgical residents and attending surgeons toward a competency-based training and assessment program used to restructure general surgical training in the Netherlands in 2009. In 2011, all residents (n = 51) and attending surgeons (n = 108) in 1 training region, consisting of 7 hospitals, were surveyed. Participants were asked to rate the importance of the CanMEDS competencies and the suitability of the adopted assessment tools. Items were rated on a 5-point Likert scale and considered relevant when at least 80% of the respondents rated an item with a score of 4 or 5 (indicating a positive attitude). Reliability was evaluated by calculating the Cronbach's α, and the Mann-Whitney test was applied to assess differences between groups. The response rate was 88% (n = 140). The CanMEDS framework demonstrated good reliability (Cronbach's α = 0.87). However, the importance of the competencies 'Manager' (78%) and 'Health Advocate' (70%) was undervalued. The assessment tools failed to achieve an acceptable reliability (Cronbach's α = 0.55), and individual tools were predominantly considered unsuitable for assessment. Exceptions were the tools 'in-training evaluation report' (91%) and 'objective structured assessment of technical skill' (82%). No significant differences were found between the residents and the attending surgeons. This study has demonstrated that, 2 years after the reform of the general surgical residency program, residents and attending surgeons in a large training region in the Netherlands do not acknowledge the importance of all CanMEDS competencies and consider the assessment tools generally unsuitable for competence evaluation. Copyright © 2013 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

ITERATIVE EVALUATION IN A MOBILE COUNSELING AND TESTING PROGRAM TO REACH PEOPLE OF COLOR AT RISK FOR HIV—NEW STRATEGIES IMPROVE PROGRAM ACCEPTABILITY, EFFECTIVENESS, AND EVALUATION CAPABILITIES

PubMed Central

Spielberg, Freya; Kurth, Ann; Reidy, William; McKnight, Teka; Dikobe, Wame; Wilson, Charles

2016-01-01

This article highlights findings from an evaluation that explored the impact of mobile versus clinic-based testing, rapid versus central-lab based testing, incentives for testing, and the use of a computer counseling program to guide counseling and automate evaluation in a mobile program reaching people of color at risk for HIV. The program’s results show that an increased focus on mobile outreach using rapid testing, incentives and health information technology tools may improve program acceptability, quality, productivity and timeliness of reports. This article describes program design decisions based on continuous quality assessment efforts. It also examines the impact of the Computer Assessment and Risk Reduction Education computer tool on HIV testing rates, staff perception of counseling quality, program productivity, and on the timeliness of evaluation reports. The article concludes with a discussion of implications for programmatic responses to the Centers for Disease Control and Prevention’s HIV testing recommendations. PMID:21689041

Using Corporate-Based Methods To Assess Technical Communication Programs.

ERIC Educational Resources Information Center

Faber, Brenton; Bekins, Linn; Karis, Bill

2002-01-01

Investigates methods of program assessment used by corporate learning sites and profiles value added methods as a way to both construct and evaluate academic programs in technical communication. Examines and critiques assessment methods from corporate training environments including methods employed by corporate universities and value added…

Counselor Competence, Performance Assessment, and Program Evaluation: Using Psychometric Instruments

ERIC Educational Resources Information Center

Tate, Kevin A.; Bloom, Margaret L.; Tassara, Marcel H.; Caperton, William

2014-01-01

Psychometric instruments have been underutilized by counselor educators in performance assessment and program evaluation efforts. As such, we conducted a review of the literature that revealed 41 instruments fit for such efforts. We described and critiqued these instruments along four dimensions--"Target Domain," "Format,"…

Evaluating adolescent pregnancy programs: rethinking our priorities.

PubMed

Stahler, G J; DuCette, J P

1991-01-01

Noting that impact evaluations of adolescent pregnancy programs are characterized by poor quality, the authors recommend using a different standard in assessing the value of programs. While the number of adolescent pregnancy programs has multiplied during the last 3 decades, little is known about their impact in ameliorating the negative consequences of too-early childbearing. An ideal evaluation of these programs would randomly select and randomly assign subjects to experimental and control groups. But evaluations conducted by individual program generally face obstacles that limit the randomness of the study. most individual programs lack the financial resources and do not employ the full-time professional evaluators needed to carry out a valid evaluation. These factors result in too short an evaluation period, incomplete and inaccurate data, and lack of randomness in the assignment of control groups. To more accurately assess the impact of the programs, the authors recommend that individual programs focus on process evaluation and collection of complete and reliable data on their clients. From the onset, a program should have a clear description of its content, logic of intervention, and method of implementation. It should maintain thorough records on client characteristics, service utilization, and should conduct long-term follow-ups. For rigorous impact evaluations, programs should rely on 3rd party entities. These independent organizations -- universities or research institutes -- do not have a stake in the outcome of the evaluation, making the study all the more objective. Furthermore, they provide experienced researchers.

An Interprofessional Program Evaluation Case Study: Utilizing Multiple Measures To Assess What Matters. AIR 1997 Annual Forum Paper.

ERIC Educational Resources Information Center

Delaney, Anne Marie

This paper reviews the first two years of a model program-evaluation case study which is intended to show: (1) how program evaluation can contribute to academic and professional degree programs; (2) how qualitative and quantitative techniques can be used to produce reliable measures for evaluation studies; and (3) how the role of the institutional…

A CAD (Classroom Assessment Design) of a Computer Programming Course

ERIC Educational Resources Information Center

Hawi, Nazir S.

2012-01-01

This paper presents a CAD (classroom assessment design) of an entry-level undergraduate computer programming course "Computer Programming I". CAD has been the product of a long experience in teaching computer programming courses including teaching "Computer Programming I" 22 times. Each semester, CAD is evaluated and modified…

NASA Aviation Safety Program Systems Analysis/Program Assessment Metrics Review

NASA Technical Reports Server (NTRS)

Louis, Garrick E.; Anderson, Katherine; Ahmad, Tisan; Bouabid, Ali; Siriwardana, Maya; Guilbaud, Patrick

2003-01-01

The goal of this project is to evaluate the metrics and processes used by NASA's Aviation Safety Program in assessing technologies that contribute to NASA's aviation safety goals. There were three objectives for reaching this goal. First, NASA's main objectives for aviation safety were documented and their consistency was checked against the main objectives of the Aviation Safety Program. Next, the metrics used for technology investment by the Program Assessment function of AvSP were evaluated. Finally, other metrics that could be used by the Program Assessment Team (PAT) were identified and evaluated. This investigation revealed that the objectives are in fact consistent across organizational levels at NASA and with the FAA. Some of the major issues discussed in this study which should be further investigated, are the removal of the Cost and Return-on-Investment metrics, the lack of the metrics to measure the balance of investment and technology, the interdependencies between some of the metric risk driver categories, and the conflict between 'fatal accident rate' and 'accident rate' in the language of the Aviation Safety goal as stated in different sources.

Competency assessment of microbiology medical laboratory technologists in Ontario, Canada.

PubMed

Desjardins, Marc; Fleming, Christine Ann

2014-08-01

Accreditation in Ontario, Canada, requires that licensed clinical laboratories participate in external quality assessment (also known as proficiency testing) and perform competency evaluation of their staff. To assess the extent of ongoing competency assessment practices, the Quality Management Program--Laboratory Services (QMP-LS) Microbiology Committee surveyed all 112 licensed Ontario microbiology laboratories. The questionnaire consisted of a total of 21 questions that included yes/no, multiple-choice, and short-answer formats. Participants were asked to provide information about existing programs, the frequency of testing, what areas are evaluated, and how results are communicated to the staff. Of the 111 responding laboratories, 6 indicated they did not have a formal evaluation program since they perform only limited bacteriology testing. Of the remaining 105 respondents, 87% perform evaluations at least annually or every 2 years, and 61% include any test or task performed, whereas 16% and 10% focus only on problem areas and high-volume complex tasks, respectively. The most common methods of evaluation were review of external quality assessment (EQA) challenges, direct observation, and worksheet review. With the exception of one participant, all communicate results to staff, and most take remedial action to correct the deficiencies. Although most accredited laboratories have a program to assess the ongoing competency of their staff, the methods used are not standardized or consistently applied, indicating that there is room for improvement. The survey successfully highlighted potential areas for improvement and allowed the QMP-LS Microbiology Committee to provide guidance to Ontario laboratories for establishing or improving existing microbiology-specific competency assessment programs. Copyright © 2014, American Society for Microbiology. All Rights Reserved.

Retaining minorities in engineering: Assessment of a program prototype

NASA Astrophysics Data System (ADS)

Good, Jennifer Marie (Phillips)

Program assessment is an essential part of healthy program development. Assessment should include multiple considerations, dimensions, and outcomes that match the program's objectives. As a newly formed retention program, the Auburn University Minority Engineering Program, designed to help pre-engineering minority students make the transition into their freshman year of university studies, incorporated evaluation and assessment into all three components of the program (the interactive learning laboratory, critical-thinking workshops, and Sunday-evening tutorials) from the program's inception. If students successfully adapted to the university environment and the demands of the pre-engineering course of study, then retention of minority students in the College of Engineering should improve. Data were gathered on the students involved in the various program components. Students who entered the Minority Engineering Program were pre- and posttested on three standardized subtests (critical thinking, mathematics, and science reasoning) of the Collegiate Assessment of Academic Proficiency. The first-quarter grade-point averages of the students were also gathered to compare their grades to freshman students in previous quarters within the College of Engineering. Qualitative data were also gathered on this same group of students. An analysis of the data revealed that student achievement is affected by involvement in the Minority Engineering Program. Specifically, the first quarter grade point averages of students involved in the program exceeded those of their peers in earlier years of study prior to the program's existence. In addition, mathematics and science reasoning scores on standardized tests increased pre- to postintervention. Comments collected in journals and files also demonstrated use of critical-thinking and problem-solving skills employed by the students. Recommendations for alterations of the program were made based on the outcome of the program evaluation. Further suggestions for research in minority engineering program development and evaluation were also discussed.

Evaluating Continuing Education Needs and Program Effectiveness Using a Survey of Virginia's SHARP Logger Program Participants

ERIC Educational Resources Information Center

Barrett, Scott M.; Bolding, M. Chad; Munsell, John F.

2012-01-01

Virginia's SHARP logger program is a Cooperative Extension program currently providing training to over 1,500 loggers, foresters, and others. We conducted a mail survey of SHARP loggers to characterize program participants, assess programming needs, and evaluate program effectiveness. Results indicate a diverse group of participants in terms of…

Using a Crystal Ball Instead of a Rear-View Mirror: Helping State Legislators Assess the Future Impacts of Major Federal Legislation

ERIC Educational Resources Information Center

Alter, Joel; Patterson, John

2006-01-01

Typically, program evaluation agencies in the legislative branch of state government examine programs that have already been implemented. These evaluations often consider whether a program achieved the legislature's original goals or complied with statutory requirements. Program evaluations frequently determine whether executive branch agencies…

An Evaluation of On-Line, Interactive Tutorials Designed to Teach Practice Concepts

ERIC Educational Resources Information Center

Seabury, Brett A.

2005-01-01

This paper presents an evaluation of two on-line-based programs designed to teach practice skills. One program teaches crisis intervention and the other teaches suicide assessment. The evaluation of the use of these programs compares outcomes for two groups of students, one using the interactive program outside a class context and the other using…

Planning, Implementation, and Evaluation of AIDS Education Programs for Dentists.

ERIC Educational Resources Information Center

Gerbert, Barbara; And Others

1991-01-01

An office-based continuing education program on acquired immune deficiency syndrome (AIDS) for dentists is described, including needs assessment, model development, local piloting, national implementation with 119 dentists, and evaluation phases. Program evaluation indicated an improvement in risk perception, knowledge, and practice resulted, but…

Outcomes Assessment in Accredited Health Information Management Programs

ERIC Educational Resources Information Center

Bennett, Dorine

2010-01-01

The purpose of this study was to determine the use and perceived usefulness of outcomes assessment methods in health information management programs. Additional characteristics of the outcomes assessment practices were recognized. The findings were evaluated for significant differences in results based on age of the program, type of institution,…

An Assessment of an Operational Educational Accountability System for Continuing Education in the Health Professions.

ERIC Educational Resources Information Center

Walsh, Patrick L.

1982-01-01

The Educational Quality Assessment and Assurance System for continuing education in health sciences includes three components to measure inputs, processes, and outcomes. They are (1) Program Coordinator Competency List, (2) Quality Assessment and Assurance Program, and (3) evaluation of educational programs. (SK)

Assessment of Effectiveness in Preprimary and School Social Work Programs.

ERIC Educational Resources Information Center

Radin, Norma

A model for evaluating effectiveness in preschool and school social work programs is proposed. The paradigm has two major dimensions: criteria for evaluating success, and modalities for assessing whether the criteria have been met. Three criteria are suggested including improved views of significant others, and seven modalities including test…

Core Professionalism Education in Surgery: A Systematic Review

PubMed Central

Sarıoğlu Büke, Akile; Karabilgin Öztürkçü, Özlem Sürel; Yılmaz, Yusuf; Sayek, İskender

2018-01-01

Background: Professionalism education is one of the major elements of surgical residency education. Aims: To evaluate the studies on core professionalism education programs in surgical professionalism education. Study Design: Systematic review. Methods: This systematic literature review was performed to analyze core professionalism programs for surgical residency education published in English with at least three of the following features: program developmental model/instructional design method, aims and competencies, methods of teaching, methods of assessment, and program evaluation model or method. A total of 27083 articles were retrieved using EBSCOHOST, PubMed, Science Direct, Web of Science, and manual search. Results: Eight articles met the selection criteria. The instructional design method was presented in only one article, which described the Analysis, Design, Development, Implementation, and Evaluation model. Six articles were based on the Accreditation Council for Graduate Medical Education criterion, although there was significant variability in content. The most common teaching method was role modeling with scenario- and case-based learning. A wide range of assessment methods for evaluating professionalism education were reported. The Kirkpatrick model was reported in one article as a method for program evaluation. Conclusion: It is suggested that for a core surgical professionalism education program, developmental/instructional design model, aims and competencies, content, teaching methods, assessment methods, and program evaluation methods/models should be well defined, and the content should be comparable. PMID:29553464

External quality-assurance programs managed by the U.S. Geological Survey in support of the National Atmospheric Deposition Program/National Trends Network

USGS Publications Warehouse

Latysh, Natalie E.; Wetherbee, Gregory A.

2005-01-01

The U.S. Geological Survey, Branch of Quality Systems, operates the external quality-assurance programs for the National Atmospheric Deposition Program/National Trends Network (NADP/NTN). Beginning in 1978, six different programs have been implemented?the intersite-comparison program, the blind-audit program, the sample-handling evaluation program, the field-audit program, the interlaboratory-comparison program, and the collocated-sampler program. Each program was designed to measure error contributed by specific components in the data-collection process. The intersite-comparison program, which was discontinued in 2004, was designed to assess the accuracy and reliability of field pH and specific-conductance measurements made by site operators. The blind-audit and sample-handling evaluation programs, which also were discontinued in 2002 and 2004, respectively, assessed contamination that may result from sampling equipment and routine handling and processing of the wet-deposition samples. The field-audit program assesses the effects of sample handling, processing, and field exposure. The interlaboratory-comparison program evaluates bias and precision of analytical results produced by the contract laboratory for NADP, the Illinois State Water Survey, Central Analytical Laboratory, and compares its performance with the performance of international laboratories. The collocated-sampler program assesses the overall precision of wet-deposition data collected by NADP/NTN. This report documents historical operations and the operating procedures for each of these external quality-assurance programs. USGS quality-assurance information allows NADP/NTN data users to discern between actual environmental trends and inherent measurement variability.

Evaluating Integrative Cancer Clinics With the Claim Assessment Profile: An Example With the InspireHealth Clinic.

PubMed

Hilton, Lara; Elfenbaum, Pamela; Jain, Shamini; Sprengel, Meredith; Jonas, Wayne B

2018-03-01

The evaluation of freestanding integrative cancer clinical programs is challenging and is rarely done. We have developed an approach called the Claim Assessment Profile (CAP) to identify whether evaluation of a practice is justified, feasible, and likely to provide useful information. A CAP was performed in order to (1) clarify the healing claims at InspireHealth, an integrative oncology treatment program, by defining the most important impacts on its clients; (2) gather information about current research capacity at the clinic; and (3) create a program theory and path model for use in prospective research. This case study design incorporates methods from a variety of rapid assessment approaches. Procedures included site visits to observe the program, structured qualitative interviews with 26 providers and staff, surveys to capture descriptive data about the program, and observational data on program implementation. The InspireHealth program is a well-established, multi-site, thriving integrative oncology clinical practice that focuses on patient support, motivation, and health behavior engagement. It delivers patient-centered care via a standardized treatment protocol. There arehigh levels of research interest from staff and resources by which to conduct research. This analysis provides the primary descriptive and claims clarification of an integrative oncology treatment program, an evaluation readiness report, a detailed logic model explicating program theory, and a clinical outcomes path model for conducting prospective research. Prospective evaluation of this program would be feasible and valuable, adding to our knowledge base of integrative cancer therapies.

Troubleshooting Assessment and Enhancement (TAE) Program: Test and Evaluation

DTIC Science & Technology

1991-04-01

Navy Personnel Research and Development Center San Diego, California 92152-6800 TN-91-13 April 1991 AD-A23 6 323 Troubleshooting Assessment and...00002 NPRDC-TN-91-13 April 1991 Troubleshooting Assessment and Enhancement (TAE) Program: Test and Evaluation Harry B. Conner Navy Personnel Research ...McLachlan Director, Training Systems Department Approved for public release; distribution is unlimited. Navy Personnel Research and Development Center

A mentor-based portfolio program to evaluate pharmacy students' self-assessment skills.

PubMed

Kalata, Lindsay R; Abate, Marie A

2013-05-13

Objective. To evaluate pharmacy students' self-assessment skills with an electronic portfolio program using mentor evaluators. Design. First-year (P1) and second-year (P2) pharmacy students used online portfolios that required self-assessments of specific graded class assignments. Using a rubric, faculty and alumni mentors evaluated students' self-assessments and provided feedback. Assessment. Eighty-four P1 students, 74 P2 students, and 59 mentors participated in the portfolio program during 2010-2011. Both student groups performed well overall, with only a small number of resubmissions required. P1 students showed significant improvements across semesters for 2 of the self-assessment questions; P2 students' scores did not differ significantly. The P1 scores were significantly higher than P2 scores for 3 questions during spring 2011. Mentors and students had similar levels of agreement with the extent to which students put forth their best effort on the self-assessments. Conclusion. An electronic portfolio using mentors based inside and outside the school provided students with many opportunities to practice their self-assessment skills. This system represents a useful method of incorporating self-assessments into the curriculum that allows for feedback to be provided to the students.

A Mentor-Based Portfolio Program to Evaluate Pharmacy Students’ Self-Assessment Skills

PubMed Central

Kalata, Lindsay R.

2013-01-01

Objective. To evaluate pharmacy students' self-assessment skills with an electronic portfolio program using mentor evaluators. Design. First-year (P1) and second-year (P2) pharmacy students used online portfolios that required self-assessments of specific graded class assignments. Using a rubric, faculty and alumni mentors evaluated students' self-assessments and provided feedback. Assessment. Eighty-four P1 students, 74 P2 students, and 59 mentors participated in the portfolio program during 2010-2011. Both student groups performed well overall, with only a small number of resubmissions required. P1 students showed significant improvements across semesters for 2 of the self-assessment questions; P2 students' scores did not differ significantly. The P1 scores were significantly higher than P2 scores for 3 questions during spring 2011. Mentors and students had similar levels of agreement with the extent to which students put forth their best effort on the self-assessments. Conclusion. An electronic portfolio using mentors based inside and outside the school provided students with many opportunities to practice their self-assessment skills. This system represents a useful method of incorporating self-assessments into the curriculum that allows for feedback to be provided to the students. PMID:23716749

Redesigning and Aligning Assessment and Evaluation for a Federally Funded Math and Science Teacher Educational Program

ERIC Educational Resources Information Center

Hardre, Patricia L.; Slater, Janis; Nanny, Mark

2010-01-01

This paper examines the redesign of evaluation components for a teacher professional development project funded by the National Science Foundation. It focuses on aligning evaluation instrumentation and strategies with program goals, research goals and program evaluation best practices. The study identifies weaknesses in the original (year 1)…

Assessing communications effectiveness in meeting corporate goals of public health organizations.

PubMed

Brown, Gordon D; Bopp, Kenneth D; Boren, Suzanne Austin

2005-01-01

Much evaluation of health communications in public health is considered from a program perspective of smoking cessation, weight reduction, education on sexually transmitted diseases, etc. These studies have advanced the knowledge base of communications theory and evaluation and have contributed to program effectiveness. In program-based evaluation the communications process is structured as part of the program itself. This article extends program-based communications evaluation to view communications from the perspective of the consumer and how effectively public health departments respond to consumer expectations. It develops a conceptual model for evaluating elements of communications such as its importance in defining mission and goals within the community, managing strategic constituencies, and enlisting individuals and groups as customers and co-producers of health. It gives a broader perspective on how communications in public heath organizations are managed and a basis for assessing whether they are being managed effectively.

Toward an Evaluation Framework for Doctoral Education in Social Work: A 10-Year Retrospective of One PhD Program's Assessment Experiences

ERIC Educational Resources Information Center

Bentley, Kia J.

2013-01-01

This article presents a framework for evaluation in social work doctoral education and details 10 years of successes and challenges in one PhD program's use of the framework, including planning and implementing specific assessment activities around student learning outcomes and larger program goals. The article argues that a range of…

Manned space station environmental control and life support system computer-aided technology assessment program

NASA Technical Reports Server (NTRS)

Hall, J. B., Jr.; Pickett, S. J.; Sage, K. H.

1984-01-01

A computer program for assessing manned space station environmental control and life support systems technology is described. The methodology, mission model parameters, evaluation criteria, and data base for 17 candidate technologies for providing metabolic oxygen and water to the crew are discussed. Examples are presented which demonstrate the capability of the program to evaluate candidate technology options for evolving space station requirements.

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program.

PubMed

Jaafaripooyan, Ebrahim

2014-09-01

Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs' over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs.

Using ePortfolios to Assess Applied and Collaborative Learning and Academic Identity in a Summer Research Program for Community College Students

ERIC Educational Resources Information Center

Singer-Freeman, Karen; Bastone, Linda; Skrivanek, Joseph

2016-01-01

We evaluate the extent to which ePortfolios can be used to assess applied and collaborative learning and academic identity among community college students from underrepresented minority groups who participated in a summer research program. Thirty-eight students were evaluated by their research sponsor and two or three naïve faculty evaluators.…

Barriers to Evaluation

ERIC Educational Resources Information Center

Charley, Lois

2007-01-01

According to the Department of Health and Human Services, "good program evaluations assess programs performance, measure impacts on families and communities, and document program successes." With this information, programs are able to direct limited resources to where they are most needed and most effective in their communities. This article is…

Manipulator system man-machine interface evaluation program. [technology assessment

NASA Technical Reports Server (NTRS)

Malone, T. B.; Kirkpatrick, M.; Shields, N. L.

1974-01-01

Application and requirements for remote manipulator systems for future space missions were investigated. A manipulator evaluation program was established to study the effects of various systems parameters on operator performance of tasks necessary for remotely manned missions. The program and laboratory facilities are described. Evaluation criteria and philosophy are discussed.

Cost Analysis for Educational Program Evaluation.

ERIC Educational Resources Information Center

Haller, Emil J.

Concerned with the problem of determining program costs as part of the evaluation process, this article helps the practitioner arrive at useful conceptions of the term "cost" and procedures for assessing the costs of an educational program. Its purpose is to help design costing procedures for evaluation situations that are commonly encountered. An…

Examining Criteria for Evaluating Educational Programs.

ERIC Educational Resources Information Center

Shapiro, Edna

The negative results to an assessment of a Bank Street-sponsored Follow Through program raised questions about conventional ways of assessing the effects of educational programs. Children in a Follow Through program using the developmental-interaction approach and children in conventional classrooms were observed and tested for comparison. The…

Program Development and External Assessment.

ERIC Educational Resources Information Center

Minnis, D. L.

Although the development of new models is essential to the improvement of teacher preparation programs, the California system of accreditation of teacher education programs seems to hinder innovative program development and evaluation. External assessment in California is based on a discrepancy model which works best when applied to static…

Apollo Photograph Evaluation (APE) programming manual

NASA Technical Reports Server (NTRS)

Kim, I. J.

1974-01-01

This document describes the programming techniques used to implement the equations of the Apollo Photograph Evaluation (APE) program on the UNIVAC 1108 computer and contains detailed descriptions of the program structure, a User's Guide section to provide the necessary information for proper operation of the program, and information for the assessment of the program's adaptability to future problems.

Evaluation of the Cosmetology Program at Caldwell Community College and Technical Institute--Fall, 1981.

ERIC Educational Resources Information Center

Pipes, V. David

In fall 1981, the cosmetology program at Caldwell Community College and Technical Institute (CCC&TI) was evaluated as part of a process to create a model for the periodic evaluation of all occupational programs at the school. In addition to collecting information for planning and program improvement, the study sought to assess the achievement of…

Process Evaluation of a School-Based Weight Gain Prevention Program: The Dutch Obesity Intervention in Teenagers (DOiT)

ERIC Educational Resources Information Center

Singh, A. S.; Chinapaw, M. J. M.; Brug, J.; van Mechelen, W.

2009-01-01

Health promotion programs benefit from an accompanying process evaluation since it can provide more insight in the strengths and weaknesses of a program. A process evaluation was conducted to assess the reach, implementation, satisfaction and maintenance of a school-based program aimed at the prevention of excessive weight gain among Dutch…

Assessment of the Teaching Behavior of the Instructors of an Out-of-School Program

ERIC Educational Resources Information Center

Rodríguez-Naveiras, Elena; Borges, África

2015-01-01

Out-of-school programs for students with high abilities are especially relevant when their needs are not covered in formal education. The evaluation of these programs is essential and it can be carried out from different evaluative approaches. The evaluation of the behavior of the people who implement the programs is an important aspect in the…

Assessment of village health worker training program in Tuguegarao, Philippine.

PubMed

Kim, Jung Min; Koh, Kwang Wook; Oak, Chul Ho; Jung, Woo Hyuk; Kim, Sung Hyun; Park, Dae Hee

2009-11-01

This study was performed to evaluate the effectiveness of 'village health worker training program' which aimed to build community participatory health promotion capacity of community leaders in villages of low developed country and to develop methods for further development of the program. The intervention group were 134 community leaders from 25 barangays (village). Control group were 149 form 4 barangays. Intervention group participated 3-day training program. Questionnaire was developed based on 'Health Promotion Capacity Checklist' which assessed capacity in 4 feathers; 'knowledge', 'skill', 'commitment', and 'resource'. Each feather was assessed in 4 point rating scale. Capacity scores between intervention group and control group were examined to identify changes between the pre- and post-intervention periods. A qualitative evaluation of the program was conducted to assess the appropriateness of the program. The program was conducted in Tuguegarao city, Philippine in January, 2009. The result showed significant increases in the total health promotion capacity and each feather of health promotion capacities between pre and post assessment of intervention group. But there was no significant change in that of control group. Participants marked high level of satisfaction for preparedness, selection of main subjects and education method. Qualitative evaluation revealed that training program facilitated community participatory health promotion capacity of participants. This study suggested that the Village health worker training program is effective for building health promotion capacity of community leaders and it can be a main method for helping low developed countries with further development.

Evidence-Based Medicine and State Health Care Coverage: The Washington Health Technology Assessment Program.

PubMed

Rothman, David J; Blackwood, Kristy L; Adair, Whitney; Rothman, Sheila M

2018-04-01

To evaluate the Washington State Health Technology Assessment Program (WHTAP). Washington State Health Technology Assessment Program proceedings in Seattle, Washington. We assessed the program through observation of its proceedings over a 5-year period, 2009-2014. We conducted detailed analyses of the documents it produced and reviewed relevant literature. Washington State Health Technology Assessment Program is unique compared to other state and federal programs. It has successfully applied evidence-based medicine to health care decision making, limited by the strength of available data. It claims cost savings, but they are not substantiated. Washington State Health Technology Assessment Program is a useful model for other states considering implementation of technology assessment programs. We provide key lessons for improving WHTAP's process. © Health Research and Educational Trust.

Changes in University Students after Joining a Service Leadership Program in China

ERIC Educational Resources Information Center

Shek, Daniel T. L.; Lin, Li

2016-01-01

This study examined the effectiveness of a 4.5-day service leadership program for students from Chinese universities using objective outcome evaluation. The participants were assessed before and after the program, with two post-test measurements (immediate assessment and assessment 12 days after the completion of class learning). At pretest and…

The Effectiveness of a Web-Based Motor Skill Assessment Training Program

ERIC Educational Resources Information Center

Kelly, Luke E.; Moran, Thomas E.

2010-01-01

The purpose of this study was to evaluate the effectiveness of a web-based, intereactive video assessment program on teaching preservice physical education majors to assess the motor skill of kicking. The program provided component specific feedback through tutorial, guided practice, and competency training options. The 72 participants were…

Assessing the Effectiveness of Multimedia in Language Learning Software.

ERIC Educational Resources Information Center

Chun, Dorothy M.; Plass, Jan L.

In this paper, the effectiveness of a "CyberBuch," a multimedia program for reading authentic German texts, is assessed in three areas. First, based on user evaluation of the visual interface design, the usability of the program is assessed with particular regard to user reaction to the multimedia components of the program. Second,…

78 FR 51719 - Draft Toxicological Review of Benzo[a]pyrene: In Support of the Summary Information in the...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-08-21

... Environmental Assessment (NCEA) within the EPA Office of Research and Development (ORD). EPA is releasing this... Program is a human health assessment program that evaluates quantitative and qualitative risk information... IRIS Program, EPA provides the highest quality science-based human health assessments to support the...

Does Computer-Based Motor Skill Assessment Training Transfer to Live Assessing?

ERIC Educational Resources Information Center

Kelly, Luke E.; Taliaferro, Andrea; Krause, Jennifer

2012-01-01

Developing competency in motor skill assessment has been identified as a critical need in physical educator preparation. We conducted this study to evaluate (a) the effectiveness of a web-based instructional program--Motor Skill Assessment Program (MSAP)--for developing assessment competency, and specifically (b) whether competency developed by…

Curricular Innovation in an Undergraduate Medical Program: What Is "Appropriate" Assessment?

ERIC Educational Resources Information Center

Ruhe, Valerie; Boudreau, J. Donald

2011-01-01

In post-secondary education, there is a widely-held belief in a "gold standard" for evaluative studies of curricular innovations. In this context, "appropriate" assessment is understood to refer to experimental designs and statistically significant differences in group outcomes. Yet in our evaluative study of a medical undergraduate program, we…

Community Jobs Outcomes Assessment & Program Evaluation.

ERIC Educational Resources Information Center

Case, Annette; Burchfield, Erin; Sommers, Paul

Unemployment wage data were evaluated to assess employment, job retention, and wage progression for graduates of Community Jobs (CJ), a short-term public job creation program for the hard to employ in the state of Washington. The following were among the findings: (1) 66% of all participants were employed after graduating from CJ; (2) 53% were…

Data-Based Locally Directed Evaluation of Vocational Education Programs. Component 3. Assessing Student Career Interests.

ERIC Educational Resources Information Center

Florida State Univ., Tallahassee. Program of Vocational Education.

Part of a system by which local education agency (LEA) personnel may evaluate secondary and postsecondary vocational education programs, this third of eight components focuses on assessment of student career interests. Procedures covered include determination of the student population to be surveyed, selection of the instrument, and discrepancy…

Value-Added Models for Teacher Preparation Programs: Validity and Reliability Threats, and a Manageable Alternative

ERIC Educational Resources Information Center

Brady, Michael P.; Heiser, Lawrence A.; McCormick, Jazarae K.; Forgan, James

2016-01-01

High-stakes standardized student assessments are increasingly used in value-added evaluation models to connect teacher performance to P-12 student learning. These assessments are also being used to evaluate teacher preparation programs, despite validity and reliability threats. A more rational model linking student performance to candidates who…

Process evaluation of the Enabling Mothers toPrevent Pediatric Obesity Through Web-Based Learning and Reciprocal Determinism (EMPOWER) randomized control trial.

PubMed

Knowlden, Adam P; Sharma, Manoj

2014-09-01

Family-and-home-based interventions are an important vehicle for preventing childhood obesity. Systematic process evaluations have not been routinely conducted in assessment of these interventions. The purpose of this study was to plan and conduct a process evaluation of the Enabling Mothers to Prevent Pediatric Obesity Through Web-Based Learning and Reciprocal Determinism (EMPOWER) randomized control trial. The trial was composed of two web-based, mother-centered interventions for prevention of obesity in children between 4 and 6 years of age. Process evaluation used the components of program fidelity, dose delivered, dose received, context, reach, and recruitment. Categorical process evaluation data (program fidelity, dose delivered, dose exposure, and context) were assessed using Program Implementation Index (PII) values. Continuous process evaluation variables (dose satisfaction and recruitment) were assessed using ANOVA tests to evaluate mean differences between groups (experimental and control) and sessions (sessions 1 through 5). Process evaluation results found that both groups (experimental and control) were equivalent, and interventions were administered as planned. Analysis of web-based intervention process objectives requires tailoring of process evaluation models for online delivery. Dissemination of process evaluation results can advance best practices for implementing effective online health promotion programs. © 2014 Society for Public Health Education.

Florida maintenance rating program (MRP) assessment and enhancement : final report, May 28, 2008 [summary].

DOT National Transportation Integrated Search

2008-01-01

The Florida Department of Transportation : (FDOT) has used its maintenance rating : program (MRP) to evaluate the states : highway maintenance conditions and : determine asset maintenance needs since : 1985. Periodic evaluation of the program is :...

Implementation of Portfolio Assessment in a Competency-based Dental Hygiene Program.

ERIC Educational Resources Information Center

Gadbury-Amyot, Cynthia C.; Holt, Lorie P.; Overman, Pamela R.; Schmidt, Colleen R.

2000-01-01

Describes the implementation of a portfolio assessment program in the dental hygiene program at the University of Missouri School of Dentistry. Tables provide examples of program competencies and related portfolio entries, the complete scoring rubric for portfolios, and the student portfolio evaluation survey. Concludes that although portfolio…

Program Evaluation of Services for the Homeless: Challenges and Strategies.

ERIC Educational Resources Information Center

Mercier, Celine; And Others

1992-01-01

Research strategies, including types of evaluations, designs, and indicators, developed to assess programs for chronic alcoholics and mentally ill homeless people in Canada are reviewed. Findings from previous evaluations are summarized, and the implications for evaluation practice are considered. (SLD)

Reaching Your Program Goals: The Secret to a Successful Relationship with Your Evaluator

NASA Astrophysics Data System (ADS)

Warburton, J.; Crowley, S.; Larson, A.

2012-12-01

PolarTREC (Teachers and Researchers Exploring and Collaborating) is a National Science Foundation (NSF) funded program in which K-12 teachers participate in hands-on field research experiences in the Polar Regions. PolarTREC has become a leader in the evaluation of teacher research experiences (TREs) and offers a model for outside evaluation for education and public outreach. Programs such as TRE's offer a direct package of outreach opportunities for effective broader impacts. This presentation offers a model for the evaluative process beginning with project design and goals, which involve an outside evaluator, and that provide funders with an outlined scope of projects as well as outcomes and products for use by the public to advance scientific understanding. Outcomes are determined before the program is developed to ensure that all components are striving for program efficacy. Formative and summative evaluations for all audiences ensure robust program reports via survey's, knowledge assessments, conference calls, outreach implementation plans for multiple diverse audiences and in-depth case studies with teachers back in their classrooms. PolarTREC has become part of a larger, international working group of evaluators and program managers on program assessment. As part of this working group, sharing best practices for effective evaluation to better support science efforts, inform funders, and communicate with the public, has been integral to the evolution and advancement of our successful evaluation model.

Evaluation of a community based childhood injury prevention program.

PubMed Central

Bablouzian, L.; Freedman, E. S.; Wolski, K. E.; Fried, L. E.

1997-01-01

OBJECTIVES: This pilot study evaluates the effectiveness of a community based childhood injury prevention program on the reduction of home hazards. METHODS: High risk pregnant women, who were enrolled in a home visiting program that augments existing health and human services, received initial home safety assessments. Clients received education about injury prevention practices, in addition to receiving selected home safety supplies. Fourteen questions from the initial assessment tool were repeated upon discharge from the program. Matched analyses were conducted to evaluate differences from initial assessment to discharge. RESULTS: A significantly larger proportion of homes were assessed as safe at discharge, compared with the initial assessment, for the following hazards: children riding unbuckled in all auto travel, Massachusetts Poison Center sticker on the telephone, outlet plugs in all unused electrical outlets, safety latches on cabinets and drawers, and syrup of ipecac in the home. CONCLUSIONS: A community based childhood injury prevention program providing education and safety supplies to clients significantly reduced four home hazards for which safety supplies were provided. Education and promotion of the proper use of child restraint systems in automobiles significantly reduced a fifth hazard, children riding unbuckled in auto travel. This program appears to reduce the prevalence of home hazards and, therefore, to increase home safety. PMID:9113841

Using EPAS[TM] to Evaluate School-Based Intervention Programs: GEAR UP. Case Study

ERIC Educational Resources Information Center

ACT, Inc., 2007

2007-01-01

This brief examines how the ACT's EPAS[TM] (Educational Planning and Assessment System) can be used to evaluate school-based intervention programs. Specific evaluation considered is that of the federal government's Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP), an initiative designed to increase the college awareness…

Assessing the Quality of the Business and Management Education in Higher Education

ERIC Educational Resources Information Center

Lee, Lung-Sheng; Ko, Hui-Min; Wang, Mei-Tyng; Pan, Ying-Ju

2014-01-01

As the third-party planner and implementer of higher education institutional and program evaluations, the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT) completed program evaluations for all 145 undergraduate business and management (B&M) programs in 43 universities/colleges from 2006 to 2010. In the 145 programs…

Evaluation Methodologies for Estimating the Likelihood of Program Implementation Failure

ERIC Educational Resources Information Center

Durand, Roger; Decker, Phillip J.; Kirkman, Dorothy M.

2014-01-01

Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the "why" of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities.…

Evaluation of Illinois Benedictine's Freshman Advising Program via the New Benedictine Advising Survey.

ERIC Educational Resources Information Center

Iaccino, James F.

Many colleges and universities lack a formal evaluation system to assess their advising programs. Illinois Benedictine College, however, has consistently evaluated its Freshman Advising Program (FAP) on an annual basis ever since its inception. In 1986, the college decided to implement the new Benedictine Advising Survey in the exit interview…

A multi-level assessment of a program to teach medical students to teach.

PubMed

Blatt, Benjamin; Greenberg, Larrie

2007-02-01

Few longitudinal programs exist to teach senior students (MS4s) to be teachers, nor have there been any reports of comprehensive program evaluation in this area. The primary objectives of this study were to describe our ongoing faculty development effort and to develop a multi-level program evaluation, using Dixon's model. The TALKS (Teaching and Learning Communication Skills) program is a senior elective and open to all MS4s. We evaluated our program through assessment of its participants at three levels: level 1, opinion; level 2, competence; and level 3, performance; but not level 4, patient outcomes. The authors used a retrospective, pre-post questionnaire to assess MS4 attitudes about their educational experiences, a traditional instrument to assess their teaching, an interaction analysis technique using Bloom's taxonomy to assess MS4s' feedback skills, and a SP exam to assess MS4 communication skills. The authors hypothesized that MS4s participating in TALKS would view medical education more positively and informatively, would demonstrate important principles in giving feedback, would be assessed as excellent teachers, and would perform better than controls in an SP exam emphasizing communication skills. Results revealed that MS4s' ratings as teachers were very good to excellent, with the highest scores on the items "knowledgeable, supportive of me, and answering questions clearly." (Level 1, Opinion) MS4s' perceptions of their knowledge, attitudes and skills increased significantly from the pre to the post-questionnaire. (Level 2, Competence) MS4 feedback skills to MS2s revealed they did more talking than ideal, often at the lowest levels of Bloom's taxonomy. (Level 3, Performance) MS4s demonstrated better communication skills than controls on an evaluation by professional SPs. (Level 3, Performance).

Assessing the Value of the Enviroscape Watershed Learning Module

ERIC Educational Resources Information Center

Edwards, Warren Patrick

2013-01-01

Scope and Method of Study: The researcher's evaluation of the West Atlanta Watershed Alliance's (WAWA) programs highlighted that few if any of the offered educational programs included a program evaluation, especially the most promising, the Enviroscape® Watershed learning module. The education programs that were customized and developed by the…

SPECS 2. Component Two: Program Budgeting and Cost Accounting.

ERIC Educational Resources Information Center

DuVall, Lloyd A.

The School Planning, Evaluation, and Communication System (SPECS) is an overall design for districtwide program assessment and development. It is both a way of interrelating planning and evaluation of individual programs and a way of comparing the outcomes of all district programs with the district's community-defined educational goals. Component…

Benefits of the NADE Certification Process: Self-Knowledge, Informed Choices & Programmatic Strength

ERIC Educational Resources Information Center

Greci, Dana

2016-01-01

Research shows that systematic, ongoing program evaluation is needed to adequately assess the effectiveness of developmental programs. Ongoing evaluation provides both (1) validation of programs to educational institutions and legislators; and (2) impetus for program improvement. The author of this article works in the Developmental Education…

Evaluation as a critical factor of success in local public health accreditation programs.

PubMed

Tremain, Beverly; Davis, Mary; Joly, Brenda; Edgar, Mark; Kushion, Mary L; Schmidt, Rita

2007-01-01

This article presents the variety of approaches used to conduct evaluations of performance improvement or accreditation systems, while illustrating the complexity of conducting evaluations to inform local public health practice. We, in addition, hope to inform the Exploring Accreditation Program about relevant experiences involving accreditation and performance assessment processes, specifically evaluation, as it debates and discusses a national voluntary model. A background of each state is given. To further explore these issues, interviews were conducted with each state's evaluator to gain more in-depth information on the many different evaluation strategies and approaches used. On the basis of the interviews, the authors provide several overall themes, which suggest that evaluation is a critical tool and success factor for performance assessment or accreditation programs.

Assessing the Subsequent Effect of a Formative Evaluation on a Program.

ERIC Educational Resources Information Center

Brown, J. Lynne; Kiernan, Nancy Ellen

2001-01-01

Conducted a formative evaluation of an osteoporosis prevention health education program using several methods, including questionnaires completed by 256 women, and then compared formative evaluation results to those of a summative evaluation focusing on the same target group. Results show the usefulness of formative evaluation for strengthening…

Measuring Success in Obesity Prevention: A Synthesis of Health Promotion Switzerland's Long-Term Monitoring and Evaluation Strategy

PubMed Central

Ackermann, Günter; Kirschner, Michael; Guggenbühl, Lisa; Abel, Bettina; Klohn, Axel; Mattig, Thomas

2015-01-01

Aims Since 2007, Health Promotion Switzerland has implemented a national priority program for a healthy body weight. This article provides insight into the methodological challenges and results of the program evaluation. Methods Evaluation of the long-term program required targeted monitoring and evaluation projects addressing different outcome levels. The evaluation was carried out according to the Swiss Model for Outcome Classification (SMOC), a model designed to classify the effects of health promotion and prevention efforts. Results The results presented in this article emphasize both content and methods. The national program successfully achieved outcomes on many different levels within complex societal structures. The evaluation system built around the SMOC enabled assessment of program progress and the development of key indicators. However, it is not possible to determine definitively to what extent the national program helped stabilize the prevalence of obesity in Switzerland. Conclusion The model has shown its utility in providing a basis for evaluation and monitoring of the national program. Continuous analysis of data from evaluation and monitoring has made it possible to check the plausibility of suspected causal relationships as well as to establish an overall perspective and assessment of effectiveness supported by a growing body of evidence. PMID:25765161

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program

PubMed Central

Jaafaripooyan, Ebrahim

2014-01-01

Background: Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. Methods: A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Results: Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs’ over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Conclusion: Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs. PMID:25279381

Evaluating the implementation and impacts of middle grades inquiry-based engineering design modules

NASA Astrophysics Data System (ADS)

Harlan, Jessica M.

This dissertation uses a format where I present and discuss three articles that were written in conjunction with my work on a research and evaluation team. The articles are based on a multi-year project researching and evaluating the design, development, and implementation of a middle grades integrated Science, Technology, Engineering, and Mathematics (STEM) program. Each of the articles demonstrates the complexity in researching and evaluating curricular development in multifaceted, rapidly changing environments. While the focus of each article differs, they all examine research and evaluation in the context of the design and development of middle grades engineering-design modules. The selected articles address challenges associated with assessing program objectives and evaluating program quality in complex education programs.The first article, presented in Chapter 2, provides an overview of the nature of the EYE program and examines the extent to which participation in the EYE program resulted in the achievement of program objectives. There is evidence that EYE Module participation has a positive impact on participating students as well as teachers. This study also revealed challenges associated with determining the impact of program participation simultaneously with program revisions and assessment development. The second article, presented in Chapter 3, examined the evaluation of fidelity of implementation of inquiry-oriented educational programs. This article was intended to examine one way of triangulating information to determine fidelity while considering variation in implementation consistent with program theories of learning. When applying this model to implementation of the EYE program, we found many teachers were implementing the modules with low to moderate fidelity, especially math teachers. The third article, presented in Chapter 4, examined the factor structure of an occupational values scale intended to measure student interest in STEM careers. When reviewing the assessment's evidence base, we found no published literature regarding the assessment's psychometric properties. We found that the AWE Work Values scale assesses two separate sets of occupational values: 1) using analytical and problem solving skills and 2) personal satisfaction. In Chapter 5, I discuss the implications of the research presented in these articles to the field of integrated STEM. This includes a need to further develop and test tools and methods for measuring program impact. While the articles address several issues related to research and evaluation within the context of K-12 integrated STEM education, the concepts and implications also apply to the general field of instructional design. The primary lesson is the need to systematically evaluate designed instruction, including measuring fidelity of implementation and designing evaluations in advance to ensure appropriate data are gathered. Recommendations for future research include identification of common outcomes for integrated STEM, psychometric testing of integrated STEM attitude measures, impacts of other factors such as interest in STEM, and development and testing of instructional design models for integrated STEM.

Comparative Evaluation of Financing Programs: Insights From California’s Experience

DOE Office of Scientific and Technical Information (OSTI.GOV)

Deason, Jeff

Berkeley Lab examines criteria for a comparative assessment of multiple financing programs for energy efficiency, developed through a statewide public process in California. The state legislature directed the California Alternative Energy and Advanced Transportation Financing Authority (CAEATFA) to develop these criteria. CAEATFA's report to the legislature, an invaluable reference for other jurisdictions considering these topics, discusses the proposed criteria and the rationales behind them in detail. Berkeley Lab's brief focuses on several salient issues that emerged during the criteria development and discussion process. Many of these issues are likely to arise in other states that plan to evaluate the impactsmore » of energy efficiency financing programs, whether for a single program or multiple programs. Issues discussed in the brief include: -The stakeholder process to develop the proposed assessment criteria -Attribution of outcomes - such as energy savings - to financing programs vs. other drivers -Choosing the outcome metric of primary interest: program take-up levels vs. savings -The use of net benefits vs. benefit-cost ratios for cost-effectiveness evaluation -Non-energy factors -Consumer protection factors -Market transformation impacts -Accommodating varying program goals in a multi-program evaluation -Accounting for costs and risks borne by various parties, including taxpayers and utility customers, in cost-effectiveness analysis -How to account for potential synergies among programs in a multi-program evaluation« less

Socioeconomic impact assessment in ex ante evaluations: a case study on the rural development programs of the European Union.

PubMed

Vidueira, Pablo; Díaz-Puente, José M; Rivera, María

2014-08-01

Ex ante impact assessment has become a fundamental tool for effective program management, and thus, a compulsory task when establishing a new program in the European Union (EU). This article aims to analyze benefits from ex ante impact assessment, methodologies followed, and difficulties encountered. This is done through the case study on the rural development programs (RDPs) in the EU. Results regarding methodologies are then contrasted with the international context in order to provide solid insights to evaluators and program managing authorities facing ex ante impact assessment. All European RDPs from the period 2007 through 2013 (a total of 88) and their corresponding available ex ante evaluations (a total of 70) were analyzed focusing on the socioeconomic impact assessment. Only 46.6% of the regions provide quantified impact estimations on socioeconomic impacts in spite of it being a compulsory task demanded by the European Commission (EC). Recommended methods by the EC are mostly used, but there is a lack of mixed method approaches since qualitative methods are used in substitution of quantitative ones. Two main difficulties argued were the complexity of program impacts and the lack of needed program information. Qualitative approaches on their own have been found as not suitable for ex ante impact assessment, while quantitative approaches-such as microsimulation models-provide a good approximation to actual impacts. However, time and budgetary constraints make that quantitative and mixed methods should be mainly applied on the most relevant impacts for the program success. © The Author(s) 2014.

10 CFR 712.36 - Medical assessment process.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 10 Energy 4 2011-01-01 2011-01-01 false Medical assessment process. 712.36 Section 712.36 Energy DEPARTMENT OF ENERGY HUMAN RELIABILITY PROGRAM Medical Standards § 712.36 Medical assessment process. (a) The... the SOMD must integrate the medical evaluations, psychological evaluations, psychiatric evaluations...

Evaluating Integrative Cancer Clinics With the Claim Assessment Profile: An Example With the InspireHealth Clinic

PubMed Central

Hilton, Lara; Elfenbaum, Pamela; Jain, Shamini; Sprengel, Meredith; Jonas, Wayne B.

2016-01-01

Background: The evaluation of freestanding integrative cancer clinical programs is challenging and is rarely done. We have developed an approach called the Claim Assessment Profile (CAP) to identify whether evaluation of a practice is justified, feasible, and likely to provide useful information. Objectives: A CAP was performed in order to (1) clarify the healing claims at InspireHealth, an integrative oncology treatment program, by defining the most important impacts on its clients; (2) gather information about current research capacity at the clinic; and (3) create a program theory and path model for use in prospective research. Study Design/Methods: This case study design incorporates methods from a variety of rapid assessment approaches. Procedures included site visits to observe the program, structured qualitative interviews with 26 providers and staff, surveys to capture descriptive data about the program, and observational data on program implementation. Results: The InspireHealth program is a well-established, multi-site, thriving integrative oncology clinical practice that focuses on patient support, motivation, and health behavior engagement. It delivers patient-centered care via a standardized treatment protocol. There arehigh levels of research interest from staff and resources by which to conduct research. Conclusions: This analysis provides the primary descriptive and claims clarification of an integrative oncology treatment program, an evaluation readiness report, a detailed logic model explicating program theory, and a clinical outcomes path model for conducting prospective research. Prospective evaluation of this program would be feasible and valuable, adding to our knowledge base of integrative cancer therapies. PMID:29444602

Evaluation of the CAPE and CORE Programs for Sensory-Impaired Multi-Handicapped Children in the Province of Alberta. Final Report.

ERIC Educational Resources Information Center

Rutland Consulting Group Ltd.

The report presents summaries of evaluations of the Coordinated Assessment and Program Planning for Education (CAPE) Program and the Coordinated Rehabilitation and Education (CORE) program for multi-handicapped sensory impaired and/or communication and behavior disordered children and their families in Alberta, Canada. Each program is evaluated…

Analysis and production of the traffic incident management state self-assessments (TIM SA) national report.

DOT National Transportation Integrated Search

2008-09-29

The Traffic Incident Management Self-Assessment (TIM SA) provides a means for evaluating progress in achievement of individual TIM program components and overall TIM program success in three areas: Program and Institutional Issues; Operational Issues...

Methods and Purposes for Conducting Students' Course Evaluations Reported by North American Dental School and Dental Hygiene Program Leaders: A Preliminary Survey Study.

PubMed

Reinke, Robin E; Enright, Tonya; Love, Rebecca; Patel, Shalizeh A; Ali, Ala Omar; Horvath, Zsuzsa

2018-03-01

The aim of this preliminary survey study was to determine the perceptions of leaders of dental schools and dental hygiene programs regarding methods of and purposes for conducting students' course evaluations and their role in course improvement, curriculum design, and faculty assessment. A short electronic survey was distributed in 2016 to the academic deans of all 76 dental schools in the U.S. and Canada and a convenience sample of program directors of 232 of the total 332 accredited dental hygiene programs. Individuals from 93 institutions responded for an overall response rate of 30%: 30 of 76 dental schools (39.5% response rate) and 63 of the 232 dental hygiene programs (27% response rate). All of the respondents (100%) reported that their institutions' full-time faculty members were assessed by students in course evaluations for each course and semester they taught. However, only 78% reported that their part-time faculty members were evaluated by students. Course evaluations were mandatory in 62% (n=58) of the responding institutions, with the remaining 38% (n=35) optional. Respondents indicated course directors received the evaluation results for purposes of annual review (n=73, 78%) and instructional review (n=70, 75%). Further investigation of the use and effects of student evaluations is needed to better understand their role in faculty assessment and other aspects of the administration of dental schools and dental hygiene programs.

An Assessment System for Competence Based Education: The Educational Development, Dissemination, and Evaluation Training Program.

ERIC Educational Resources Information Center

Hood, Paul D.; Blackwell, Laird

This manual provides a description of the development and a guide to the use of the assessment resources developed in connection with the Far West Development, Dissemination, and Evaluation (DD&E) Functional Competence Training Program. The document concentrates on a user-oriented description of the content, validation, and use of the final…

Evaluation of the Enhanced Assessment Grants (EAGs) SimScientists Program: Site Visit Findings. CRESST Report 791

ERIC Educational Resources Information Center

Herman, Joan; Dai, Yunyun; Htut, Aye Mon; Martinez, Marcela; Rivera, Nichole

2011-01-01

This evaluation report addresses the implementation, utility, and feasibility of SimScientists' simulation-based assessments for middle school science classrooms, with particular attention to the use of accommodations available in the program. Data were collected from a convenience sample of five schools and eight teachers across three states…

The Use of Workforce Assessment as a Component of Career and Technical Education Program Evaluation

ERIC Educational Resources Information Center

Bartlett, Kenneth R.; Schleif, Nicole L.; Bowen, Mauvalyn M.

2011-01-01

This research project examined the extent to which Career and Technical Education (CTE)-related programs use workforce needs assessment as a component of their evaluation activities. An employer perspective was used to develop a conceptual framework drawing on strategic human resource management theory. The extent and methods utilized for…

Intervention mapping: a process for developing theory- and evidence-based health education programs.

PubMed

Bartholomew, L K; Parcel, G S; Kok, G

1998-10-01

The practice of health education involves three major program-planning activities: needs assessment, program development, and evaluation. Over the past 20 years, significant enhancements have been made to the conceptual base and practice of health education. Models that outline explicit procedures and detailed conceptualization of community assessment and evaluation have been developed. Other advancements include the application of theory to health education and promotion program development and implementation. However, there remains a need for more explicit specification of the processes by which one uses theory and empirical findings to develop interventions. This article presents the origins, purpose, and description of Intervention Mapping, a framework for health education intervention development. Intervention Mapping is composed of five steps: (1) creating a matrix of proximal program objectives, (2) selecting theory-based intervention methods and practical strategies, (3) designing and organizing a program, (4) specifying adoption and implementation plans, and (5) generating program evaluation plans.

Satellite power system: Concept development and evaluation program. Volume 3: Power transmission and reception. Technical summary and assessment

NASA Technical Reports Server (NTRS)

Dietz, R. H.; Arndt, G. D.; Seyl, J. W.; Leopold, L.; Kelley, J. S.

1981-01-01

Efforts in the DOE/NASA concept development and evaluation program are discussed for the solar power satellite power transmission and reception system. A technical summary is provided together with a summary of system assessment activities. System options and system definition drivers are described. Major system assessment activities were in support of the reference system definition, solid state system studies, critical technology supporting investigations, and various system and subsystem tradeoffs. These activities are described together with reference system updates and alternative concepts for each of the subsystem areas. Conclusions reached as a result of the numerous analytical and experimental evaluations are presented. Remaining issues for a possible follow-on program are identified.

Assessment of Evidence-based Management Training Program: Application of a Logic Model.

PubMed

Guo, Ruiling; Farnsworth, Tracy J; Hermanson, Patrick M

2016-06-01

The purposes of this study were to apply a logic model to plan and implement an evidence-based management (EBMgt) educational training program for healthcare administrators and to examine whether a logic model is a useful tool for evaluating the outcomes of the educational program. The logic model was used as a conceptual framework to guide the investigators in developing an EBMgt educational training program and evaluating the outcomes of the program. The major components of the logic model were constructed as inputs, outputs, and outcomes/impacts. The investigators delineated the logic model based on the results of the needs assessment survey. Two 3-hour training workshops were delivered to 30 participants. To assess the outcomes of the EBMgt educational program, pre- and post-tests and self-reflection surveys were conducted. The data were collected and analyzed descriptively and inferentially, using the IBM Statistical Package for the Social Sciences (SPSS) 22.0. A paired sample t-test was performed to compare the differences in participants' EBMgt knowledge and skills prior to and after the training. The assessment results showed that there was a statistically significant difference in participants' EBMgt knowledge and information searching skills before and after the training (p< 0.001). Participants' confidence in using the EBMgt approach for decision-making was significantly increased after the training workshops (p< 0.001). Eighty-three percent of participants indicated that the knowledge and skills they gained through the training program could be used for future management decision-making in their healthcare organizations. The overall evaluation results of the program were positive. It is suggested that the logic model is a useful tool for program planning, implementation, and evaluation, and it also improves the outcomes of the educational program.

Instructor Perspectives of Multiple-Choice Questions in Summative Assessment for Novice Programmers

ERIC Educational Resources Information Center

Shuhidan, Shuhaida; Hamilton, Margaret; D'Souza, Daryl

2010-01-01

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with…

Tailored program evaluation: Past, present, future.

PubMed

Suggs, L Suzanne; Cowdery, Joan E; Carroll, Jennifer B

2006-11-01

This paper discusses measurement issues related to the evaluation of computer-tailored health behavior change programs. As the first generation of commercially available tailored products is utilized in health promotion programming, programmers and researchers are becoming aware of the unique challenges that the evaluation of these programs presents. A project is presented that used an online tailored health behavior assessment (HBA) in a worksite setting. Process and outcome evaluation methods are described and include the challenges faced, and strategies proposed and implemented, for meeting them. Implications for future research in tailored program development, implementation, and evaluation are also discussed.

Environment, Safety, and Health Self-Assessment Report, Fiscal Year 2008

DOE Office of Scientific and Technical Information (OSTI.GOV)

Chernowski, John

2009-02-27

Lawrence Berkeley National Laboratory's Environment, Safety, and Health (ES&H) Self-Assessment Program ensures that Integrated Safety Management (ISM) is implemented institutionally and by all divisions. The Self-Assessment Program, managed by the Office of Contract Assurance (OCA), provides for an internal evaluation of all ES&H programs and systems at LBNL. The functions of the program are to ensure that work is conducted safely, and with minimal negative impact to workers, the public, and the environment. The Self-Assessment Program is also the mechanism used to institute continuous improvements to the Laboratory's ES&H programs. The program is described in LBNL/PUB 5344, Environment, Safety, andmore » Health Self-Assessment Program and is composed of four distinct assessments: the Division Self-Assessment, the Management of Environment, Safety, and Health (MESH) review, ES&H Technical Assurance, and the Appendix B Self-Assessment. The Division Self-Assessment uses the five core functions and seven guiding principles of ISM as the basis of evaluation. Metrics are created to measure performance in fulfilling ISM core functions and guiding principles, as well as promoting compliance with applicable regulations. The five core functions of ISM are as follows: (1) Define the Scope of Work; (2) Identify and Analyze Hazards; (3) Control the Hazards; (4) Perform the Work; and (5) Feedback and Improvement. The seven guiding principles of ISM are as follows: (1) Line Management Responsibility for ES&H; (2) Clear Roles and Responsibilities; (3) Competence Commensurate with Responsibilities; (4) Balanced Priorities; (5) Identification of ES&H Standards and Requirements; (6) Hazard Controls Tailored to the Work Performed; and (7) Operations Authorization. Performance indicators are developed by consensus with OCA, representatives from each division, and Environment, Health, and Safety (EH&S) Division program managers. Line management of each division performs the Division Self-Assessment annually. The primary focus of the review is workplace safety. The MESH review is an evaluation of division management of ES&H in its research and operations, focusing on implementation and effectiveness of the division's ISM plan. It is a peer review performed by members of the LBNL Safety Review Committee (SRC), with staff support from OCA. Each division receives a MESH review every two to four years, depending on the results of the previous review. The ES&H Technical Assurance Program (TAP) provides the framework for systematic reviews of ES&H programs and processes. The intent of ES&H Technical Assurance assessments is to provide assurance that ES&H programs and processes comply with their guiding regulations, are effective, and are properly implemented by LBNL divisions. The Appendix B Performance Evaluation and Measurement Plan (PEMP) requires that LBNL sustain and enhance the effectiveness of integrated safety, health, and environmental protection through a strong and well-deployed system. Information required for Appendix B is provided by EH&S Division functional managers. The annual Appendix B report is submitted at the close of the fiscal year. This assessment is the Department of Energy's (DOE) primary mechanism for evaluating LBNL's contract performance in ISM.« less

Evaluation of the Military Functional Assessment Program: Preliminary Assessment of the Construct Validity Using an Archived Database of Clinical Data.

PubMed

Kelley, Amanda M; Ranes, Bethany M; Estrada, Art; Grandizio, Catherine M

2015-01-01

Several important factors must be considered when deciding to return a soldier to duty after a traumatic brain injury (TBI). Premature return increases risk for not only second-impact syndrome during the acute phase but also permanent changes from repetitive concussions. Thus, there is a critical need for return-to-duty (RTD) assessment criteria that encompass the spectrum of injury and disease experienced by US soldiers, particularly TBI. To provide evidence-based standards to eventually serve as criteria for operational competence and performance of a soldier after injury. Specifically, the relationships between clinical assessments and novel military-specific tasks were evaluated. Exploratory analyses (including nonparametric tests and Spearman rank correlations) of an archived database. A total of 79 patients with TBI who participated in an RTD assessment program at a US Army rehabilitation and recovery center. Military Functional Assessment Program (to determine a soldier's operational competence and performance after TBI) tasks; Dizziness Handicap Inventory; Dynamic Visual Acuity (vestibular function); Sensory Organization Test (postural control); Repeatable Battery for the Assessment of Neuropsychological Status (neuropsychological screening test); Beck Depression Inventory-II; Beck Anxiety Inventory; Comprehensive Trail Making Test (visual search and sequencing); posttraumatic stress disorder checklist military version; Alcohol Use Disorders Identification Test; Epworth Sleepiness Scale; Patient Health Questionnaire; and Military Acute Concussion Evaluation. Selected military operational assessment tasks correlated significantly with clinical measures of vestibular function, psychological well-being, and cognitive function. Differences on occupational therapy assessments, a concussion screening tool, and a self-report health questionnaire were seen between those who passed and those who failed the RTD assessment. Specifically, those who passed the RTD assessment scored more favorably on these clinical assessments. This study demonstrated convergent validity between Military Functional Assessment Program tasks and clinical assessment scores. The Military Functional Assessment Program shows promise for augmenting decision making related to RTD and soldier skills. Additional research is needed to determine the effectiveness of this program in predicting RTD success.

An Evaluation of an Alternative Teacher Certification Program: A Matter of Trust

ERIC Educational Resources Information Center

Grubbs, Scott T.

2009-01-01

For the internal evaluator, the issue of trust can be one of the more difficult ethical matters to manage. An organization might choose to conduct an internal program evaluation for a variety of reasons, including capitalizing on the abilities of invested stakeholders to assess aspects of a program, building an institutional culture oriented…

A Mixed Methods Approach to Understanding School Counseling Program Evaluation: High School Counselors' Methods and Perceptions

ERIC Educational Resources Information Center

Aucoin, Jennifer Mangrum

2013-01-01

The purpose of this mixed methods concurrent triangulation study was to examine the program evaluation practices of high school counselors. A total of 294 high school counselors in Texas were assessed using a mixed methods concurrent triangulation design. A researcher-developed survey, the School Counseling Program Evaluation Questionnaire…

Residential Wilderness Programs: The Role of Social Support in Influencing Self-Evaluations of Male Adolescents

ERIC Educational Resources Information Center

Cook, Emily C.

2008-01-01

This qualitative study explores the aspects of a residential wilderness experience that informed self-evaluations in male adolescents, ages 12-16. To assess change in self-evaluations and program factors associated with change, qualitative interviews were conducted with adolescents upon entry to the program and four months later. Participants'…

CIRCUS: Comprehensive Program of Assessment Services for Preprimary Children

ERIC Educational Resources Information Center

Raths, James; Katz, Lilian G.

1975-01-01

Reviews CIRCUS, a comprehensive battery of assessment devices designed for use in nursey schools and kindergartens to diagnose the instructional needs of individual children and evaluate educational programs. (EVH)

Evaluating Mentoring Programs. P/PV Brief

ERIC Educational Resources Information Center

Grossman, Jean Baldwin

2009-01-01

This methodological brief is designed to provide both program operators and researchers with practical advice about how to assess a program's implementation and impact. Adapted from an article that first appeared in "The Handbook of Youth Mentoring" (DuBois and Karcher, ed. 2005), the brief focuses on the evaluation of mentoring programs, but the…

Pennsylvania's Family Literacy Programs: Results of a Statewide Evaluation, 1999-2000.

ERIC Educational Resources Information Center

Van Horn, Barbara; Kassab, Cathy; Grinder, Elisabeth

Pennsylvania's family literacy programs were subjected to a statewide evaluation in 1999-2000. Data were collected through focus groups with parents, a survey of staff in community agencies and organizations, and analyses of program data and the findings of selected assessments administered to adults and children involved in the program.…

ENVIRONMENTAL MONITORING AND ASSESSMENT PROGRAM - AGROECOSYSTEM PILOT FIELD PROGRAM PLAN - 1993

EPA Science Inventory

The Agroecosystem Resource Group (ARG) of the Environmental Monitoring and Assessment Program (EMAP) has developed a five-year (1991-1995) strategy for the development, evaluation, and implementation of a suite of indicators for monitoring agroecosystem status and trends on a reg...

Assessing Operation Purple: A Program Evaluation of a Summer Camp for Military Youth

DTIC Science & Technology

2012-01-01

However, it should be noted that the four FOCUS studies employed evaluation designs with no control or comparison group . Furthermore, despite...evaluation design was more robust than other military youth program evaluations that do not have a control or comparison group (Beardslee, Lester, et al...Operation Purple: A Program Evaluation of a Summer Camp for Military Youth camp group ); the remaining group formed the no-camp, or control , group . We

Training Evaluation as an Integral Component of Training for Performance.

ERIC Educational Resources Information Center

Lapp, H. J., Jr.

A training evaluation system should address four major areas: reaction, learning, behavior, and results. The training evaluation system at GPU Nuclear Corporation addresses each of these areas through practical approaches such as course and program evaluation. GPU's program evaluation instrument uses a Likert-type scale to assess task development,…

Higher Education Trends (1997-1999): Program Evaluation. ERIC-HE Trends.

ERIC Educational Resources Information Center

Kezar, Adrianna J.

The amount of literature on program evaluation decreased in 1996, continuing a trend begun in the late 1980s. One exception to this is the literature on assessment. Another frequent issue is the technique of evaluation. Many examples of research on evaluation are from international settings, where accountability and evaluation appear to be…

15 CFR 286.11 - Listings.

Code of Federal Regulations, 2010 CFR

2010-01-01

... CONFORMITY ASSESSMENT SYSTEM EVALUATION (NVCASE) PROGRAM § 286.11 Listings. (a) NIST maintains lists of all bodies holding current NIST program certificates, together with the assessment areas for which they are issued. (b) NIST also maintains lists of those qualified conformity assessment bodies that are currently...

15 CFR 286.11 - Listings.

Code of Federal Regulations, 2013 CFR

2013-01-01

... CONFORMITY ASSESSMENT SYSTEM EVALUATION (NVCASE) PROGRAM § 286.11 Listings. (a) NIST maintains lists of all bodies holding current NIST program certificates, together with the assessment areas for which they are issued. (b) NIST also maintains lists of those qualified conformity assessment bodies that are currently...

15 CFR 286.11 - Listings.

Code of Federal Regulations, 2014 CFR

2014-01-01

... CONFORMITY ASSESSMENT SYSTEM EVALUATION (NVCASE) PROGRAM § 286.11 Listings. (a) NIST maintains lists of all bodies holding current NIST program certificates, together with the assessment areas for which they are issued. (b) NIST also maintains lists of those qualified conformity assessment bodies that are currently...

15 CFR 286.11 - Listings.

Code of Federal Regulations, 2011 CFR

2011-01-01

... CONFORMITY ASSESSMENT SYSTEM EVALUATION (NVCASE) PROGRAM § 286.11 Listings. (a) NIST maintains lists of all bodies holding current NIST program certificates, together with the assessment areas for which they are issued. (b) NIST also maintains lists of those qualified conformity assessment bodies that are currently...

15 CFR 286.11 - Listings.

Code of Federal Regulations, 2012 CFR

2012-01-01

... CONFORMITY ASSESSMENT SYSTEM EVALUATION (NVCASE) PROGRAM § 286.11 Listings. (a) NIST maintains lists of all bodies holding current NIST program certificates, together with the assessment areas for which they are issued. (b) NIST also maintains lists of those qualified conformity assessment bodies that are currently...

Competency Assessment of Microbiology Medical Laboratory Technologists in Ontario, Canada

PubMed Central

Fleming, Christine Ann

2014-01-01

Accreditation in Ontario, Canada, requires that licensed clinical laboratories participate in external quality assessment (also known as proficiency testing) and perform competency evaluation of their staff. To assess the extent of ongoing competency assessment practices, the Quality Management Program—Laboratory Services (QMP-LS) Microbiology Committee surveyed all 112 licensed Ontario microbiology laboratories. The questionnaire consisted of a total of 21 questions that included yes/no, multiple-choice, and short-answer formats. Participants were asked to provide information about existing programs, the frequency of testing, what areas are evaluated, and how results are communicated to the staff. Of the 111 responding laboratories, 6 indicated they did not have a formal evaluation program since they perform only limited bacteriology testing. Of the remaining 105 respondents, 87% perform evaluations at least annually or every 2 years, and 61% include any test or task performed, whereas 16% and 10% focus only on problem areas and high-volume complex tasks, respectively. The most common methods of evaluation were review of external quality assessment (EQA) challenges, direct observation, and worksheet review. With the exception of one participant, all communicate results to staff, and most take remedial action to correct the deficiencies. Although most accredited laboratories have a program to assess the ongoing competency of their staff, the methods used are not standardized or consistently applied, indicating that there is room for improvement. The survey successfully highlighted potential areas for improvement and allowed the QMP-LS Microbiology Committee to provide guidance to Ontario laboratories for establishing or improving existing microbiology-specific competency assessment programs. PMID:24899030

International ITS Program Assessment

DOT National Transportation Integrated Search

1995-08-01

THIS PAPER ATTEMPTS TO MAKE AN UPDATED COMPARISON OF WORLDWIDE ACTIVITIES IN INTELLIGENT TRANSPORTATION SYSTEMS (ITS) AND TO RECOMMEND A STRATEGY FOR CONTINUING INTERNATIONAL PROGRAM ASSESSMENT FOR DISCUSSION AT THE BENEFITS, EVALUATION AND COSTS (BE...

Evaluating the Effect of Learning Style and Student Background on Self-Assessment Accuracy

ERIC Educational Resources Information Center

Alaoutinen, Satu

2012-01-01

This study evaluates a new taxonomy-based self-assessment scale and examines factors that affect assessment accuracy and course performance. The scale is based on Bloom's Revised Taxonomy and is evaluated by comparing students' self-assessment results with course performance in a programming course. Correlation has been used to reveal possible…

A Self-Assessment Procedure for Use in Evaluation Training

ERIC Educational Resources Information Center

Stufflebeam, Daniel L.; Wingate, Lori A.

2005-01-01

This article describes the Self-Assessment of Program Evaluation Expertise instrument and procedure developed to help participants assess their learning gains in a 3-week evaluation institute. Participants completed the instrument in a pre- and posttest format. To reduce both the threat of embarrassment from individual results and the temptation…

Examining the Effectiveness of a Learning Outcomes Assessment Program: A Four Frames Perspective

ERIC Educational Resources Information Center

Schoepp, Kevin; Tezcan-Unal, Burcu

2017-01-01

Assessment of learning outcomes at the program level is essential to evaluate whether students are achieving what is expected of them as graduates. In this article we present the results of a study in which faculty focus groups were consulted so as to understand the subjective issues that surround the learning outcomes assessment program of an…

Teaching and evaluation of ethics and professionalism

PubMed Central

Pauls, Merril A.

2012-01-01

Abstract Objective To document the scope of the teaching and evaluation of ethics and professionalism in Canadian family medicine postgraduate training programs, and to identify barriers to the teaching and evaluation of ethics and professionalism. Design A survey was developed in collaboration with the Committee on Ethics of the College of Family Physicians of Canada. The data are reported descriptively and in aggregate. Setting Canadian postgraduate family medicine training programs. Participants Between June and December of 2008, all 17 Canadian postgraduate family medicine training programs were invited to participate. Main outcome measures The first part of the survey explored the structure, resources, methods, scheduled hours, and barriers to teaching ethics and professionalism. The second section focused on end-of-rotation evaluations, other evaluation strategies, and barriers related to the evaluation of ethics and professionalism. Results Eighty-eight percent of programs completed the survey. Most respondents (87%) had learning objectives specifically for ethics and professionalism, and 87% had family doctors with training or interest in the area leading their efforts. Two-thirds of responding programs had less than 10 hours of scheduled instruction per year, and the most common barriers to effective teaching were the need for faculty development, competing learning needs, and lack of resident interest. Ninety-three percent of respondents assessed ethics and professionalism on their end-of-rotation evaluations, with 86% assessing specific domains. The most common barriers to evaluation were a lack of suitable tools and a lack of faculty comfort and interest. Conclusion By far most Canadian family medicine postgraduate training programs had learning objectives and designated faculty leads in ethics and professionalism, yet there was little curricular time dedicated to these areas and a perceived lack of resident interest and faculty expertise. Most programs evaluated ethics and professionalism as part of their end-of-rotation evaluations, but only a small number used novel means of evaluation, and most cited a lack of suitable assessment tools as an important barrier. PMID:23242906

Iowa CASAS Pilot Project Reports: An Initial Evaluation of CASAS Effectiveness in Iowa's Adult Basic Education Programs.

ERIC Educational Resources Information Center

Strom, Mary L.

In fall 1992, the Iowa Department of Education began pilot tests of the Comprehensive Adult Student Assessment System (CASAS), an assessment system evaluating reading, math, and problem solving in a life skills context for adult remedial programs. This document provides reports from the nine community colleges that served as test sites, describing…

Assessing Change in the Teaching Practice of Faculty in a Faculty Development Program for Primary Care Physicians: Toward a Mixed Method Evaluation Approach.

ERIC Educational Resources Information Center

Pinheiro, Sandro O.; Rohrer, Jonathan D.; Heimann, C. F. Larry

This paper describes a mixed method evaluation study that was developed to assess faculty teaching behavior change in a faculty development fellowship program for community-based hospital faculty. Principles of adult learning were taught to faculty participants over the fellowship period. These included instruction in teaching methods, group…

The Development of a Comprehensive Student Basic Skills Assessment Program at St. Louis Community College's Florissant Valley Campus.

ERIC Educational Resources Information Center

Webb, Mel

A comprehensive student basic skills assessment program was developed at St. Louis Community College (SLCC) at Florissant Valley to appraise student readiness to take courses, gather information for counseling and advising, diagnose student problems, and evaluate program efficiency and effectiveness. The steps taken in developing the program were:…

Evaluation readiness: improved evaluation planning using a data inventory framework.

PubMed

Cohen, A B; Hall, K C; Cohodes, D R

1985-01-01

Factors intrinsic to many programs, such as ambiguously stated objectives, inadequately defined performance measures, and incomplete or unreliable databases, often conspire to limit the evaluability of these programs. Current evaluation planning approaches are somewhat constrained in their ability to overcome these obstacles and to achieve full preparedness for evaluation. In this paper, the concept of evaluation readiness is introduced as a complement to other evaluation planning approaches, most notably that of evaluability assessment. The basic products of evaluation readiness--the formal program definition and the data inventory framework--are described, along with a guide for assuring more timely and appropriate evaluation response capability to support the decision making needs of program managers. The utility of evaluation readiness for program planning, as well as for effective management, is also discussed.

Evaluating Nutrition Education Programming by Using a Dietary Screener

ERIC Educational Resources Information Center

Schultz, Jennifer; Litchfield, Ruth

2016-01-01

Short dietary assessment instruments known as screeners have potential for use in evaluating nutrition education programming because detecting change in dietary intake can demonstrate movement toward program goals. Using screeners results in objective dietary intake data but involves less administrative time, training, and cost than other…

Automated Writing Evaluation Program's Effect on Student Writing Achievement

ERIC Educational Resources Information Center

Holman, Lester Donnie

2011-01-01

In an ex post facto causal-comparative research design, this study investigated the effectiveness of Automated Writing Evaluation (AWE) programs on raising the student writing achievement. Tennessee Comprehensive Assessment Program (TCAP) writing achievement scores from the 2010 administration were utilized for this study. The independent variable…

Space Tug systems study (storable). Volume 3: Executive summary

NASA Technical Reports Server (NTRS)

1974-01-01

Space tug program options that consider key issues and mission requirements are assessed, component and subsystem candidates are evaluated, and tug configurations synthesized. Three tug program options are defined and evaluated.

Demonstrating Support for the Formative and Summative Assessment Paradigm in a School-Based Intensive Mental Health Program

ERIC Educational Resources Information Center

Puddy, Richard W.; Boles, Richard E.; Dreyer, Meredith L.; Maikranz, Julie; Roberts, Michael C.; Vernberg, Eric M.

2008-01-01

We illustrate the use of formative and summative assessment in evaluating a therapeutic classroom program for children with serious emotional disturbances. Information was analyzed based on data gathered for clinical decision-making during treatment (formative assessment) and measurement of outcomes at discharge (summative assessment) from a…

Evolving from Quantity to Quality: A New Yardstick for Assessment

ERIC Educational Resources Information Center

Fulcher, Keston H.; Orem, Chris D.

2010-01-01

Higher education experts tout learning outcomes assessment as a vehicle for program improvement. To this end the authors share a rubric designed explicitly to evaluate the quality of assessment and how it leads to program improvement. The rubric contains six general assessment areas, which are further broken down into 14 elements. Embedded within…

Process and Outcome Evaluation of an Art Therapy Program for People Living with HIV/AIDS

ERIC Educational Resources Information Center

Feldman, Matthew B.; Betts, Donna J.; Blausey, Daniel

2014-01-01

Program evaluation offers an opportunity for improving the implementation and impact of art therapy. This article describes a process and outcomes evaluation of an art therapy program within the mental health services unit of a community-based organization for people living with HIV/AIDS. The aims were to assess utilization patterns and program…

Washington State Even Start 1993-1994: Final Evaluation. A Report to the Office of Adult Literacy.

ERIC Educational Resources Information Center

Iglitzin, Lynne; Wandschneider, Mary

All 18 Washington State Even Start sites participated in the program's evaluation. Site coordinators administered the assessment and evaluation measures to the adults served by the program and to teachers working with children at both entry and exit from the program. An indepth study was conducted of 134 families for whom there were complete sets…

Evaluation of a Commercially Available Program and in Situ Training by Parents to Teach Abduction-Prevention Skills to Children

ERIC Educational Resources Information Center

Beck, Kimberly V.; Miltenberger, Raymond G.

2009-01-01

Child abduction is a serious problem; therefore, it is essential that researchers evaluate the efficacy of commercially available abduction-prevention programs. A multiple baseline design across participants (ages 6 to 8 years) was used to evaluate the effects of a training program, The Safe Side. Experimenters assessed safety responses in situ in…

[Evaluation of a program to promote network building between disciplinary agencies and informal community organizations: trial in a community comprehensive support center].

PubMed

Murayama, Hiroshi; Kojima, Tomoko; Tomaru, Meiko; Narabu, Harumi; Tachibana, Reiko; Yamaguchi, Takuhiro; Murashima, Sachiyo

2010-10-01

To examine the effectiveness of a program promoting network building between disciplinary agencies and informal community organizations (IGOs) comprising community residents, by implemention with staff of a community comprehensive support center (CJCSG). The program was implemented for nine staff of a GGSG in Setagaya Ward, Tokyo for a year. For process evaluation, items were assessed concerning the contents of the program such as satisfaction and understandability, participants' goal attainment level at each period of the program, and program satisfaction as a whole. Outcome evaluation included measurement of participants' self-efficacy regarding network building with ICOs before and after the program, using interviews of the members who completed the program. Eight out of the nine participants completed the program. All positively evaluated the contents of the program and their own goal attainment at each period of the program. After its completion, they felt highly satisfied. Moreover, there was an improvement in the cognition of the participants, including self-efficacy on network building with IGOs and the atmosphere in the GGSG with regard to network building. The efficacy of this program could be confirmed as demonstrated by the staff of the CCSC, although a more detailed assessment of validity and effectiveness will be necessary in the future.

Qualitative Analysis of Written Reflections during a Teaching Certificate Program

PubMed Central

Castleberry, Ashley N.; Payakachat, Nalin; Ashby, Sarah; Nolen, Amanda; Carle, Martha; Neill, Kathryn K.

2016-01-01

Objective. To evaluate the success of a teaching certificate program by qualitatively evaluating the content and extent of participants’ reflections. Methods. Two investigators independently identified themes within midpoint and final reflection essays across six program years. Each essay was evaluated to determine the extent of reflection in prompted teaching-related topic areas (strengths, weaknesses, assessment, feedback). Results. Twenty-eight themes were identified within 132 essays. Common themes encompassed content delivery, student assessment, personal successes, and challenges encountered. Deep reflection was exhibited, with 48% of essays achieving the highest level of critical reflection. Extent of reflection trended higher from midpoint to final essays, with significant increases in the strengths and feedback areas. Conclusion. The teaching certificate program fostered critical reflection and self-reported positive behavior change in teaching, thus providing a high-quality professional development opportunity. Such programs should strongly consider emphasizing critical reflection through required reflective exercises at multiple points within program curricula. PMID:26941436

An evaluation of the nursing success program: reading comprehension, graduation rates, and diversity.

PubMed

Symes, Lene; Tart, Kathryn; Travis, Lucille

2005-01-01

The Nursing Success Program was developed to enhance retention of baccalaureate nursing students. Reading comprehension scores are used to identify students who are at risk for failure and direct them into the retention program that addresses their skill deficits. To evaluate the program, the authors assessed reading comprehension, graduation rates, and ethnic diversity.

Results from a Multi-Modal Program Evaluation of a Four Year Statewide Juvenile Sex Offender Treatment and Reentry Program

ERIC Educational Resources Information Center

Underwood, Lee A.; Dailey, Frances L. L.; Merino, Carrie; Crump, Yolanda

2015-01-01

The results of the Program Evaluation show the OJJ Statewide Sex Offender Treatment program is exceptionally productive in meeting over 90% of its established performance markers. These markers included successful screening and assessment of risk and psychosocial needs, completion of initial and master treatment plans, establishment of sex…

Gendered Citizenship and the Individualization of Environmental Responsibility: Evaluating a Campus Common Reading Program

ERIC Educational Resources Information Center

Kennedy, Emily Huddart; Boyd, Amanda

2018-01-01

Campus common reading programs are intended to stimulate critical thinking and dialogue across disciplines yet scarce evidence exists to evaluate the success of such programs. We assess the extent to which engagement in an environmentally-themed common reading program is related to (1) concern for waste-related issues, (2) beliefs that addressing…

An Evaluative Case Study of Nine Virtual High School Programs

ERIC Educational Resources Information Center

Bannister, Clara M.

2009-01-01

The purpose of this study was to conduct a program evaluation of nine virtual high school programs in the states of Alabama, California, Florida, Idaho, Colorado, Illinois, Michigan, North Carolina and Massachusetts. In order to assess the quality and effectiveness of the virtual high school programs in the study in terms of overall quality and…

Welding and Fabricating Technology Program Needs Assessment.

ERIC Educational Resources Information Center

Oakland Community Coll., Farmington, MI. Office of Institutional Planning and Analysis.

In 1992, Oakland Community College (OCC) conducted a needs assessment study to assist in reviewing and evaluating proposed changes to the college's existing Welding and Fabricating Program. A literature review was undertaken, examining industry forecasts, related programs at other institutions of higher education, and data supplied by the U.S.…

Food Sanitation and Safety Self-Assessment Instrument for School Nutrition Programs.

ERIC Educational Resources Information Center

California State Dept. of Education, Sacramento.

Like food-service establishments, child nutrition programs are responsible for preserving the quality and wholesomeness of food. Proper food-handling practices prevent contamination and job-related accidents. Application of the evaluation instrument presented in this document to individual programs helps to define proper practices, assess the…

The Impact of the Measures of Academic Progress (MAP) Program on Student Reading Achievement

ERIC Educational Resources Information Center

Cordray, David S.; Pion, Georgine M.; Brandt, Chris; Molefe, Ayrin

2013-01-01

One of the most widely used commercially available systems incorporating benchmark assessment and training in differentiated instruction is the Northwest Evaluation Association's (NWEA) Measures of Academic Progress (MAP) program. The MAP program involves two components: (1) computer-adaptive assessments administered to students three to four…

An Evaluation of TCITY: The Twin City Institute for Talented Youth. Report #1 in Evaluation Report Series.

ERIC Educational Resources Information Center

Stake, Robert E.; Gjerde, Craig

This evaluation of the Twin City Institute for Talented Youth, a summer program for gifted students in grades 9 through 12, consists of two parts: a description of the program; and the evaluators' assessments, including advocate and adversary reports. Achievement tests were not used for evaluation. Evaluative comments follow each segment of the…

Evaluation of an external quality assessment program for HIV testing in Haiti, 2006-2011.

PubMed

Louis, Frantz Jean; Anselme, Renette; Ndongmo, Clement; Buteau, Josiane; Boncy, Jacques; Dahourou, Georges; Vertefeuille, John; Marston, Barbara; Balajee, S Arunmozhi

2013-12-01

To evaluate an external quality assessment (EQA) program for human immunodeficiency virus (HIV) rapid diagnostics testing by the Haitian National Public Health Laboratory (French acronym: LNSP). Acceptable performance was defined as any proficiency testing (PT) score more than 80%. The PT database was reviewed and analyzed to assess the testing performance of the participating laboratories and the impact of the program over time. A total of 242 laboratories participated in the EQA program from 2006 through 2011; participation increased from 70 laboratories in 2006 to 159 in 2011. In 2006, 49 (70%) laboratories had a PT score of 80% or above; by 2011, 145 (97.5%) laboratories were proficient (P < .05). The EQA program for HIV testing ensures quality of testing and allowed the LNSP to document improvements in the quality of HIV rapid testing over time.

What Matters for Excellence in PhD Programs? Latent Constructs of Doctoral Program Quality Used by Early Career Social Scientists

ERIC Educational Resources Information Center

Morrison, Emory; Rudd, Elizabeth; Zumeta, William; Nerad, Maresi

2011-01-01

This paper unpacks how social science doctorate-holders come to evaluate overall excellence in their PhD training programs based on their domain-specific assessments of aspects of their programs. Latent class analysis reveals that social scientists 6-10 years beyond their PhD evaluate the quality of their doctoral program with one of two…

Notification: Review of the Contract Management Assessment Program

EPA Pesticide Factsheets

Project #OA-FY14-0034, October 30, 2013. The Office of Inspector General plans to begin the preliminary research phase of an audit evaluating the Office of Acquisition Management’s Contract Management Assessment Program (CMAP).

Superfund Site Assessment Process

EPA Pesticide Factsheets

Learn about the site assessment process used by the federal Superfund program to evaluate releases of hazardous substances that may pose a threat to human health or the environment and select an appropriate program for sites needing cleanup.

Final Report on the Multicultural/Diversity Assessment Project.

ERIC Educational Resources Information Center

Ambrosio, Anthony L.

The Emporia State University Multicultural/Diversity Project developed a set of assessment instruments and a model evaluation plan to assess multicultural/diversity (MCD) outcomes in teacher education and general education programs. Assessment instruments and techniques were constructed to evaluate the impact of coursework on student attitudes,…

Assessing Growth of Student Reasoning Skills in Honors

ERIC Educational Resources Information Center

Wood-Nartker, Jeanneane; Hinck, Shelly; Hullender, Ren

2016-01-01

Assessment and evaluation practices within honors programs have attracted considerable attention within the honors academic community, e.g., the spring/summer 2006 volume of the "Journal of the National Collegiate Honors Council." Calls for carefully created and constructed assessment activities within honors programs have met with mixed…

Evaluating participation in water resource management: A review

NASA Astrophysics Data System (ADS)

Carr, G.; BlöSchl, G.; Loucks, D. P.

2012-11-01

Key documents such as the European Water Framework Directive and the U.S. Clean Water Act state that public and stakeholder participation in water resource management is required. Participation aims to enhance resource management and involve individuals and groups in a democratic way. Evaluation of participatory programs and projects is necessary to assess whether these objectives are being achieved and to identify how participatory programs and projects can be improved. The different methods of evaluation can be classified into three groups: (i) process evaluation assesses the quality of participation process, for example, whether it is legitimate and promotes equal power between participants, (ii) intermediary outcome evaluation assesses the achievement of mainly nontangible outcomes, such as trust and communication, as well as short- to medium-term tangible outcomes, such as agreements and institutional change, and (iii) resource management outcome evaluation assesses the achievement of changes in resource management, such as water quality improvements. Process evaluation forms a major component of the literature but can rarely indicate whether a participation program improves water resource management. Resource management outcome evaluation is challenging because resource changes often emerge beyond the typical period covered by the evaluation and because changes cannot always be clearly related to participation activities. Intermediary outcome evaluation has been given less attention than process evaluation but can identify some real achievements and side benefits that emerge through participation. This review suggests that intermediary outcome evaluation should play a more important role in evaluating participation in water resource management.

Participatory Evaluation and Learning: A Case Example Involving Ripple Effects Mapping of a Tourism Assessment Program

ERIC Educational Resources Information Center

Bhattacharyya, Rani; Templin, Elizabeth; Messer, Cynthia; Chazdon, Scott

2017-01-01

Engaging communities through research-based participatory evaluation and learning methods can be rewarding for both a community and Extension. A case study of a community tourism development program evaluation shows how participatory evaluation and learning can be mutually reinforcing activities. Many communities value the opportunity to reflect…

Students Training for Academic Readiness (STAR): Year Three Evaluation Report

ERIC Educational Resources Information Center

Rainey, Katharine; Sheehan, Daniel; Maloney, Catherine

2010-01-01

This report presents findings from the Year 3 evaluation of Texas' state-level Gaining Early Awareness and Readiness for Undergraduate Programs, or GEAR UP, grant. GEAR UP grant requirements include an evaluation component designed to assess program effectiveness and to measure progress toward project goals. To this end, the evaluation considers…

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2013 CFR

2013-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2014 CFR

2014-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2012 CFR

2012-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2011 CFR

2011-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

Evaluation of a Sandplay Program for Preschoolers in a Multiethnic Neighborhood

ERIC Educational Resources Information Center

Rousseau, Cecile; Benoit, Maryse; Lacroix, Louise; Gauthier, Marie-France

2009-01-01

Background: This evaluative study assesses the efficacy of a school-based secondary prevention program consisting of creative expression workshops for immigrant and refugee preschoolers in a predominantly South Asian multiethnic neighborhood. Coincidentally, the program began in the wake of the tsunami. Method: Pretest and posttest data were…

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2010 CFR

2010-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... the reading ability of students in a program over time to a similar group of students not... example, a performance measure for a literacy program may include the percentage of students receiving...

Evaluating a Nutrition Education Program for Pregnant Teen-Agers: Cognitive vs. Behavioral Outcomes.

ERIC Educational Resources Information Center

Perkin, Judy

1983-01-01

A manual on nutrition during pregnancy and lactation was developed and used with pregnant teenagers. Evaluation of the program showed that, while participants' knowledge of nutrition improved, their eating habits did not. The need for behavioral assessment of nutrition education programs is pointed out. (Author/PP)

The Economic Cost of Substance Abuse Treatment in the State of Florida

ERIC Educational Resources Information Center

Alexandre, Pierre K.; Beulaygue, Isabelle C.; French, Michael T.; McCollister, Kathryn E.; Popovici, Ioana; Sayed, Bisma A.

2012-01-01

Objective: Public and private stakeholders of substance abuse treatment services require economic cost data to guide program evaluations and funding decisions. Background: Rigorous cost assessments have been conducted for several treatment programs across the United States, but a systematic and comprehensive evaluation of programs in a particular…

Career Development in Language Education Programs

ERIC Educational Resources Information Center

Shawer, Saad Fathy; Alkahtani, Saad Ali

2013-01-01

This study assesses the influence of a two-year language program evaluation on program directors and faculty career development. The study makes use of mixed-paradigms (positivism and qualitative interpretive), mixed-strategies (survey research and qualitative evaluation), one-way analysis of variance (ANOVA) and a post-hoc test of multiple…

Evaluating a Development Program.

ERIC Educational Resources Information Center

Frantzreb, Arthur C.

Problems in evaluating the success of an institutional resource development program are examined and suggestions are offered for effective assessment. Such a program is seen as a sequence of events in the artful management of people, from the planning stage to obtaining a financial commitment in writing from persons who want to share their assets…

Qualitative and Semiquantitative Assessment of Exposure to Engineered Nanomaterials within the French EpiNano Program: Inter- and Intramethod Reliability Study.

PubMed

Guseva Canu, Irina; Jezewski-Serra, Delphine; Delabre, Laurène; Ducamp, Stéphane; Iwatsubo, Yuriko; Audignon-Durand, Sabine; Ducros, Cécile; Radauceanu, Anca; Durand, Catherine; Witschger, Olivier; Flahaut, Emmanuel

2017-01-01

The relatively recent development of industries working with nanomaterials has created challenges for exposure assessment. In this article, we propose a relatively simple approach to assessing nanomaterial exposures for the purposes of epidemiological studies of workers in these industries. This method consists of an onsite industrial hygiene visit of facilities carried out individually and a description of workstations where nano-objects and their agglomerates and aggregates (NOAA) are present using a standardized tool, the Onsite technical logbook. To assess its reliability, we implemented this approach for assessing exposure to NOAA in workplaces at seven workstations which synthesize and functionalize carbon nanotubes. The prediction of exposure to NOAA using this method exhibited substantial agreement with that of the reference method, the latter being based on an onsite group visit, an expert's report and exposure measurements (Cohen kappa = 0.70, sensitivity = 0.88, specificity = 0.92). Intramethod comparison of results for exposure prediction showed moderate agreement between the three evaluators (two program team evaluators and one external evaluator) (weighted Fleiss kappa = 0.60, P = 0.003). Interevaluator reliability of the semiquantitative exposure characterization results was excellent between the two evaluators from the program team (Spearman rho = 0.93, P = 0.03) and fair when these two evaluators' results were compared with the external evaluator's results. The project was undertaken within the framework of the French epidemiological surveillance program EpiNano. This study allowed a first reliability assessment of the EpiNano method. However, to further validate this method a comparison with robust quantitative exposure measurement data is necessary. © The Author 2017. Published by Oxford University Press on behalf of the British Occupational Hygiene Society.

Taxonomy for Assessing Evaluation Competencies in Extension

ERIC Educational Resources Information Center

Rodgers, Michelle S.; Hillaker, Barbara D.; Haas, Bruce E.; Peters, Cheryl

2012-01-01

Evaluation of public service programming is becoming increasingly important with current funding realities. The taxonomy of evaluation competencies compiled by Ghere et al. (2006) provided the starting place for Taxonomy for Assessing Evaluation Competencies in Extension. The Michigan State University Extension case study described here presents a…

The Internet as a communication tool for orthopedic spine fellowships in the United States.

PubMed

Silvestre, Jason; Guzman, Javier Z; Skovrlj, Branko; Overley, Samuel C; Cho, Samuel K; Qureshi, Sheeraz A; Hecht, Andrew C

2015-04-01

Orthopedic residents seeking additional training in spine surgery commonly use the Internet to manage their fellowship applications. Although studies have assessed the accessibility and content of Web sites in other medical specialties, none have looked at orthopedic spine fellowship Web sites (SFWs). The purpose of this study was to evaluate the accessibility of information from commonly used databases and assess the content of SFWs. This was a Web site accessibility and content evaluation study. A comprehensive list of available orthopedic spine fellowship programs was compiled by accessing program lists from the SF Match, North American Spine Society, Fellowship and Residency Electronic Interactive Database (FREIDA), and Orthopaedicsone.com (Ortho1). These databases were assessed for accessibility of information including viable links to SFWs and responsive program contacts. A Google search was used to identify SFWs not readily available on these national databases. SFWs were evaluated based on online education and recruitment content. Evaluators found 45 SFWs of 63 active programs (71%). Available SFWs were often not readily accessible from national program lists, and no program afforded a direct link to their SFW from SF Match. Approximately half of all programs responded via e-mail. Although many programs described surgical experience (91%) and research requirements (87%) during the fellowship, less than half mentioned didactic instruction (46%), journal clubs (41%), and national meetings or courses attended (28%). Evaluators found an average 45% of fellow recruitment content. Comparison of SFWs by program characteristics revealed three significant differences. Programs with greater than one fellowship position had greater online education content than programs with a single fellow (p=.022). Spine fellowships affiliated with an orthopedic residency program maintained greater education (p=.006) and recruitment (p=.046) content on their SFWs. Most orthopedic spine surgery programs underuse the Internet for fellow education and recruitment. The inaccessibility of information and paucity of content on SFWs allow for future opportunity to optimize these resources. Copyright © 2015 Elsevier Inc. All rights reserved.

Impact of a provider training program on the treatment of children with autism spectrum disorder at psychosocial care units in Brazil.

PubMed

Silva, Luciana C; Teixeira, Maria C T V; Ribeiro, Edith L; Paula, Cristiane S

2017-12-18

To develop, implement, and verify the impact of a training program for health care providers working with children with autism spectrum disorder (ASD) in psychosocial care centers for children and adolescents (Centro de Atenção Psicossocial à Infância e à Adolescência - CAPSi) in São Paulo, Brazil. This quasi-experimental study was conducted with 14 professionals from four CAPSi units. The training program consisted of six phases: 1) pre-intervention observation; 2) meeting with staff to assess the main needs of the training program; 3) developing materials for training and evaluation; 4) meetings to discuss program implementation; 5) a final meeting for case discussion and evaluation; and 6) distance supervision. Three measures were used to evaluate the training program: i) the Knowledge, Attitudes, and Practices (KAP) questionnaire; ii) videos containing questions designed to assess program comprehension; and iii) a satisfaction survey. Thirteen videos were produced to as visual aids for use during the training program, and a further 26 videos were developed to evaluate it. The program was well evaluated by the participants. The video responses and KAP questionnaire scores suggest that staff knowledge and attitudes improved after training. The positive findings of this study suggest that the tested training program is feasible for use with multidisciplinary teams working in the CAPSi environment.

Evaluation Methods Used during the Assessment of an Academic Program at Micro-Level

ERIC Educational Resources Information Center

Chinembiri, France

2017-01-01

This paper majors on the evaluation criteria and methods used for the assessment of health education programs in the United States. The choice of the topic is dictated by the fact that there is need to improve on the quality of the graduates that the different nursing and medical institutions in the United States produce. By looking at the…

NURSING 911: an orientation program to improve retention of online RN-BSN students.

PubMed

Gilmore, Melanie; Lyons, Evadna M

2012-01-01

This article describes the implementation and evaluation of an eight-hour, comprehensive, face-to-face orientation program designed to improve student retention in a newly developed online RN to BSN program. A total of 179 newly enrolled RN to BSN students participated in the orientation program and evaluated the process. Student attrition decreased from 20 percent to less than 1 percent after the orientation program was extended and improved to include a technology assessment and an online practice course. A quality online program requires a well-designed orientation that includes technological assessments and hands-on, active participation by the learner. The newly improved and designed course has become effective in student retention and transition into the online learning environment.

Professional Development through Organizational Assessment: Using APPA's Facilities Management Evaluation Program

ERIC Educational Resources Information Center

Medlin, E. Lander; Judd, R. Holly

2013-01-01

APPA's Facilities Management Evaluation Program (FMEP) provides an integrated system to optimize organizational performance. The criteria for evaluation not only provide a tool for organizational continuous improvement, they serve as a compelling leadership development tool essential for today's facilities management professional. The senior…

Assessment Tool Development for Extracurricular Smet Programs for Girls

NASA Astrophysics Data System (ADS)

House, Jody; Johnson, Molly; Borthwick, Geoffrey

Many different programs have been designed to increase girls' interest in and exposure to science, mathematics, engineering, and technology (SMET). Two of these programs are discussed and contrasted in the dimensions of length, level of science content, pedagogical approach, degree of self- vs. parent-selected participants, and amount of communitybuilding content. Two different evaluation tools were used. For one program, a modified version of the University of Pittsburgh's undergraduate engineering attitude assessment survey was used. Program participants' responses were compared to those from a fifth grade, mixed-sex science class. The only gender difference found was in the area of parental encouragement. The girls in the special class were more encouraged to participate in SMET areas. For the second program, a new age-appropriate tool developed specifically for these types of programs was used, and the tool itself was evaluated. The results indicate that the new tool has construct validity. On the basis of these preliminary results, a long-term plan for the continued development of the assessment tool is outlined.

The Community Research Scholars Initiative: A Mid‐Project Assessment

PubMed Central

Pike, Earl; Sehgal, Ashwini R.; Fischer, Robert L.; Collins, Cyleste

2015-01-01

Abstract Community organizations addressing health and human service needs generally have minimal capacity for research and evaluation. As a result, they are often inadequately equipped to independently carry out activities that can be critical for their own success, such as conducting needs assessments, identifying best practices, and evaluating outcomes. Moreover, they are unable to develop equitable partnerships with academic researchers to conduct community‐based research. This paper reports on the progress of the Community Research Scholar Initiative (CRSI), a program that aims to enhance community research and evaluation capacity through training of selected employees from Greater Cleveland community organizations. The intensive 2‐year CRSI program includes didactic instruction, fieldwork, multiple levels of community and academic engagement, leadership training, and a mentored research project. The first cohort of CRSI Scholars, their community organizations, and other community stakeholders have incorporated program lessons into their practices and operations. The CRSI program evaluation indicates: the importance of careful Scholar selection; the need to engage executive leadership from Scholar organizations; the value of a curriculum integrating classwork, fieldwork, and community engagement; and the need for continual scholar skill and knowledge assessment. These findings and lessons learned guide other efforts to enhance community organization research and evaluation capacity. PMID:26073663

[Process evaluation in relation to effectiveness assessment: experiences with school-based programs].

PubMed

Ariza, Carles; Villalbí, Joan R; Sánchez-Martínez, Francesca; Nebot, Manel

2011-06-01

Evaluation of public health interventions usually focus on the quality of design and research methods, and less on the quality of the intervention or process evaluation. In process evaluation of school-based interventions, key issues are how completely the intervention is carried out and adherence to the protocol. In addition, exploration of intermediate variables, such as those that influence (and often predict) preventable behavior, is highly useful. This article describes the basic concepts in this topic, using examples of the effectiveness of some preventive interventions carried out in schools. The interventions discussed were mainly quasi-experimental studies, based on data from programs promoted by public health teams in the city of Barcelona. Data from process evaluation of preventive programs in secondary schools that underwent formal assessment of their effectiveness is provided. The examples are drawn from preventive programs of HIV infection or unprotected sexual intercourse (PRESSEC program) and drug consumption prevention (the PASE, PASE.bcn and x kpts programs). These examples show why the intervention process influences the impact of the programs and their results. Thorough planning of process evaluation is essential to obtain valid indicators that will identify, in the effectiveness evaluation of the intervention, the most efficacious strategies to obtain positive outcomes. Copyright © 2011 Sociedad Española de Salud Pública y Administración Sanitaria. Published by Elsevier Espana. All rights reserved.

Survey of university programs in remote sensing funded under grants from the NASA University-Space Applications program

NASA Technical Reports Server (NTRS)

Madigan, J. A.; Earhart, R. W.

1978-01-01

NASA's Office of Space and Terrestrial Applications (OSTA) is currently assessing approaches to transferring NASA technology to both the public and private sectors. As part of this assessment, NASA is evaluating the effectiveness of an ongoing program in remote sensing technology transfer conducted by 20 university contractors/grantees, each supported totally or partially by NASA funds. The University-Space Applications program has as its objective the demonstration of practical benefits from the use of remote sensing technology to a broad spectrum of new users, principally in state and local governments. To evaluate the University-Space Applications program, NASA has a near-term requirement for data on each university effort including total funding, funding sources, length of program, program description, and effectiveness measures.

Evaluation of Current Assessment Methods in Engineering Entrepreneurship Education

ERIC Educational Resources Information Center

Purzer, Senay; Fila, Nicholas; Nataraja, Kavin

2016-01-01

Quality assessment is an essential component of education that allows educators to support student learning and improve educational programs. The purpose of this study is to evaluate the current state of assessment in engineering entrepreneurship education. We identified 52 assessment instruments covered in 29 journal articles and conference…

Evaluation Report of the Fall River Middle School Research and Development Center Title VII Program, 1973-1974.

ERIC Educational Resources Information Center

Heuristics, Inc., Dedham, MA.

This report presents the evaluation of the 1973-1974 Fall River Middle School Research and Development Center, a project funded by ESEA Title VII as a model demonstration bilingual program beginning in 1972-1973. The evaluation focused on the assessment of the degree of accomplishment of product objectives for each component of the program. The…

A Supplement to the Component Four Implementation Booklet. Broad Goal Definition in Emerald School District. SPECS: School Planning, Evaluation, and Communication System.

ERIC Educational Resources Information Center

Nagle, John M.; Walker, Harold E.

The School Planning, Evaluation, and Communication System (SPECS) is an overall design for districtwide program assessment and development. It is both a way of interrelating planning and evaluation of individual programs and a way of comparing the outcomes of all district programs with the district's community-defined educational goals. This…

A Functional Model for Management of Large Scale Assessments.

ERIC Educational Resources Information Center

Banta, Trudy W.; And Others

This functional model for managing large-scale program evaluations was developed and validated in connection with the assessment of Tennessee's Nutrition Education and Training Program. Management of such a large-scale assessment requires the development of a structure for the organization; distribution and recovery of large quantities of…

Assessment Program Technical Progress Report, 1998-1999.

ERIC Educational Resources Information Center

Eickmeyer, Barbara; Hill, Stephen; Kozak, Kathryn

This report records the assessment and institutional research activities at Coconino Community College during the 1998-1999 academic year. These activities are part of an effort to evaluate institutional effectiveness with respect to college mission, programs, and instruction. It focuses on past and present assessment activities and future goals,…

Sustaining Assessment: A Post-Epidemiological Approach Using "The Program Evaluation Standards"

ERIC Educational Resources Information Center

Axelson, Rick D.; Flick, Arend

2009-01-01

At a recent assessment workshop, one of the authors' colleagues, Andrew Stuart Bergerson, offered a novel metaphor for building their university's assessment program. "We need to spread the assessment virus on campus," Drew remarked, half seriously. Although they had a good laugh over this image at the time, it encapsulates the faculty-driven…

Assessing Organizational Readiness for a Participatory Occupational Health/Health Promotion Intervention in Skilled Nursing Facilities

PubMed Central

Zhang, Yuan; Flum, Marian; West, Cheryl; Punnett, Laura

2018-01-01

The long-term care sector is characterized by high morbidity and employee turnover, along with associated costs. Effective health protection and health promotion are important to improve physical and psychosocial well-being of caregivers. Assessment of organizational readiness for change is an essential precursor to the successful implementation of workplace programs addressing work climate, structure of tasks and relationships, and other issues that may be perceived as challenging by some within the institution. This study qualitatively assessed readiness of five skilled nursing facilities for a participatory occupational health/health promotion intervention. Selection criteria were developed to screen for program feasibility and ability to conduct prospective evaluations, and information was collected from managers and employees (interviews and focus groups). Three centers were selected for the program, and the first year of formative evaluation and intervention experience was then reviewed to evaluate and modify our selection criteria after the fact. Lessons learned include adding assessment of communication and the structure of problem solving to our selection criteria, improving methods to assess management support in a concrete (potentially nonverbal) form, and obtaining a stated financial commitment and resources to enable the team to function. Assessment of organizational readiness for change is challenging, although necessary to implement effective and sustainable health promotion programs in specific organizations. PMID:25715335

Getting Ready for College: An Implementation and Early Impacts Study of Eight Texas Developmental Summer Bridge Programs. NCPR Brief

ERIC Educational Resources Information Center

Wathington, Heather D.; Barnett, Elisabeth A.; Weissman, Evan; Teres, Jedediah; Pretlow, Joshua; Nakanishi, Aki

2011-01-01

In 2009, the National Center for Postsecondary Research (NCPR) launched an evaluation of eight developmental summer bridge programs in Texas to assess whether these programs reduce the need for developmental coursework and improve student outcomes in college. The evaluation uses an experimental design to measure the effects of these programs on…

Hanford internal dosimetry program manual

DOE Office of Scientific and Technical Information (OSTI.GOV)

Carbaugh, E.H.; Sula, M.J.; Bihl, D.E.

1989-10-01

This document describes the Hanford Internal Dosimetry program. Program Services include administrating the bioassay monitoring program, evaluating and documenting assessments of internal exposure and dose, ensuring that analytical laboratories conform to requirements, selecting and applying appropriate models and procedures for evaluating internal radionuclide deposition and the resulting dose, and technically guiding and supporting Hanford contractors in matters regarding internal dosimetry. 13 refs., 16 figs., 42 tabs.

The influence of financial incentive programs in promoting sustainable forestry on the nation's family forests

Treesearch

Michael A. Kilgore; John L. Greene; Michael G. Jacobson; Thomas J. Straka; Steven E. Daniels

2007-01-01

Financial incentive programs were evaluated to assess their contribution to promoting sustainable forestry practices on the nation’s family forests. The evaluation consisted of an extensive review of the literature on financial incentive programs, a mail survey of the lead administrator of financial incentive programs in each state forestry agency, and focus groups...

The influence of financial incentive programs in promoting sustainable forestry on the nation's family forests

Treesearch

Michael A. Kilgore; John L. Greene; Michael G. Jacobson; Thomas J. Straka; Steven E. Daniels

2006-01-01

Financial incentive programs were evaluated to assess their contribution to promoting sustainable forestry practices on the nation’s family forests. The evaluation consisted of an extensive review of the literature on financial incentive programs, a mail survey of the lead administrator of financial incentive programs in each state forestry agency, and focus groups...

Getting Teachers in on the Act: Evaluation of a Theater- and Classroom-Based Youth Violence Prevention Program

ERIC Educational Resources Information Center

Zucker, Marla; Spinazzola, Joseph; Pollack, Amie Alley; Pepe, Lauren; Barry, Stephanie; Zhang, Lynda; van der Kolk, Bessel

2010-01-01

This study replicated and extended our previous evaluation of Urban Improv (UI), a theater-based youth violence prevention (YVP) program developed for urban youth. It assessed the replicability of positive program impacts when implemented by nonprogram originators, as well as the utility of a comprehensive version of the UI program that included a…

The National Cross-Site Evaluation of High-Risk Youth Programs. Preventing Substance Abuse: Major Findings from the National Cross-Site Evaluation of High-Risk Youth Programs. Monograph Series.

ERIC Educational Resources Information Center

Springer, J. Fred; Sambrano, Soledad; Sale, Elizabeth; Kasim, Rafa; Herman, Jack

This multiple-site study assessed 48 prevention programs for high-risk youth funded by the Center for Substance Abuse Prevention, identifying program characteristics associated with strong substance abuse prevention outcomes. Data analysis indicated that substance abuse programs reduced rates of substance use, and the positive effects of program…

Evaluation of Project P.A.T.H.S. (Secondary 1 Program) by the Program Participants: Findings Based on the Full Implementation Phase

ERIC Educational Resources Information Center

Shek, Daniel T. L.; Sun, Rachel C. F.

2008-01-01

A total of 207 schools (N = 33,693 students) participated in the Secondary 1 Program of Project P.A.T.H.S. in the Full Implementation Phase (2006-07). Participants responded to a Subjective Outcome Evaluation Form (Form A) to assess their views of the program, instructors, and perceived effectiveness after program completion. Utilizing the…

Pre- and posttest evaluation of a breast cancer risk assessment program for nurse practitioners.

PubMed

Edwards, Quannetta T; Seibert, Diane

2010-07-01

Numerous studies have shown that healthcare providers, including nurse practitioners (NPs) fail to provide breast cancer risk assessment (BrCRA) in primary care settings. A potential barrier to the use of BrCRA is insufficient knowledge or training of risk assessment. The purpose of this study was to analyze the outcome of a BrCRA program developed to enhance NPs' knowledge of risk assessment and use of empiric risk assessment models. Thirty-five NPs participated in a before-after (pretest-posttest design) study evaluating the effectiveness of a BrCRA education program conducted at a national NP conference. Demographics, pre/post knowledge, and course satisfaction measures were all examined as a part of this pilot study. Continuing education through the implementation of a BrCRA program significantly increased NPs knowledge in assessing breast cancer risk and the use of empiric risk assessment models. Many healthcare providers, including NPs, are inadequately prepared to assess a woman's risk for breast cancer. Understanding breast cancer risk assessment is essential if NPs are to provide appropriate counseling, management, and referral strategies needed to reduce a woman's risk for developing the disease. Continuing education provides one means to enhance NP's knowledge of BrCRA.

Milestones: Critical Elements in Clinical Informatics Fellowship Programs

PubMed Central

Lehmann, Christoph U.; Munger, Benson

2016-01-01

Summary Background Milestones refer to points along a continuum of a competency from novice to expert. Resident and fellow assessment and program evaluation processes adopted by the ACGME include the mandate that programs report the educational progress of residents and fellows twice annually utilizing Milestones developed by a specialty specific ACGME working group of experts. Milestones in clinical training programs are largely unmapped to specific assessment tools. Residents and fellows are mainly assessed using locally derived assessment instruments. These assessments are then reviewed by the Clinical Competency Committee which assigns and reports trainee ratings using the specialty specific reporting Milestones. Methods and Results The challenge and opportunity facing the nascent specialty of Clinical Informatics is how to optimally utilize this framework across a growing number of accredited fellowships. The authors review how a mapped milestone framework, in which each required sub-competency is mapped to a single milestone assessment grid, can enable the use of milestones for multiple uses including individualized learning plans, fellow assessments, and program evaluation. Furthermore, such a mapped strategy will foster the ability to compare fellow progress within and between Clinical Informatics Fellowships in a structured and reliable fashion. Clinical Informatics currently has far less variability across programs and thus could easily utilize a more tightly defined set of milestones with a clear mapping to sub-competencies. This approach would enable greater standardization of assessment instruments and processes across programs while allowing for variability in how those sub-competencies are taught. Conclusions A mapped strategy for Milestones offers significant advantages for Clinical Informatics programs. PMID:27081414

Comprehensive Educational Assessment for the States: The Duplex Design.

ERIC Educational Resources Information Center

Bock, R. Darrell; Mislevy, Robert

State testing programs often attempt to provide annual information for use in student guidance and qualification, school and program evaluation, and broad policy decisions. With the development of a new type of assessment instrument, the "duplex design," the several functions of state testing programs can be served in a single test…

Outcome Assessment of the Visiting Fulbright Scholar Program

ERIC Educational Resources Information Center

US Department of State, 2005

2005-01-01

The Office of Policy and Evaluation of the Bureau of Educational and Cultural Affairs of the U.S. Department of State contracted with SRI International to conduct a series of assessments of outcomes and impacts of various specialized exchange programs under the overall umbrella of the Fulbright Educational Exchange Program, the U.S. government's…

Multicultural Conflict Resolution: Development, Implementation and Assessment of a Program for Third Graders.

ERIC Educational Resources Information Center

Graham, Benjamin C.; Pulvino, Charles

2000-01-01

Presents an intervention that outlines the formulation, implementation, and assessment of one counselor's attempt to increase student skills in the area of conflict resolution through a 6-week, curriculum-based, conflict resolution program for third-graders. Program evaluation indicates that it was successful in challenging students'…

76 FR 13648 - Proposed Collection; Comment Request; Process Evaluation of the NIH Roadmap Epigenomics Program

Federal Register 2010, 2011, 2012, 2013, 2014

2011-03-14

... assess program process and progress, is non-experimental. The assessment is based on secondary source... programs of the Agency. To reduce response bias and to make the survey as accessible as possible to busy principal investigators, the survey will be Web-based. Frequency of Response: Once. Affected Public...

Assessing the New Competencies for Resident Education: A Model from an Emergency Medicine Program.

ERIC Educational Resources Information Center

Reisdorff, Earl J.; Hayes, Oliver W.; Carlson, Dale J.; Walker, Gregory L.

2001-01-01

Based on the experience of Michigan State University's emergency medicine residency program, proposes a practical method for modifying an existing student evaluation format. The model provides a template other programs could use in assessing residents' acquisition of the knowledge, skills, and attitudes reflected in the six general competencies…

TOXICITY TESTING, RISK ASSESSMENT, AND OPTIONS FOR DREDGED MATERIAL MANAGEMENT

EPA Science Inventory

Programs for evaluating proposed discharges of dredged material into waters of the United States specify a tiered testing and evaluation protocol that includes performance of acute and chronic bioassays to assess toxicity of the dredged sediments. Although these evaluations refl...

Evaluation of Training Programs for Rural Development

ERIC Educational Resources Information Center

Indira, A.

2008-01-01

An Evaluation of the "Impact Assessment of the Training Programs" of a National Level Training Institution in India was conducted using the Kirkpatrick Method (KP Method). The studied Institution takes up research, provides training, offers consultancy and initiates action in the rural sector of India. The evaluation study used a…

Indian Program Evaluation and Needs Assessment. Hearings Before the United States Senate Select Committee on Indian Affairs, 95th Congress, 2nd Session on S. 2712--Relating to the Evaluation and Assessment of Indian Programs and Needs, and for Other Purposes (April 17 and 21, 1978).

ERIC Educational Resources Information Center

Congress of the U.S., Washington, DC. Senate Select Committee on Indian Affairs.

Included in this document are the text of S. 2712, statements by witnesses, and letters and comments from both government and American Indian representatives. The purpose of this legislation is to require all federal agencies which administer special programs and services for Indians to provide an annual report to Congress to facilitate program…

Verification approach for the Shuttle/Payload Contamination Evaluation computer program - Spacelab induced environment

NASA Technical Reports Server (NTRS)

Bareiss, L. E.

1978-01-01

The paper presents a compilation of the results of a systems level Shuttle/payload contamination analysis and related computer modeling activities. The current technical assessment of the contamination problems anticipated during the Spacelab program are discussed and recommendations are presented on contamination abatement designs and operational procedures based on experience gained in the field of contamination analysis and assessment, dating back to the pre-Skylab era. The ultimate test of the Shuttle/Payload Contamination Evaluation program will be through comparison of predictions with measured levels of contamination during actual flight.

Technology assessment in medicine. The role of the American Medical Association.

PubMed

McGivney, W T; Hendee, W R

1988-12-01

Public policy decisions about health care and decisions about an individual patient's care are only as sound as the data and information on which they are founded. The formal systematic evaluation of the safety, effectiveness, and cost-effectiveness of a medical technology provides such information and is essential to the appropriate application of technology in patient care. The American Medical Association (AMA) historically has recognized the need for the practicing medical community to develop and communicate accurate and balanced evaluative information to physicians and other health care professionals. In recent years, the association has established formal programs to accomplish this objective. This article describes the AMA's three major assessment programs, the Diagnostic and Therapeutic Technology Assessment program, the Council on Scientific Affairs, and AMA Drug Evaluations. The implications of these activities for patient care and public policy are discussed.

Evaluation and capacity building to improve precollege science and mathematics achievement in the US: 10 CFR, Part 605. Technical progress report, June--December 1992

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

1992-12-31

The National Center for Improving Science Education has undertaken activities to achieve evaluation goals for DOE`s Precollege programs: develop means to determine program quality; develop means for determining the contribution of DOE precollege programs to both teacher enhancement and student achievement; provide evaluation designs and instruments and reports of program quality and impact; and strengthen both DOE`s and the Labs` capacity to do both short- and long-term planning as well as deliver effective programs and evaluation. Appendices include evaluation/technical assistance report, profiling teacher research participation and teacher development programs, teacher surveys, impact assessment design, and teacher research participation programs anecdotesmore » for 8 labs.« less

Evaluation of Embedded System Component Utilized in Delivery Integrated Design Project Course

NASA Astrophysics Data System (ADS)

Junid, Syed Abdul Mutalib Al; Hussaini, Yusnira; Nazmie Osman, Fairul; Razak, Abdul Hadi Abdul; Idros, Mohd Faizul Md; Karimi Halim, Abdul

2018-03-01

This paper reports the evaluation of the embedded system component utilized in delivering the integrated electronic engineering design project course. The evaluation is conducted based on the report project submitted as to fulfil the assessment criteria for the integrated electronic engineering design project course named; engineering system design. Six projects were assessed in this evaluation. The evaluation covers the type of controller, programming language and the number of embedded component utilization as well. From the evaluation, the C-programming based language is the best solution preferred by the students which provide them flexibility in the programming. Moreover, the Analog to Digital converter is intensively used in the projects which include sensors in their proposed design. As a conclusion, in delivering the integrated design project course, the knowledge over the embedded system solution is very important since the high density of the knowledge acquired in accomplishing the project assigned.

A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming.

PubMed

Hansen, William B; Derzon, James H; Reese, Eric L

2014-06-01

We propose a method for creating groups against which outcomes of local pretest-posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness. © The Author(s) 2012.

Chapter 11: Sample Design Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Khawaja, M. Sami; Rushton, Josh

Evaluating an energy efficiency program requires assessing the total energy and demand saved through all of the energy efficiency measures provided by the program. For large programs, the direct assessment of savings for each participant would be cost-prohibitive. Even if a program is small enough that a full census could be managed, such an undertaking would almost always be an inefficient use of evaluation resources. The bulk of this chapter describes methods for minimizing and quantifying sampling error. Measurement error and regression error are discussed in various contexts in other chapters.

Endocrine Disruptor Screening Program Tier 1 Assessments

EPA Pesticide Factsheets

EPA has completed weight-of-evidence (WoE) assessments under the Endocrine Distruptor Screening Program (EDSP) for 52 pesticides included in the final list of chemicals for Tier 1 screening. See weight of evidence reports and data evaluation records.

Evaluation of a Dual Bilingual Program: English-Spanish and English-Southeast Asian Languages.

ERIC Educational Resources Information Center

Wheeler, George; And Others

An external evaluation of the Title VII Bilingual Education Program was designed to: (1) gather a subjective assessment from personnel working with the program; (2) observe classroom teaching and activities utilized in bilingual instruction; and (3) measure differences in levels of achievement. The instructional personnel appear to accept the…

Development, Implementation, and Evaluation of a Health Promotion Program in a College Setting.

ERIC Educational Resources Information Center

Fisher, Shirley P.; Fisher, Michele M.

1995-01-01

An evaluation of a health promotion program for employees at a New Jersey state college assessed the physical fitness and exercise program with educational components. Comparisons of experimental and control subjects indicated significant differences on high density lipoprotein (HDL), total cholesterol/HDL ratio, triglycerides, and body weight in…

Evaluation of an Innovative Post-Arrest Diversion Program: 12-Month Recidivism Analysis

ERIC Educational Resources Information Center

Dembo, Richard; Walters, Wansley; Wareham, Jennifer; Burgos, Catherimarty; Schmeidler, James; Hoge, Robert; Underwood, Lee

2008-01-01

We report the results of an evaluation study of a diversion program, operated by the Miami-Dade County Juvenile Assessment Center in Florida. The Post-Arrest Diversion (PAD) program represents an innovative approach to treatment and intervention within the juvenile justice system that utilizes standardized psychosocial risks and needs assessment…

An Evaluation of the Technical Adequacy of a Revised Measure of Quality Indicators of Transition

ERIC Educational Resources Information Center

Morningstar, Mary E.; Lee, Hyunjoo; Lattin, Dana L.; Murray, Angela K.

2016-01-01

This study confirmed the reliability and validity of the Quality Indicators of Exemplary Transition Programs Needs Assessment-2 (QI-2). Quality transition program indicators were identified through a systematic synthesis of transition research, policies, and program evaluation measures. To verify reliability and validity of the QI-2, we…

Evaluation Criteria for Micro-CAI: A Psychometric Approach

PubMed Central

Wallace, Douglas; Slichter, Mark; Bolwell, Christine

1985-01-01

The increased use of microcomputer-based instructional programs has resulted in a greater need for third-party evaluation of the software. This in turn has prompted the development of micro-CAI evaluation tools. The present project sought to develop a prototype instrument to assess the impact of CAI program presentation characteristics on students. Data analysis and scale construction was conducted using standard item reliability analyses and factor analytic techniques. Adequate subscale reliabilities and factor structures were found, suggesting that a psychometric approach to CAI evaluation may possess some merit. Efforts to assess the utility of the resultant instrument are currently underway.

Process Evaluation of a Workers' Health Surveillance Program for Meat Processing Workers.

PubMed

van Holland, Berry J; Brouwer, Sandra; de Boer, Michiel R; Reneman, Michiel F; Soer, Remko

2017-09-01

Objective To evaluate the implementation process of a workers' health surveillance (WHS) program in a Dutch meat processing company. Methods Workers from five plants were eligible to participate in the WHS program. The program consisted of four evaluative components and an intervention component. Qualitative and quantitative methods were used to evaluate seven process aspects. Data were gathered by interviews with stakeholders, participant questionnaires, and from registries of the company and occupational health service. Results Two recruitment strategies were used: open invitation or automatic participation. Of the 986 eligible workers, 305 participated in the program. Average reach was 53 %. Two out of five program components could not be assessed on dose delivered, dose received and fidelity. If components were assessable, 85-100 % of the components was delivered, 66-100 % of the components was received by participants, and fidelity was 100 %. Participants were satisfied with the WHS program (mean score 7.6). Contextual factors that facilitated implementation were among others societal developments and management support. Factors that formed barriers were program novelty and delayed follow-up. Conclusion The WHS program was well received by participants. Not all participants were offered the same number of program components, and not all components were performed according to protocol. Deviation from protocol is an indication of program failure and may affect program effectiveness.

Portfolios: An Alternative Method of Student and Program Assessment

PubMed Central

Hannam, Susan E.

1995-01-01

The use of performance-based evaluation and alternative assessment techniques has become essential for curriculum programs seeking Commission of Accreditation of Allied Health Education Programs (CAAHEP) accreditation. In athletic training education, few assessment models exist to assess student performance over the entire course of their educational program. This article describes a model of assessment-a student athletic training portfolio of “best works.” The portfolio can serve as a method to assess student development and to assess program effectiveness. The goals of the program include purposes specific to the five NATA performance domains. In addition, four types of portfolio evidence are described: artifacts, attestations, productions, and reproductions. Quality assignments and projects completed by students as they progress through a six-semester program are identified relative to the type of evidence and the domain(s) they represent. The portfolio assists with student development, provides feedback for curriculum planning, allows for student/faculty collaboration and “coaching” of the student, and assists with job searching. This information will serve as a useful model for those athletic training programs looking for an alternative method of assessing student and program outcomes. PMID:16558359

Program Evaluation: Experienced Agencies Follow a Similar Model for Prioritizing Research. Report to the Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia, Committee on Homeland Security and Governmental Affairs, U.S. Senate. GAO-11-176

ERIC Educational Resources Information Center

Kingsbury, Nancy

2011-01-01

Amid efforts to improve performance and constrain spending, federal agencies are being asked to expand the use of rigorous program evaluation in decision-making. In addition to performance data, indepth program evaluation studies are often needed for assessing program impact or designing improvements. Agencies can also use their evaluation…

Regional Issue Identification and Assessment (RIIA). Volume III. Institutional barriers to developing power generation facilities in the Pacific Northwest

DOE Office of Scientific and Technical Information (OSTI.GOV)

Morris, F. A.; Sawyer, C. H.; Maxwell, J. H.

1979-10-01

The Regional Assessments Division in the US Department of Energy (DOE) has undertaken a program to assess the probable consequences of various national energy policies in regions of the United States and to evaluate the constraints on national energy policy imposed by conditions in these regions. The program is referred to as the Regional Issues Identification and Assessment (RIIA) Program. Currently the RIIA Program is evaluating the Trendlong Mid-Mid scenario, a pattern of energy development for 1985 and 1990 derived from the Project Independence Evaluation System (PIES) model. This scenario assumes a medium annual growth rate in both the nationalmore » demand for and national supply of energy. It has been disaggregated to specify the generating capacity to be supplied by each energy source in each state. Pacific Northwest Laboratory (PNL) has the responsibility for evaluating the scenario for the Federal Region 10, consisting of Alaska, Idaho, Oregon, and Washington. PNL is identifying impacts and constraints associated with realizing the scenario in a variety of categories, including air and water quality impacts, health and safety effects, and socioeconomic impacts. This report summarizes the analysis of one such category: institutional constraints - defined to include legal, organizational, and political barriers to the achievement of the scenario in the Northwest.« less

Some questions and answers about the Satellite Power System (SPS)

NASA Technical Reports Server (NTRS)

1980-01-01

Progress in the evaluation of the concept of obtaining significant amount of electrical energy from space through the Satellite Power System is reported. The Concept Development and Evaluation Program plan is described including: systems definition, environmental assessment, societal assessment, and comparative assessment.

1998 FFTF annual system assessment reports

DOE Office of Scientific and Technical Information (OSTI.GOV)

Guttenberg, S.

The health of FFTF systems was assessed assuming a continued facility standby condition. The review was accomplished in accordance with the guidelines of FFTF-EI-083, Plant Evaluation Program. The attached document includes an executive summary of the significant conclusions and assessment reports for each system evaluated.

Evaluation of the Transit Reliability Information Program

DOT National Transportation Integrated Search

1982-06-01

This report presents an evaluation of the rail portion of the Transit Reliability Information Program (TRIP), which was designed to collect and analyze equipment reliability data on U.S. transit systems. This assessment was conducted at the end of it...

Solar energy program evaluation: an introduction

DOE Office of Scientific and Technical Information (OSTI.GOV)

deLeon, P.

The Program Evaluation Methodology provides an overview of the practice and methodology of program evaluation and defines more precisely the evaluation techniques and methodologies that would be most appropriate to government organizations which are actively involved in the research, development, and commercialization of solar energy systems. Formal evaluation cannot be treated as a single methodological approach for assessing a program. There are four basic types of evaluation designs - the pre-experimental design; the quasi-experimental design based on time series; the quasi-experimental design based on comparison groups; and the true experimental design. This report is organized to first introduce the rolemore » and issues of evaluation. This is to provide a set of issues to organize the subsequent sections detailing the national solar energy programs. Then, these two themes are integrated by examining the evaluation strategies and methodologies tailored to fit the particular needs of the various individual solar energy programs. (MCW)« less

Competency assessment in laboratory medicine: Standardization and utility for technical staff assessment and recertification in Saudi Arabia.

PubMed

Nemenqani, Dalal M; Tekian, Ara; Park, Yoon Soo

2017-04-01

The assessment of technical staff members' competency has been a challenge for laboratory workers, to ensure patient safety and high quality services. The aim of this study was to (1) investigate awareness on best ways to assess lab competencies; (2) identify existing institutional methods of competency assessment and how staff perceptions; and (3) gather opinions of respondents about a proposed program for competency assessment in laboratory medicine. A cross-sectional survey was conducted, followed by an interview and discussion with laboratory stakeholders about a proposed competency assessment program that included all the six procedural elements of laboratory personnel competency assessment. An online questionnaire was sent via email to different hospitals in Saudi Arabia through survey monkey. A proposed competency assessment program was circulated via email to laboratory stakeholders who agreed to be enrolled in structured interviews. A total of 47 out of the 168 (25.3%) laboratory workers responded to the emailed survey administered via survey monkey. Among the survey respondents, 16 out of the 47 (34%) participated in the structured interview and the discussion and formed the community of practice group that provided insight and opinion about the proposed competency program. Among stakeholders, 87.2% practiced in accredited laboratories. Over half (52%) of respondents positively rated the proposed program. Results of interviews and discussions revealed suggestions about continuous ongoing assessment, such as the inclusion of laboratory quality management and safety as separate items to be unified for all sections. The proposed competency assessment program overcomes challenges noted in competency assessment and has been positively received by stakeholders. This program will be validated by a group of experts then implemented as part of a core curriculum for laboratory staff, in their assessment, certification, recertification, registration, evaluation and licensure in sample laboratories in Saudi Arabia. The program will be monitored and evaluated during and after implementation for processes and outcomes. Conclusions will be utilized for national competency program. This study represents an important step towards the implementation of a standardized laboratory competence assessment program at a national level.

Comparative Analysis of the Religious Orientation and Spiritual and Character Development of Christian Student-Athletes at a Christian University and a Secular University: An Exploratory Study

ERIC Educational Resources Information Center

Schultz, Donald Glen

2009-01-01

Program evaluation is an essential part of the ongoing success of any organization. Program evaluations can be done for entire organizations or for any entity within that organization. Christian university athletic programs need to be evaluated frequently to assess whether the organization is fulfilling its goals and objectives. This study…

A framework for improving the cost-effectiveness of DSM program evaluations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sonnenblick, R.; Eto, J.

The prudence of utility demand-side management (DSM) investments hinges on their performance, yet evaluating performance is complicated because the energy saved by DSM programs can never be observed directly but only inferred. This study frames and begins to answer the following questions: (1) how well do current evaluation methods perform in improving confidence in the measurement of energy savings produced by DSM programs; (2) in view of this performance, how can limited evaluation resources be best allocated to maximize the value of the information they provide? The authors review three major classes of methods for estimating annual energy savings: trackingmore » database (sometimes called engineering estimates), end-use metering, and billing analysis and examine them in light of the uncertainties in current estimates of DSM program measure lifetimes. The authors assess the accuracy and precision of each method and construct trade-off curves to examine the costs of increases in accuracy or precision. Several approaches for improving evaluations for the purpose of assessing program cost effectiveness are demonstrated. The methods can be easily generalized to other evaluation objectives, such as shared savings incentive payments.« less

Outcome evaluation of a 3-year senior health and wellness initiative.

PubMed

Kuczmarksi, Marie Fanelli; Cotugna, Nancy

2009-02-01

The Wilmington Senior Center developed a 3-year multidisciplinary health and wellness initiative entitled The Time of Your Life to promote healthy aging for participants attending countywide senior centers. The purpose of this paper is to describe the development and implementation of the evaluation methodology and outcomes of this senior health and wellness project that consisted of 11 programs in total. Long-term programming can present several challenges in both implementation and evaluation, but assessing outcomes is critical to documenting program value and accountability. Participant-completed surveys and post-series focus groups were utilized as the evaluation tools. Our findings indicate that the program's objectives of increasing knowledge and motivating adoption or maintenance of a healthy lifestyle were met to varying degrees, while a third objective of assessing the usefulness of resources to inform health-related decisions was unable to be evaluated due to the cross-sectional nature of the data collection. Post-series focus groups indicated that topics of most interest to seniors were money management, medications and staying mentally and physically fit. The evaluations showed that long-term programming, while challenging, can be a successful and sustainable format for senior health education.

Coverage of Large-Scale Food Fortification of Edible Oil, Wheat Flour, and Maize Flour Varies Greatly by Vehicle and Country but Is Consistently Lower among the Most Vulnerable: Results from Coverage Surveys in 8 Countries123

PubMed Central

Aaron, Grant J; Friesen, Valerie M; Jungjohann, Svenja; Garrett, Greg S; Myatt, Mark

2017-01-01

Background: Large-scale food fortification (LSFF) of commonly consumed food vehicles is widely implemented in low- and middle-income countries. Many programs have monitoring information gaps and most countries fail to assess program coverage. Objective: The aim of this work was to present LSFF coverage survey findings (overall and in vulnerable populations) from 18 programs (7 wheat flour, 4 maize flour, and 7 edible oil programs) conducted in 8 countries between 2013 and 2015. Methods: A Fortification Assessment Coverage Toolkit (FACT) was developed to standardize the assessments. Three indicators were used to assess the relations between coverage and vulnerability: 1) poverty, 2) poor dietary diversity, and 3) rural residence. Three measures of coverage were assessed: 1) consumption of the vehicle, 2) consumption of a fortifiable vehicle, and 3) consumption of a fortified vehicle. Individual program performance was assessed based on the following: 1) achieving overall coverage ≥50%, 2) achieving coverage of ≥75% in ≥1 vulnerable group, and 3) achieving equity in coverage for ≥1 vulnerable group. Results: Coverage varied widely by food vehicle and country. Only 2 of the 18 LSFF programs assessed met all 3 program performance criteria. The 2 main program bottlenecks were a poor choice of vehicle and failure to fortify a fortifiable vehicle (i.e., absence of fortification). Conclusions: The results highlight the importance of sound program design and routine monitoring and evaluation. There is strong evidence of the impact and cost-effectiveness of LSFF; however, impact can only be achieved when the necessary activities and processes during program design and implementation are followed. The FACT approach fills an important gap in the availability of standardized tools. The LSFF programs assessed here need to be re-evaluated to determine whether to further invest in the programs, whether other vehicles are appropriate, and whether other approaches are needed. PMID:28404836

Coverage of Large-Scale Food Fortification of Edible Oil, Wheat Flour, and Maize Flour Varies Greatly by Vehicle and Country but Is Consistently Lower among the Most Vulnerable: Results from Coverage Surveys in 8 Countries.

PubMed

Aaron, Grant J; Friesen, Valerie M; Jungjohann, Svenja; Garrett, Greg S; Neufeld, Lynnette M; Myatt, Mark

2017-05-01

Background: Large-scale food fortification (LSFF) of commonly consumed food vehicles is widely implemented in low- and middle-income countries. Many programs have monitoring information gaps and most countries fail to assess program coverage. Objective: The aim of this work was to present LSFF coverage survey findings (overall and in vulnerable populations) from 18 programs (7 wheat flour, 4 maize flour, and 7 edible oil programs) conducted in 8 countries between 2013 and 2015. Methods: A Fortification Assessment Coverage Toolkit (FACT) was developed to standardize the assessments. Three indicators were used to assess the relations between coverage and vulnerability: 1 ) poverty, 2 ) poor dietary diversity, and 3 ) rural residence. Three measures of coverage were assessed: 1 ) consumption of the vehicle, 2 ) consumption of a fortifiable vehicle, and 3 ) consumption of a fortified vehicle. Individual program performance was assessed based on the following: 1 ) achieving overall coverage ≥50%, 2) achieving coverage of ≥75% in ≥1 vulnerable group, and 3 ) achieving equity in coverage for ≥1 vulnerable group. Results: Coverage varied widely by food vehicle and country. Only 2 of the 18 LSFF programs assessed met all 3 program performance criteria. The 2 main program bottlenecks were a poor choice of vehicle and failure to fortify a fortifiable vehicle (i.e., absence of fortification). Conclusions: The results highlight the importance of sound program design and routine monitoring and evaluation. There is strong evidence of the impact and cost-effectiveness of LSFF; however, impact can only be achieved when the necessary activities and processes during program design and implementation are followed. The FACT approach fills an important gap in the availability of standardized tools. The LSFF programs assessed here need to be re-evaluated to determine whether to further invest in the programs, whether other vehicles are appropriate, and whether other approaches are needed.

Test Review for Preschool-Wide Evaluation Tool (PreSET) Manual: Assessing Universal Program-Wide Positive Behavior Support in Early Childhood

ERIC Educational Resources Information Center

Rodriguez, Billie Jo

2013-01-01

The Preschool-Wide Evaluation Tool (PreSET; Steed & Pomerleau, 2012) is published by Paul H. Brookes Publishing Company in Baltimore, MD. The PreSET purports to measure universal and program-wide features of early childhood programs' implementation fidelity of program-wide positive behavior intervention and support (PW-PBIS) and is,…

Assessment of Alternative Student Aid Delivery Systems: Assessment of the Current Delivery System.

ERIC Educational Resources Information Center

Advanced Technology, Inc., Reston, VA.

The effects of the current system for delivering federal financial assistance to students under the Pell Grant, Guaranteed Student Loan (GSL), and campus-based programs are analyzed. Information is included on the use of the assessment model, which combines program evaluation, systems research, and policy analysis methodologies.…

Evaluation of a Teaching Assistant Program for Third-Year Pharmacy Students.

PubMed

Bradley, Courtney L; Khanova, Julia; Scolaro, Kelly L

2016-11-25

Objectives. To determine if a teaching assistant (TA) program for third-year pharmacy students (PY3s) improves confidence in teaching abilities. Additionally, 3 assessment methods (faculty, student, and TA self-evaluations) were compared for similarities and correlations. Methods. An application and interview process was used to select 21 pharmacy students to serve as TAs for the Pharmaceutical Care Laboratory course for 2 semesters. Participants' self-perceived confidence in teaching abilities was assessed at the start, midpoint, and conclusion of the program. The relationships between the scores were analyzed using 3 assessment methods. Results. All 21 TAs agreed to participate in the study and completed the 2 teaching semesters. The TAs confidence in overall teaching abilities increased significantly (80.7 vs 91.4, p <0.001). There was a significant difference between the three assessment scores in the fall ( p =0.027) and spring ( p <0.001) semesters. However, no correlation was found among the assessment scores. Conclusions. The TA program was effective in improving confidence in teaching abilities. The lack of correlation among the assessment methods highlights the importance of various forms of feedback.

Structured learning for robotic surgery utilizing a proficiency score: a pilot study.

PubMed

Hung, Andrew J; Bottyan, Thomas; Clifford, Thomas G; Serang, Sarfaraz; Nakhoda, Zein K; Shah, Swar H; Yokoi, Hana; Aron, Monish; Gill, Inderbir S

2017-01-01

We evaluated feasibility and benefit of implementing structured learning in a robotics program. Furthermore, we assessed validity of a proficiency assessment tool for stepwise graduation. Teaching cases included robotic radical prostatectomy and partial nephrectomy. Procedure steps were categorized: basic, intermediate, and advanced. An assessment tool ["proficiency score" (PS)] was developed to evaluate ability to safely and autonomously complete a step. Graduation required a passing PS (PS ≥ 3) on three consecutive attempts. PS and validated global evaluative assessment of robotic skills (GEARS) were evaluated for completed steps. Linear regression was utilized to determine postgraduate year/PS relationship (construct validity). Spearman's rank correlation coefficient measured correlation between PS and GEARS evaluations (concurrent validity). Intraclass correlation (ICC) evaluated PS agreement between evaluator classes. Twenty-one robotic trainees participated within the pilot program, completing a median of 14 (2-69) cases each. Twenty-three study evaluators scored 14 (1-60) cases. Over 4 months, 229/294 (78 %) cases were designated "teaching" cases. Residents completed 91 % of possible evaluations; faculty completed 78 %. Verbal and quantitative feedback received by trainees increased significantly (p = 0.002, p < 0.001, respectively). Average PS increased with PGY (post-graduate year) for basic and intermediate steps (regression slopes: 0.402 (p < 0.0001), 0.323 (p < 0.0001), respectively) (construct validation). Overall, PS correlated highly with GEARS (ρ = 0.81, p < 0.0001) (concurrent validity). ICC was 0.77 (95 % CI 0.61-0.88) for resident evaluations. Structured learning can be implemented in an academic robotic program with high levels of trainee and evaluator participation, encouraging both quantitative and verbal feedback. A proficiency assessment tool developed for step-specific proficiency has construct and concurrent validity.

15 CFR 286.2 - Scope.

Code of Federal Regulations, 2014 CFR

2014-01-01

... accreditation body which satisfies the established requirements. (b) NIST operates the NVCASE program as follows: (1) Conformity level: Under this program NIST accepts requests for evaluations of U.S. bodies involved in activities related to conformity assessment. NIST does not perform conformity assessments as...

15 CFR 286.2 - Scope.

Code of Federal Regulations, 2012 CFR

2012-01-01

... accreditation body which satisfies the established requirements. (b) NIST operates the NVCASE program as follows: (1) Conformity level: Under this program NIST accepts requests for evaluations of U.S. bodies involved in activities related to conformity assessment. NIST does not perform conformity assessments as...

15 CFR 286.2 - Scope.

Code of Federal Regulations, 2011 CFR

2011-01-01

... accreditation body which satisfies the established requirements. (b) NIST operates the NVCASE program as follows: (1) Conformity level: Under this program NIST accepts requests for evaluations of U.S. bodies involved in activities related to conformity assessment. NIST does not perform conformity assessments as...

15 CFR 286.2 - Scope.

Code of Federal Regulations, 2010 CFR

2010-01-01

... accreditation body which satisfies the established requirements. (b) NIST operates the NVCASE program as follows: (1) Conformity level: Under this program NIST accepts requests for evaluations of U.S. bodies involved in activities related to conformity assessment. NIST does not perform conformity assessments as...

15 CFR 286.2 - Scope.

Code of Federal Regulations, 2013 CFR

2013-01-01

... accreditation body which satisfies the established requirements. (b) NIST operates the NVCASE program as follows: (1) Conformity level: Under this program NIST accepts requests for evaluations of U.S. bodies involved in activities related to conformity assessment. NIST does not perform conformity assessments as...

SMART Security Cooperation Objectives: Improving DoD Planning and Guidance

DTIC Science & Technology

2016-01-01

integrate them into a system for assessing, monitoring, and evaluating security cooperation programs and activities. This report evaluates DoD’s...effectiveness in developing SMART security coopera- tion objectives that facilitate assessment, monitoring, and evaluation . It also proposes a systematic...Cooperation Ends, Ways, and Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 RAND Evaluation and Revision of Selected

Children/Youth Physical Fitness Program Management System.

ERIC Educational Resources Information Center

Mozzini, Lou; And Others

Intended for physical fitness program managers, this book presents a system through which these professionals can justify, evaluate, develop, supervise, and promote a sound physical fitness program. Sections address the management stages of: (1) program assessment; (2) program commitment; (3) physical fitness program planning; (4) program…

Evaluating the Facilities Planning, Design, and Construction Department: The Capital Programs Management Audit.

ERIC Educational Resources Information Center

Kaiser, Harvey H.; Kirkwood, Dennis M.

2000-01-01

Presents a diagnostic model for assessing the state of an institution's capital programs management (CPM) by delineating "work processes" which comprise that function. What capital programs management is, its resources, and its phases and work processes are described, followed by case studies of the CPM Process Model as an assessment tool. (GR)

Development of a Criterion-Referenced, Performance-Based Assessment of Reading Comprehension in a Whole Literacy Program.

ERIC Educational Resources Information Center

Tibbetts, Katherine A.; And Others

This paper describes the development of a criterion-referenced, performance-based measure of third grade reading comprehension. The primary purpose of the assessment is to contribute unique and valid information for use in the formative evaluation of a whole literacy program. A secondary purpose is to supplement other program efforts to…

"Doing Geography": Evaluating an Independent Geographic Inquiry Assessment Task in an Initial Teacher Education Program

ERIC Educational Resources Information Center

Harte, Wendy; Reitano, Paul

2016-01-01

The assessment task of the final course in a bachelor of secondary education program is examined for opportunities for preservice geography teachers to achieve the course aims of integrating, consolidating, applying, and reflecting on the knowledge and skills they have learned during their initial teacher education program. The results show that…

What happened after the evaluation?

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bennett, C L

1999-03-12

An ergonomics program including a ergonomic computer workstation evaluations at a research and development facility was assessed three years after formal implementation. As part of the assessment, 53 employees who had been subjects of computer workstation evaluations were interviewed. The documented reports (ergonomic evaluation forms) of the ergonomic evaluations were used in the process of selecting the interview subjects. The evaluation forms also provided information about the aspects of the computer workstation that were discussed and recommended as part of the evaluation, although the amount of detail and completeness of the forms varied. Although the results were mixed and reflectivemore » of the multivariate psychosocial factors affecting employees working in a large organization, the findings led to recommendations for improvements of the program.« less

Evaluation of biochars by temperature programmed oxidation/mass spectroscopy

USDA-ARS?s Scientific Manuscript database

Biochar from the thermochemical conversion of biomass was evaluated by Temperature Programmed Oxidation (TPO) coupled with mass spectroscopy. This technique can be used to assess the oxidative reactivity of carbonaceous solids where higher temperature reactivity indicates greater structural order. ...

Promoting Excellence in Nursing Education (PENE): Pross evaluation model.

PubMed

Pross, Elizabeth A

2010-08-01

The purpose of this article is to examine the Promoting Excellence in Nursing Education (PENE) Pross evaluation model. A conceptual evaluation model, such as the one described here, may be useful to nurse academicians in the ongoing evaluation of educational programs, especially those with goals of excellence. Frameworks for evaluating nursing programs are necessary because they offer a way to systematically assess the educational effectiveness of complex nursing programs. This article describes the conceptual framework and its tenets of excellence. Copyright 2009 Elsevier Ltd. All rights reserved.

When Control Groups Do Not Exist: Program Evaluation without a Net.

ERIC Educational Resources Information Center

Clark, Sheldon B.; And Others

Suggestions, based on the experience of researchers at Oak Ridge (Tennessee) Associated Universities (ORAU), are offered on how evaluations in which true experimental designs are not possible can be designed for meaningful comparison. The ORAU conducts evaluation and assessment studies of educational programs intended to explore some aspect of the…

Does the Kids Cafe Program's nutrition education improve children's dietary intake? A pilot evaluation study

USDA-ARS?s Scientific Manuscript database

Our objective was to evaluate the Kids Café Program (KCP) nutrition education intervention and assess its impact on children's diet quality and body mass index (BMI) percentile. An experimental design consisting of pretest-posttest comparison groups using mixed methods was used to evaluate the 6-ses...

Students Training for Academic Readiness (STAR): Year Three Evaluation Report. Executive Summary

ERIC Educational Resources Information Center

Texas Center for Educational Research, 2010

2010-01-01

This executive summary presents findings from the Year 3 evaluation of Texas' state-level Gaining Early Awareness and Readiness for Undergraduate Programs, or GEAR UP, grant. GEAR UP grant requirements include an evaluation component designed to assess program effectiveness and to measure progress toward project goals. To this end, the evaluation…

RESTORATION PLUS: A COLLABORATIVE RESEARCH PROGRAM TO DEVELOP AND EVALUATE ECOSYSTEM RESTORATION AND MANAGEMENT OPTIONS TO ACHIEVE ECOLOGICALLY AND ECONOMICALLY SUSTAINABLE SOLUTIONS

EPA Science Inventory

EPA is evaluating ecosystem restoration and management techniques to ensure they create sustainable solutions for degraded watersheds. ORD NRMRL initiated the Restoration Plus (RePlus) program in 2002 to a) evaluate ecosystem restoration and management options, b) assess the non-...

Evaluation Techniques for the Sandy Point Discovery Center, Great Bay National Estuarine Research Reserve.

ERIC Educational Resources Information Center

Heffernan, Bernadette M.

1998-01-01

Describes work done to provide staff of the Sandy Point Discovery Center with methods for evaluating exhibits and interpretive programming. Quantitative and qualitative evaluation measures were designed to assess the program's objective of estuary education. Pretest-posttest questionnaires and interviews are used to measure subjects' knowledge and…

Assessing International Evaluations. An Example from USAID's Democracy and Governance Program

ERIC Educational Resources Information Center

Bollen, Kenneth; Paxton, Pamela; Morishima, Rumi

2005-01-01

Each year, hundreds of millions of dollars are spent in international and bilateral aid programs to assist developing countries in expanding democratic governance. Evaluating these efforts' effectiveness is important given the vast needs of the client population. This article reviews a sample of evaluations of the United States Agency for…

No Simple Answer: Critique of the Follow Through Evaluation

ERIC Educational Resources Information Center

House, Ernest R.; And Others

1978-01-01

The evaluation by Abt Associates, Inc., of the Follow Through program, an extension of Head Start designed to improve primary education for disadvantaged children, created a controversy about the program and its assessment, which is discussed in this article. The authors see serious flaws in the evaluation design and interpretation. (MF)

Evaluation of the Albuquerque Indian School Motivational Environment Program.

ERIC Educational Resources Information Center

Hiat, Albert B.; And Others

In order to evaluate the impact of a token economy behavior modification program implemented from 1970-71 in the Albuquerque Indian School (AIS), a secondary institution, a five-member evaluation team assessed standardized test results, behavioral data, and student and staff attitudes. A battery of tests (Tennessee Self-Concept Scale, SRA…

Employer Expectations of Sport Managers and Evaluation of Sport Management Programs in the United States.

ERIC Educational Resources Information Center

DeSensi, Joy T.; And Others

The report consists of a series of studies designed to assess the needs of sport management positions and evaluate sport management programs/curricula in institutions of higher education. Three surveys were conducted to determine: (1) employer expectations of sport managers; (2) employer evaluation of educational sport management…

Use of program logic models in the Southern Rural Access Program evaluation.

PubMed

Pathman, Donald; Thaker, Samruddhi; Ricketts, Thomas C; Albright, Jennifer B

2003-01-01

The Southern Rural Access Program (SRAP) evaluation team used program logic models to clarify grantees' activities, objectives, and timelines. This information was used to benchmark data from grantees' progress reports to assess the program's successes. This article presents a brief background on the use of program logic models--essentially charts or diagrams specifying a program's planned activities, objectives, and goals--for evaluating and managing a program. It discusses the structure of the logic models chosen for the SRAP and how the model concept was introduced to the grantees to promote acceptance and use of the models. The article describes how the models helped clarify the program's objectives and helped lead agencies plan and manage the many program initiatives and subcontractors in their states. Models also provided a framework for grantees to report their progress to the National Program Office and evaluators and promoted the evaluators' visibility and acceptance by the grantees. Program logics, however, increased grantees' reporting requirements and demanded substantial time of the evaluators. Program logic models, on balance, proved their merit in the SRAP through their contributions to its management and evaluation and by providing a better understanding of the program's initiatives, successes, and potential impact.

The Impact of a Developed Measurement and Evaluation Development Program on Pre-Service Physical Education Teachers' Perceptions Related to Measurement and Evaluation

ERIC Educational Resources Information Center

Arslan, Yunus; Erturan Ilker, Gokce; Demirhan, Giyasettin

2013-01-01

The purpose of this study was to investigate the impact of the Measurement and Evaluation Development Program on pre-service physical education teachers' general perceptions and competency perceptions related to alternative assessment in physical education, and their competency perceptions related to educational measurement and evaluation. The…

An Evaluation of a School-Based, Peer-Facilitated, Healthy Relationship Program for At-Risk Adolescents

ERIC Educational Resources Information Center

McLeod, David Axlyn; Jones, Robin; Cramer, Elizabeth P.

2015-01-01

There are few evaluations of peer-facilitated teenage dating violence prevention programs in the literature. To begin to address this gap, this project assessed the effectiveness of a school-based, peer-facilitated healthy relationships program among academically at-risk students. Two hundred and ninety-one ninth graders of mixed race and gender…

Supportive Services for Socially Maladjusted Children in Regular Schools. Evaluation of New York City Title I Educational Projects, 1966-67.

ERIC Educational Resources Information Center

Williams, Thelma M.

Evaluated are several programs for socially maladjusted public school children. These supportive services are an Early Identification Program, Junior Guidance classes, Special Guidance classes, and Career Guidance classes. Assessment focused on implementation of the Board of Education's plan to augment special services in these programs, and on…

Evaluation of the MIND Research Institute's Spatial-Temporal Math (ST Math) Program in California

ERIC Educational Resources Information Center

Wendt, Staci; Rice, John; Nakamoto, Jonathan

2014-01-01

The MIND Research Institute contracted with the Evaluation Research Program at WestEd to conduct an independent assessment of mathematics outcomes in elementary school grades across California that were provided with the ST Math program. Spatial-Temporal (ST) Math is a game-based instructional software designed to boost K-5 and secondary-level…

Applying Matched Sampling to Evaluate a University Tutoring Program for First-Year Students

ERIC Educational Resources Information Center

Walvoord, Mark E.; Pleitz, Jacob D.

2016-01-01

Our study used a case-control matching design to assess the influence of a voluntary tutoring program in improving first-year students' Grade Point Averages (GPA). To evaluate program effectiveness, we applied case-control matching to obtain 215 pairs of students with or without participation in tutoring, but matched on high school GPA and…

Evaluation of a Classroom Program of Creative Expression Workshops for Refugee and Immigrant Children

ERIC Educational Resources Information Center

Rousseau, Cecile; Drapeau, Aline; Lacroix, Louise; Bagilishya, Deogratias; Heusch, Nicole

2005-01-01

Objective: This evaluative study assessed the effect of a creative expression program designed to prevent emotional and behavioral problems and to enhance self-esteem in immigrant and refugee children attending multiethnic schools. Method: The 12-week program involved 138 children, aged 7 to 13, registered in both integration classes designed for…

The Joint Council on Thoracic Surgery Education (JCTSE) "Educate the Educators" Faculty Development Course: Analysis of the First 5 Years.

PubMed

Yang, Stephen C; Vaporciyan, Ara A; Mark, Rebecca J; DaRosa, Deborah A; Stritter, Frank T; Sullivan, Maura E; Verrier, Edward D

2016-12-01

Since 2010, the Joint Council on Thoracic Surgery Education, Inc (JCTSE) has sponsored an annual "Educate the Educators" (EtE) course. The goal is to provide United States academic cardiothoracic surgeons (CTS) the fundamentals of teaching skills, educational curriculum development, and using education for academic advancement. This report describes the course development and evaluation along with attendee's self-assessment of skills through the first 5 years of the program. The content of this 2½-day course was based on needs assessment surveys of CTS and residents attending annual meetings in 2009. From 2010 to 2014, EtE was offered to all CTS at training programs approved by the Accreditation Council for Graduate Medical Education. Course content was evaluated by using end-of-course evaluation forms. A 5-point Likert scale (1 = poor, 5 = excellent) was used to obtain composite assessment mean scores for the 5 years on course variables, session presentations, and self-assessments. With 963 known academic CTS in the United States, 156 (16.3%) have attended, representing 70 of 72 training programs (97%), and 1 international surgeon attended. There were also 7 program coordinators. Ratings of core course contents ranged from 4.4 to 4.8, accompanied with highly complementary comments. Through self-assessment, skills and knowledge in all content areas statistically improved significantly. The effect of the course was evaluated with a follow-up survey in which responders rated the program 4.3 on the usefulness of the information for their career and 3.9 for educational productivity. The EtE program offers an excellent opportunity for academic CTS to enhance their teaching skills, develop educational activities, and prepare for academic promotion. With its unique networking and mentorship environment, the EtE program is an important resource in the evolution of cardiothoracic surgical training in the United States. Copyright © 2016 The Society of Thoracic Surgeons. Published by Elsevier Inc. All rights reserved.

Centers of excellence: an assessment tool for cardiovascular and orthopedic programs.

PubMed

Ronning, P L; Meyer, J W

1996-10-01

As payers place more weight on contracting with hospital/health system programs that can differentiate themselves in the market as a "true" center of excellence (COE), it becomes imperative that hospitals/health systems understand the payer perspective about those programmatic attributes that can truly differentiate them from other programs. This report describes an evaluation and rating methodology for hospital/health system subspecialty programs, particularly cardiovascular and orthopedic programs, that can be used as a self-assessment tool. Using as its core a Rating Scale and Ranking Taxonomy, the evaluation and rating methodology presented here allow cardiovascular and orthopedic programs to do the following: Understand the differentiating characteristic of COE. Rate itself against detailed criteria that are being used by payers. Compare aspects of its program to premier or benchmark programs. Interpret the results to assist with strategic and operational direction. Allocate scarce resources to implement a subspecialty program that will attract payers. The Rating Scale and Ranking Taxonomy has 20 criteria for assessing cardiovascular programs and 18 criteria for orthopedic programs. The assessment process is designed to produce two important results: dialogue and action. The underpinnings of any action is a solid business plan that clarifies the program's vision, values, and mission. They are important because most programs will ultimately pursue very similar strategies and tactics; however, the most successful subspecialty programs and practices will be the ones that can execute the strategies and tactics quickly and effectively. In addition, the changes that are engendered by this targeted yet comprehensive assessment process can lead to improved clinical and functional outcomes for patients, as well as systemic improvements in the delivery of care.

CARDIOVASCULAR SCREENING OF YOUNG ATHLETES: A REVIEW OF ECONOMIC EVALUATIONS.

PubMed

Gerkens, Sophie; Van Brabandt, Hans; Desomer, Anja; Leonard, Christian; Neyt, Mattias

2017-01-01

Some experts have promoted preparticipative cardiovascular screening programs for young athletes and have claimed that such programs were cost-effective without performing a critical analysis of studies supporting this statement. In this systematic review, a critical assessment of economic evaluations on these programs is performed to determine if they really provide value for money. A systematic review of economic evaluations was performed on December 24, 2014. Web sites of health technology assessment agencies, the Cochrane database of systematic review, the National Health Service Economic Evaluation Database of the Cochrane Library, EMBASE, Medline, Psychinfo, and EconLit were searched to retrieve (reviews of) economic evaluations. No language or time restrictions were imposed and predefined selection criteria were used. Selected studies were critically assessed applying a structured data extraction sheet. Five relevant economic evaluations were critically assessed. Results of these studies were mixed. However, those in favor of screening made (methodological) incorrect choices, of which the most important one was not taking into account a no-screening alternative as comparator. Compared with no screening, other strategies (history and physical examination or history and physical examination plus electrocardiogram) were not considered cost-effective. Results of primary economic evaluations should not be blindly copied without critical assessment. Economic evaluations in this field lack the support of robust evidence. Negative consequences of screening (false positive findings, overtreatment) should also be taken into account and may cause more harm than good. A mass screening of young athletes for cardiovascular diseases does not provide value for money and should be discouraged.

ASSESSING THE SUITABILITY OF WINDBREAKS AS WILDLIFE HABITAT - 1994 PILOT PLAN

EPA Science Inventory

The Environmental Monitoring and Assessment Program's (EMAP) Agroecosystem Resource Group is developing a program to monitor and evaluate the ecological condition of United States agricultural lands. indbreaks are an important non-crop element in the Great Plains, an extensive ag...

A training program for anthropometric measurements by a dedicated nutrition support team improves nutritional status assessment of the critically ill child.

PubMed

Valla, Frederic V; Ford-Chessel, Carole; Meyer, Rosan; Berthiller, Julien; Dupenloup, Christine; Follin-Arbelet, Nathalie; Hubert, Anna; Javouhey, Etienne; Peretti, Noel

2015-03-01

The cornerstone of an optimal nutrition approach in PICUs is to evaluate the nutritional status of any patient. Anthropometric measurements and nutritional indices calculation allow for nutritional status assessment, which is not often part of routine management, as it is considered difficult to perform in this setting. We designed a study to evaluate the impact of a training program by the PICU nutritional support team on the implementation of routine anthropometric measurements on our PICU. A prospective study was performed over a 2-year period, which included: a baseline evaluation of nutritional assessment, knowledge, anthropometric measurements (weight, height, and head and mid upper arm circumferences), and nutritional indices calculation in patient files. This was followed by a training program to implement the newly developed nutrition assessment guidelines, which included anthropometrical measurements and also the interpretation of these. The impact of this nutritional assessment program was reviewed annually for 2 years after the implementation. PICU--Lyon, France. PICU nursing and medical staff, and patients admitted in February 2011, 2012, and 2013. Training program. Ninety-nine percent of staff (n = 145) attended the individual teaching. We found significant progress in nutritional awareness and confidence about nutritional assessment following the teaching program. In addition, an improvement in staff knowledge about undernutrition and its consequences were found. We enrolled 41, 55, and 91 patients in 2011, 2012, and 2013, respectively. There was a significant increase in anthropometric measurements during this time: 32%, 65% (p = 0.002), and 96% in 2013 (p < 0.001). Nutritional indices were calculated in 20%, 74% (p < 0.001), and 96% (p < 0.001) of cases. This is the first study, showing that a targeted nutritional assessment teaching program that highlights both the importance and techniques of anthropometrical measurements has successfully been implemented in a PICU. It managed to improve staff knowledge and nutritional practice.

Estimating Impacts on Students, Teachers, and Schools of Programs Targeted for Preschool through Transition to College--Lessons for Methodologists from the Designs of 117 Evaluations Being Conducted in the Investing in Innovation Fund (i3) Grant Program.

ERIC Educational Resources Information Center

Price, Cristofer; Wolf, Anne; Goodson, Barbara; Boulay, Beth

2015-01-01

When presenting results from rigorous, field-based evaluations of educational interventions, researchers often reveal methodological barriers they face in designing studies to assessing program effectiveness. The evaluations funded by the Investing in Innovation Fund (i3) present a timely opportunity for identifying relevant and pressing issues…

Self-evaluation of assessment programs: a cross-case analysis.

PubMed

Baartman, Liesbeth K J; Prins, Frans J; Kirschner, Paul A; van der Vleuten, Cees P M

2011-08-01

The goal of this article is to contribute to the validation of a self-evaluation method, which can be used by schools to evaluate the quality of their Competence Assessment Program (CAP). The outcomes of the self-evaluations of two schools are systematically compared: a novice school with little experience in competence-based education and assessment, and an innovative school with extensive experience. The self-evaluation was based on 12 quality criteria for CAPs, including both validity and reliability, and criteria stressing the importance of the formative function of assessment, such as meaningfulness and educational consequences. In each school, teachers, management and examination board participated. Results show that the two schools use different approaches to assure assessment quality. The innovative school seems to be more aware of its own strengths and weaknesses, to have a more positive attitude towards teachers, students, and educational innovations, and to explicitly involve stakeholders (i.e., teachers, students, and the work field) in their assessments. This school also had a more explicit vision of the goal of competence-based education and could design its assessments in accordance with these goals. Copyright © 2011 Elsevier Ltd. All rights reserved.

Professionalism Mini-Evaluation Exercise in Finland: A preliminary investigation introducing the Finnish version of the P-MEX instrument.

PubMed

Karukivi, Max; Kortekangas-Savolainen, Outi; Saxén, Ulla; Haapasalo-Pesu, Kirsi-Maria

2015-10-01

Teaching medical professionalism is increasingly acknowledged as an important aspect of medical education. The Professionalism Mini-Evaluation Exercise (P-MEX) is an assessment tool for evaluating medical professionalism, but no studies using it as a self-assessment instrument have been reported. This paper reports on a preliminary investigation of the Finnish version of the P-MEX instrument as an assessment and self-assessment measure. The sample in the present cross-sectional study comprised all 23 medical students and recent graduates (15 females and 8 males) participating in a summer school of psychiatry program in 2014. The two-month program combines clinical work with multifaceted teaching and intensive tutoring. At the end of the program, the participants' medical professionalism was assessed by the tutors and other members of the work community as well as the students themselves using the Finnish version of the P-MEX instrument. The P-MEX scores were compared, using the Friedman test. The mean values and SD for the P-MEX assessments were as follows: tutor assessment 3.26±0.21, work community assessment 3.32±0.26 and self-assessment 3.01±0.07. No significant gender differences were observed. The tutor and work community assessments were significantly correlated (r=0.573, p=0.040), but the self-assessment scores did not correlate with either of the other assessments. Overall, the students evaluated their skills significantly poorer in comparison to the other assessments. Although the small sample size limits the generalization of these preliminary results, the Finnish version of the P-MEX instrument appears to be a feasible measure of medical professionalism. The instrument can also be used as a self-assessment instrument, but subjective evaluations should be complemented with external assessments or feedback in order to take individual and cultural aspects into account.

Assessing Organizational Readiness for a Participatory Occupational Health/Health Promotion Intervention in Skilled Nursing Facilities.

PubMed

Zhang, Yuan; Flum, Marian; West, Cheryl; Punnett, Laura

2015-09-01

The long-term care sector is characterized by high morbidity and employee turnover, along with associated costs. Effective health protection and health promotion are important to improve physical and psychosocial well-being of caregivers. Assessment of organizational readiness for change is an essential precursor to the successful implementation of workplace programs addressing work climate, structure of tasks and relationships, and other issues that may be perceived as challenging by some within the institution. This study qualitatively assessed readiness of five skilled nursing facilities for a participatory occupational health/health promotion intervention. Selection criteria were developed to screen for program feasibility and ability to conduct prospective evaluations, and information was collected from managers and employees (interviews and focus groups). Three centers were selected for the program, and the first year of formative evaluation and intervention experience was then reviewed to evaluate and modify our selection criteria after the fact. Lessons learned include adding assessment of communication and the structure of problem solving to our selection criteria, improving methods to assess management support in a concrete (potentially nonverbal) form, and obtaining a stated financial commitment and resources to enable the team to function. Assessment of organizational readiness for change is challenging, although necessary to implement effective and sustainable health promotion programs in specific organizations. © 2015 Society for Public Health Education.

25 CFR 36.50 - Standard XVII-School program evaluation and needs assessment.

Code of Federal Regulations, 2010 CFR

2010-04-01

... assessment. Each school shall complete a formal, formative evaluation at least once every seven (7) years... assessment. 36.50 Section 36.50 Indians BUREAU OF INDIAN AFFAIRS, DEPARTMENT OF THE INTERIOR EDUCATION... other instructional materials. (12) Counseling services. (13) Medical and health services. (14) Student...

25 CFR 36.50 - Standard XVII-School program evaluation and needs assessment.

Code of Federal Regulations, 2011 CFR

2011-04-01

... assessment. Each school shall complete a formal, formative evaluation at least once every seven (7) years... assessment. 36.50 Section 36.50 Indians BUREAU OF INDIAN AFFAIRS, DEPARTMENT OF THE INTERIOR EDUCATION... other instructional materials. (12) Counseling services. (13) Medical and health services. (14) Student...

How Can We Assess Innovative Structures and Programs?

ERIC Educational Resources Information Center

Shuy, Roger W.

Preoccupation with the science of educational evaluation has led educators to overlook the art of evaluation, the ability to make quick and accurate assessments, usually without statistical props. This paper suggests common-sense steps to insure accurate assessment of innovative structures. (1) Know the nature and purpose of innovation. An urban…

Evaluation of Young Children for Early Intervention and Early Childhood Special Education.

ERIC Educational Resources Information Center

Allen, Diana

This technical assistance document provides guidelines for child assessment and eligibility determination for early intervention and early childhood special education programs in Oregon. An overview of the assessment process explains screening, eligibility evaluation, and assessment for the Individual Family Service Plan (IFSP). Legal requirements…

Evaluation of the Second Year (1968-1969) of the Sustained Primary Program for Bilingual Students in the Las Cruces, New Mexico Public School System.

ERIC Educational Resources Information Center

Cordova, Ignacio R.; And Others

Evaluating the 2nd-year K-3 bilingual program in 4 elementary schools, this document assessed (1) the increase in achievement level of Spanish-speaking pupils through use of a sustained K-3 program; (2) the student achievement level between bilingual and monolingual programs; (3) the involvement of Spanish-speaking parents as advisors and…

Analysis of School Leaders Licensure Assessment Content Category I-V Scores and Principal Internship Self-Assessment Scores for ISLLC Standards I-V

ERIC Educational Resources Information Center

Kelly, Michael D.

2016-01-01

This study compares School Leaders Licensure Assessment (SLLA) sub-scores with principal interns' self-assessment sub-scores (ISA) for a principal internship evaluation instrument in one educational leadership graduate program. The results of the study will be used to help establish the effectiveness of the current principal internship program,…

Emergency Immigration Education Act Programs: Summer E.S.L. Welcome Plus Program for Students of Limited English Proficiency (LEP), Summer Bilingual Program, and Project Omega. OREA Report.

ERIC Educational Resources Information Center

Duque, Diana L.

This report presents findings of the evaluation by the New York City public school system's Office of Research, Evaluation, and Assessment of three programs (Summer E.S.L. Welcome Plus, Summer Bilingual, and Project Omega) for immigrant students. The Summer E.S.L. (English as a Second Language) Welcome Plus program operated at 19 sites in New York…

Use of an Integrated Pest Management Assessment Administered through Turningpoint as an Educational, Needs Assessment, and Evaluation Tool

ERIC Educational Resources Information Center

Stahl, Lizabeth A. B.; Behnken, Lisa M.; Breitenbach, Fritz R.; Miller, Ryan P.; Nicolai, David; Gunsolus, Jeffrey L.

2016-01-01

University of Minnesota educators use an integrated pest management (IPM) survey conducted during private pesticide applicator training as an educational, needs assessment, and evaluation tool. By incorporating the IPM Assessment, as the survey is called, into a widely attended program and using TurningPoint audience response devices, Extension…

A Three-Year Reflective Writing Program as Part of Introductory Pharmacy Practice Experiences

PubMed Central

Vaughn, Jessica; Kerr, Kevin; Zielenski, Christopher; Toppel, Brianna; Johnson, Lauren; McCauley, Patrina; Turner, Christopher J.

2013-01-01

Objectives. To implement and evaluate a 3-year reflective writing program incorporated into introductory pharmacy practice experiences (IPPEs) in the first- through third-year of a doctor of pharmacy (PharmD) program. Design. Reflective writing was integrated into 6 IPPE courses to develop students’ lifelong learning skills. In their writing, students were required to self-assess their performance in patient care activities, identify and describe how they would incorporate learning opportunities, and then evaluate their progress. Practitioners, faculty members, and fourth-year PharmD students served as writing preceptors. Assessment. The success of the writing program was assessed by reviewing class performance and surveying writing preceptor’s opinions regarding the student’s achievement of program objectives. Class pass rates averaged greater than 99% over the 8 years of the program and the large majority of the writing preceptors reported that student learning objectives were met. A support pool of 99 writing preceptors was created. Conclusions. A 3-year reflective writing program improved pharmacy students’ reflection and reflective writing skills. PMID:23788811

ORD Land Research Program Mid-Cycle Review - January 2009

EPA Pesticide Factsheets

The purpose of the review was to evaluate progress that the Land Research Program has made since the 2005 program review and to assess the responsiveness of the Program to advice, comments, and recommendations provided by the BOSC.

77 FR 28360 - Notice of Proposed Information Collection Requests; Office of Planning, Evaluation and Policy...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-05-14

... study has two main components: (1) A review of recent research on the identification of ELs with special... descriptive in nature. The study is not a program evaluation and does not purport to assess program outcomes...

Evaluation of spacecraft technology programs (effects on communication satellite business ventures), volume 1

NASA Technical Reports Server (NTRS)

Greenburg, J. S.; Gaelick, C.; Kaplan, M.; Fishman, J.; Hopkins, C.

1985-01-01

Commercial organizations as well as government agencies invest in spacecraft (S/C) technology programs that are aimed at increasing the performance of communications satellites. The value of these programs must be measured in terms of their impacts on the financial performane of the business ventures that may ultimately utilize the communications satellites. An economic evaluation and planning capability was developed and used to assess the impact of NASA on-orbit propulsion and space power programs on typical fixed satellite service (FSS) and direct broadcast service (DBS) communications satellite business ventures. Typical FSS and DBS spin and three-axis stabilized spacecraft were configured in the absence of NASA technology programs. These spacecraft were reconfigured taking into account the anticipated results of NASA specified on-orbit propulsion and space power programs. In general, the NASA technology programs resulted in spacecraft with increased capability. The developed methodology for assessing the value of spacecraft technology programs in terms of their impact on the financial performance of communication satellite business ventures is described. Results of the assessment of NASA specified on-orbit propulsion and space power technology programs are presented for typical FSS and DBS business ventures.

Evaluation of spacecraft technology programs (effects on communication satellite business ventures), volume 1

NASA Astrophysics Data System (ADS)

Greenburg, J. S.; Gaelick, C.; Kaplan, M.; Fishman, J.; Hopkins, C.

1985-09-01

Commercial organizations as well as government agencies invest in spacecraft (S/C) technology programs that are aimed at increasing the performance of communications satellites. The value of these programs must be measured in terms of their impacts on the financial performane of the business ventures that may ultimately utilize the communications satellites. An economic evaluation and planning capability was developed and used to assess the impact of NASA on-orbit propulsion and space power programs on typical fixed satellite service (FSS) and direct broadcast service (DBS) communications satellite business ventures. Typical FSS and DBS spin and three-axis stabilized spacecraft were configured in the absence of NASA technology programs. These spacecraft were reconfigured taking into account the anticipated results of NASA specified on-orbit propulsion and space power programs. In general, the NASA technology programs resulted in spacecraft with increased capability. The developed methodology for assessing the value of spacecraft technology programs in terms of their impact on the financial performance of communication satellite business ventures is described. Results of the assessment of NASA specified on-orbit propulsion and space power technology programs are presented for typical FSS and DBS business ventures.

Incorporating Risk Assessment into the Formative Evaluation of an Authentic e-Learning Program

ERIC Educational Resources Information Center

Vesper, James L.; Kartoglu, Ümit; Herrington, Jan; Reeves, Thomas C.

2016-01-01

This paper describes the use of two different risk assessment strategies during the design and development of a complex authentic task-based e-learning program developed by the World Health Organization (WHO). The first strategy involved the use of expert reviewers and the second strategy employed the engagement of a risk assessment expert…

Evaluating the Student Learning Outcomes Assessment Process in Undergraduate Parks and Recreation Academic Programs

ERIC Educational Resources Information Center

Ross, Craig M.; Young, Sarah J.; Sturts, Jill R.

2012-01-01

Institutions of higher education are increasingly being held more accountable for assessing student learning both in and out of their classrooms along with reporting results to their stakeholders. The purpose of this study, which examined assessment of student learning outcomes in undergraduate park and recreation academic programs, was two-fold:…

An Assessment Instrument for Families: Evaluating Community-Based Residential Programs for Individuals with Deaf-Blindness.

ERIC Educational Resources Information Center

Helen Keller National Center - Technical Assistance Center, Sands Point, NY.

This community living assessment tool for parents of children with deaf-blindness was developed to help parents identify the strengths and weaknesses of their child's residential program using a user-friendly instrument. Three areas of assessment are covered: physical attributes of the home, available resources for promoting capabilities, and…

Getting the Most Bang from Your Volunteer Hour: Easy Assessments in the Dark Skies, Bright Kids Program

NASA Astrophysics Data System (ADS)

Beaton, R. L.; Sokal, K. R.; Liss, S. E.; Johnson, K. E.

2015-11-01

Dark Skies, Bright Kids! (DSBK) is an outreach organization that seeks to enhance elementary-level science literacy and encourage inquiry through fun, hands-on activities. DSBK was formed by, and is operated through, volunteer efforts from professional scientists at all career stages, e.g., from first-year undergraduate students to tenured professors. Although DSBK has amassed over 14,000 contact hours since 2009, there has been no formal evaluation of the programs impacts. Over the past year, DSBK introduced a large-scale, student-led internal assessments program with the systematic evaluation of student workbooks, volunteer surveys, and observations. While the data indicated broad-scale success for the program for both of its goals, it also revealed the organizational and educational practices that not only maximized student achievement, but also created the largest overall volunteer satisfaction with their time commitment. Here we describe DSBK in detail, summarize the student-led implementation of the assessments program, discuss how the results of the assessments have positively impacted our operations, and generalize these results for other scientist-led outreach efforts.

Barbershop Talk With Brothers: using community-based participatory research to develop and pilot test a program to reduce HIV risk among Black heterosexual men.

PubMed

Wilson, Tracey E; Fraser-White, Marilyn; Williams, Kim M; Pinto, Angelo; Agbetor, Francis; Camilien, Brignel; Henny, Kirk; Browne, Ruth C; Gousse, Yolene; Taylor, Tonya; Brown, Humberto; Taylor, Raekiela; Joseph, Michael A

2014-10-01

There is a need for feasible, evidence-based interventions that support HIV risk reduction among heterosexual Black men. In this article, we describe the process for development of the Barbershop Talk With Brothers (BTWB) program and evaluation. The BTWB program is a theoretically grounded and community-based HIV prevention program that seeks to improve individual skills and motivation to decrease sexual risk, and that builds men's interest in and capacity for improving their community's health. Formative data collection included barbershop observations and barber focus groups, brief behavioral risk assessments of men in barbershops, and focus groups and individual interviews. Based on this information and in consultation with our steering committee, we developed the BTWB program and accompanying program evaluation. From April through November 2011, 80 men were recruited and completed a baseline assessment of a pilot test of the program; 78 men completed the program and 71 completed a 3-month assessment. The pilot evaluation procedures were feasible to implement, and assessments of pre- and post-test measures indicate that key behavioral outcomes and proposed mediators of those outcomes changed in hypothesized directions. Specifically, attitudes and self-efficacy toward consistent condom use improved, and respondents reported lower levels of sexual risk behavior from baseline to follow-up (all p < 0.05). Perceptions of community empowerment also increased (p = 0.06). While HIV stigma decreased, this difference did not reach statistical significance. Our approach to community-engaged program development resulted in an acceptable, feasible approach to reaching and educating heterosexual Black men about HIV prevention in community settings.

Embedding assessment in a simulation skills training program for medical and midwifery students: A pre- and post-intervention evaluation.

PubMed

Kumar, Arunaz; Nestel, Debra; East, Christine; Hay, Margaret; Lichtwark, Irene; McLelland, Gayle; Bentley, Deidre; Hall, Helen; Fernando, Shavi; Hobson, Sebastian; Larmour, Luke; Dekoninck, Philip; Wallace, Euan M

2018-02-01

Simulation-based programs are increasingly being used to teach obstetrics and gynaecology examinations, but it is difficult to establish student learning acquired through them. Assessment may test student learning but its role in learning itself is rarely recognised. We undertook this study to assess medical and midwifery student learning through a simulation program using a pre-test and post-test design and also to evaluate use of assessment as a method of learning. The interprofessional simulation education program consisted of a brief pre-reading document, a lecture, a video demonstration and a hands-on workshop. Over a 24-month period, 405 medical and 104 midwifery students participated in the study and were assessed before and after the program. Numerical data were analysed using paired t-test and one-way analysis of variance. Students' perceptions of the role of assessment in learning were qualitatively analysed. The post-test scores were significantly higher than the pre-test (P < 0.001) with improvements in scores in both medical and midwifery groups. Students described the benefit of assessment on learning in preparation of the assessment, reinforcement of learning occurring during assessment and reflection on performance cementing previous learning as a post-assessment effect. Both medical and midwifery students demonstrated a significant improvement in their test scores and for most students the examination process itself was a positive learning experience. © 2017 The Royal Australian and New Zealand College of Obstetricians and Gynaecologists.

An Assessment of the Armed Forces of the Philippines Modernization Program

DTIC Science & Technology

2007-12-14

Sayaff Group BAC Bids and Awards Committee BCDA Bases Conversion and Development Authority BEP Bid Evaluation Plan BIR Bureau of Internal Revenue...contractor execute the requirements of the contract with regard to cost, schedule and performance (Sammet and Green 1990, 136) Bid Evaluation Plan or BEP is...adopted in the evaluation and assessment of bidders’ statements and the method and criteria for the evaluation and assessment of bids. The BEP also

[Documenting a rehabilitation program using a logic model: an advantage to the assessment process].

PubMed

Poncet, Frédérique; Swaine, Bonnie; Pradat-Diehl, Pascale

2017-03-06

The cognitive and behavioral disorders after brain injury can result in severe limitations of activities and restrictions of participation. An interdisciplinary rehabilitation program was developed in physical medicine and rehabilitation at the Pitié-Salpêtriere Hospital, Paris, France. Clinicians believe this program decreases activity limitations and improves participation in patients. However, the program’s effectiveness had never been assessed. To do this, we had to define/describe this program. However rehabilitation programs are holistic and thus complex making them difficult to describe. Therefore, to facilitate the evaluation of complex programs, including those for rehabilitation, we illustrate the use of a theoretical logic model, as proposed by Champagne, through the process of documentation of a specific complex and interdisciplinary rehabilitation program. Through participatory/collaborative research, the rehabilitation program was analyzed using three “submodels” of the logic model of intervention: causal model, intervention model and program theory model. This should facilitate the evaluation of programs, including those for rehabilitation.

Defining a staged-based process for economic and financial evaluations of mHealth programs.

PubMed

LeFevre, Amnesty E; Shillcutt, Samuel D; Broomhead, Sean; Labrique, Alain B; Jones, Tom

2017-01-01

Mobile and wireless technology for health (mHealth) has the potential to improve health outcomes by addressing critical health systems constraints that impede coverage, utilization, and effectiveness of health services. To date, few mHealth programs have been implemented at scale and there remains a paucity of evidence on their effectiveness and value for money. This paper aims to improve understanding among mHealth program managers and key stakeholders of how to select methods for economic evaluation (comparative analysis for determining value for money) and financial evaluation (determination of the cost of implementing an intervention, estimation of costs for sustaining or expanding an intervention, and assessment of its affordability). We outline a 6 stage-based process for selecting and integrating economic and financial evaluation methods into the monitoring and evaluation of mHealth solutions including (1) defining the program strategy and linkages with key outcomes, (2) assessment of effectiveness, (3) full economic evaluation or partial evaluation, (4) sub-group analyses, (5) estimating resource requirements for expansion, (6) affordability assessment and identification of models for financial sustainability. While application of these stages optimally occurs linearly, finite resources, limited technical expertise, and the timing of evaluation initiation may impede this. We recommend that analysts prioritize economic and financial evaluation methods based on programmatic linkages with health outcomes; alignment with an mHealth solution's broader stage of maturity and stage of evaluation; overarching monitoring and evaluation activities; stakeholder evidence needs; time point of initiation; and available resources for evaluations.

What follows newborn screening? An evaluation of a residential education program for parents of infants with newly diagnosed cystic fibrosis.

PubMed

Sawyer, Susan M; Glazner, Judith A

2004-08-01

The diagnosis of a severe life-limiting condition, such as cystic fibrosis (CF), is generally followed by assessment and treatment of the child and education and counseling for parents. The introduction of newborn screening for CF provides an opportunity for standardized assessment and education. The aim of this study was to evaluate a 5-day residential assessment and education program for parents of infants who receive a diagnosis of CF after newborn screening. Eligible parents had a 6- to 30-month-old infant with CF diagnosed by newborn screening. Parents were interviewed by telephone using a structured questionnaire that addressed 3 main themes: 1) initial communication of the diagnosis of CF, 2) the perceived value of the 5-day assessment and education program, and 3) the perceived advantages and disadvantages of the residential component (Care-By-Parent unit) of the program. Fifteen of 17 eligible families took part in the 5-day assessment and education program, 12 of whom used the residential Care-By-Parent unit. At the end of the program, parents believed that they had the knowledge and skills required to manage their child's CF at home. One hundred percent endorsed the timing of the assessment and education program immediately after the child's diagnosis and would recommend it to other families in the same situation. Perceived advantages of the residential program were not having to travel (89%), being able to concentrate on CF (50%), and the benefit of a "home base" at the hospital (39%). Twenty-two percent reported that financial costs related to participation (paternal time off work) were a disadvantage, 17% reported additional strain on family members caring for siblings, and 17% mentioned lack of comfort within the unit. This time-intensive residential program was evaluated positively by parents of children with newly diagnosed CF. It provides a model for education programs after the diagnosis of CF by newborn screening, as well as for other pediatric conditions that require intensive parent education.

An Analysis of Individual Teachers' Development of Instruction Based on ClassScape Program Data

ERIC Educational Resources Information Center

Parker, Jason L.

2011-01-01

This dissertation was designed to examine and assess the effectiveness of the ClassScape formative assessment tool on the planning, implementation, and evaluation of instruction at a rural middle school in western North Carolina. The teachers had the ClassScape program for 3 years, but were not using the program to plan future instruction. The…

What Do Students Learn when We Teach Peace? A Qualitative Assessment of a Theater Peace Program

ERIC Educational Resources Information Center

Duckworth, Cheryl Lynn; Allen, Barb; Williams, Teri Triguba

2012-01-01

This is a qualitative assessment of a theater arts peace education program for high-school students. We present the results of qualitative interviews with students who participated in a peace education program. They tell us in their own words what they believe they learned. Given that most peace education evaluation is quantitative or focuses on…

How to Recognize Success and Failure: Practical Assessment of an Evolving, First-Semester Laboratory Program Using Simple, Outcome-Based Tools

ERIC Educational Resources Information Center

Gron, Liz U.; Bradley, Shelly B.; McKenzie, Jennifer R.; Shinn, Sara E.; Teague, M. Warfield

2013-01-01

This paper presents the use of simple, outcome-based assessment tools to design and evaluate the first semester of a new introductory laboratory program created to teach green analytical chemistry using environmental samples. This general chemistry laboratory program, like many introductory courses, has a wide array of stakeholders within and…

Communications Arts and Technology Program Needs Assessment.

ERIC Educational Resources Information Center

Oakland Community Coll., Farmington, MI. Office of Institutional Planning and Analysis.

In 1992, Oakland Community College conducted a needs assessment study to assist in reviewing and evaluating proposed changes to the college's existing Communications Arts and Technology Program. Businesses and employers in the communications field were surveyed (n=40) regarding employment requirements and opportunities. Additional information was…

ENVIRONMENTAL MONITORING AND ASSESSMENT PROGRAM-SURFACE WATERS WESTERN PILOT STUDY: FIELD OPERATIONS MANUAL FOR WADEABLE STREAMS

EPA Science Inventory

This document describes field procedures that were used during the Environmental Monitoring and Assessment Program (EMAP) Western Pilot Study, conducted from 1999 through 2004. Objectives for EMAP involve developing appropriate scientific and technical tools for evaluating ecolo...

A program needs-driven approach to selecting dietary assessment methods for decision-making in food fortification programs.

PubMed

Coates, Jennifer; Colaiezzi, Brooke; Fiedler, John L; Wirth, James; Lividini, Keith; Rogers, Beatrice

2012-09-01

Dietary assessment data are essential for designing, monitoring, and evaluating food fortification and other food-based nutrition programs. Planners and managers must understand the validity, usefulness, and cost tradeoffs of employing alternative dietary assessment methods, but little guidance exists. To identify and apply criteria to assess the tradeoffs of using alternative dietary methods for meeting fortification programming needs. Twenty-five semistructured expert interviews were conducted and literature was reviewed for information on the validity, usefulness, and cost of using 24-hour recalls, Food Frequency Questionnaires/Fortification Rapid Assessment Tool (FFQ/FRAT), Food Balance Sheets (FBS), and Household Consumption and Expenditures Surveys (HCES) for program stage-specific information needs. Criteria were developed and applied to construct relative rankings of the four methods. Needs assessment: HCES offers the greatest suitability at the lowest cost for estimating the risk of inadequate intakes, but relative to 24-hour recall compromises validity. HCES should be used to identify vehicles and to estimate coverage and likely impact due to its low cost and moderate-to-high validity. Baseline assessment: 24-hour recall should be applied using a representative sample. Monitoring: A simple, low-cost FFQ can be used to monitor coverage. Impact evaluation: 24-hour recall should be used to assess changes in nutrient intakes. FBS have low validity relative to other methods for all programmatic purposes. Each dietary assessment method has strengths and weaknesses that vary by context and purpose. Method selection must be driven by the program's data needs, the suitability of the methods for the purpose, and a clear understanding of the tradeoffs involved.

The Assessment, Development, Assurance Pharmacist's Tool (ADAPT) for ensuring quality implementation of health promotion programs.

PubMed

Truong, Hoai-An; Taylor, Catherine R; DiPietro, Natalie A

2012-02-10

To develop and validate the Assessment, Development, Assurance Pharmacist's Tool (ADAPT), an instrument for pharmacists and student pharmacists to use in developing and implementing health promotion programs. The 36-item ADAPT instrument was developed using the framework of public health's 3 core functions (assessment, policy development, and assurance) and 10 essential services. The tool's content and usage was assessed and conducted through peer-review and initial validity testing processes. Over 20 faculty members, preceptors, and student pharmacists at 5 institutions involved in planning and implementing health promotion initiatives reviewed the instrument and conducted validity testing. The instrument took approximately 15 minutes to complete and the findings resulted in changes and improvements to elements of the programs evaluated. The ADAPT instrument fills a need to more effectively plan, develop, implement, and evaluate pharmacist-directed public health programs that are evidence-based, high-quality, and compliant with laws and regulations and facilitates documentation of pharmacists' contributions to public health.

Economic evaluation of drug abuse treatment and HIV prevention programs in pregnant women: a systematic review.

PubMed

Ruger, Jennifer Prah; Lazar, Christina M

2012-01-01

Drug abuse and transmission of HIV during pregnancy are public health problems that adversely affect pregnant women, their children and surrounding communities. Programs that address this vulnerable population have the ability to be cost-effective due to resulting cost savings for mother, child and society. Economic evaluations of programs that address these issues are an important tool to better understand the costs of services and create sustainable healthcare systems. This study critically examined economic evaluations of drug abuse treatment and HIV prevention programs in pregnant women. A systematic review was conducted using the criteria recommended by the Panel on Cost-Effectiveness in Health and Medicine and the British Medical Journal (BMJ) checklist for economic evaluations. The search identified 6 economic studies assessing drug abuse treatment for pregnant women, and 12 economic studies assessing programs that focus on prevention of mother-to-child transmission (PMTCT) of HIV. Results show that many programs for drug abuse treatment and PMTCT among pregnant women are cost-effective or even cost-saving. This study identified several shortcomings in methodology and lack of standardization of current economic evaluations. Efforts to address methodological challenges will help make future studies more comparable and have more influence on policy makers, clinicians and the public. Copyright © 2011 Elsevier Ltd. All rights reserved.

An Evaluation of Therapeutic Horseback Riding Programs for Adults with Physical Impairments.

ERIC Educational Resources Information Center

Farias-Tomaszewski, Sarah; Jenkins, Sharon Rae; Keller, Jean

2001-01-01

Assessed the outcome of a therapeutic horseback riding program for adults with physical disabilities. A one-group, pre- test/post-test design was used to evaluate changes on level of physical and global self-efficacy. Behavioral indices of self-confidence were also evaluated. Physical self-efficacy and behavioral self-confidence increased from…

Day Care in Caracas: A Day Care Homes Program Evaluation Report. Volume I: Executive Summary.

ERIC Educational Resources Information Center

de Ruesta, Maria Carlota; de Vidal, Amalia Barrios

This document provides a summary of a formative evaluation research project concerning the neighborhood day care homes program in Caracas, Venezuela. The evaluation included nine lines of study: (1) an assessment of sociodemographic conditions of Venezuelan preschool age children, legal and employment status of Venezuelan women, and general social…

Evaluating Vocational Educators' Training Programs: A Kirkpatrick-Inspired Evaluation Model

ERIC Educational Resources Information Center

Ravicchio, Fabrizio; Trentin, Guglielmo

2015-01-01

The aim of the article is to describe the assessment model adopted by the SCINTILLA Project, a project in Italy aimed at the online vocational training of young, seriously-disabled subjects and their subsequent work inclusion in smart-work mode. It will thus describe the model worked out for evaluation of the training program conceived for the…

Pilot Program on Common Status Measures Objective-Referenced Tests. Colorado Evaluation Project, Report No. 1.

ERIC Educational Resources Information Center

Colorado State Dept. of Education, Denver.

The purpose of the Colorado Evaluation Project was to field test the Common Status Measures at grades four and eleven in conjunction with a statewide assessment program based on objective-referenced testing instruments developed by the Colorado Department of Education for grades kindergarten, three, six, nine, and twelve. The evaluation was…

North Carolina State Supported Early Childhood Demonstration Centers. Second Annual Evaluation: 1970-1971.

ERIC Educational Resources Information Center

Learning Inst. of North Carolina, Durham.

The second year of a state supported early childhood education program was evaluated, using a pre-post evaluation design involving experimental and control groups. An Early Childhood Assessment Battery was administered to 720 five-year-olds enrolled in the kindergarten program by their teachers. The control group (178) were tested locally. Results…

Assessment and Evaluation in Adapted Physical Education.

ERIC Educational Resources Information Center

Kennedy, Charlotte I.; Bundschuh, Ernie

Intended for physical education teachers, the booklet describes informal and formal methods for evaluating handicapped children to determine whether they can participate in a regular physical education program with nonhandicapped students, in a regular physical education program with modification, or in a specially designed physical education…

Evaluation of an Early Childhood Preschool Program in Rural Bangladesh

ERIC Educational Resources Information Center

Aboud, Frances E.

2006-01-01

A preschool program in rural Bangladesh was evaluated in terms of cognitive and social outcomes of children. The preschools provided a half-day program, 6 days a week, with free play, stories, and instruction in literacy and math. Four hundred children between 4.5 and 6.5 years were assessed, half in preschools and half in villages where there…

School-Based Smoking Prevention Programs for Adolescents in South Korea: A Systematic Review

ERIC Educational Resources Information Center

Park, Eunok

2006-01-01

The number of research papers evaluating programs designed to prevent adolescent smoking have increased in the last 13 years in Korea. The purpose of this study was to evaluate these programs, to review the features of the studies and to systemically assess the results on the knowledge about, and attitude to, smoking and smoking behavior. Database…

A Large Sample Evaluation of a Court-Mandated Batterer Intervention Program: Investigating Differential Program Effect for African American and Caucasian Men

ERIC Educational Resources Information Center

Buttell, Frederick P.; Carney, Michelle Mohr

2006-01-01

Objective: The purpose of the present study was to (a) evaluate a 26-week batterer intervention program by investigating changes in psychological variables related to abuse (i.e., truthfulness, violence, lethality, control, alcohol use, drug use, and stress coping abilities) between pretreatment and posttreatment assessments in a large sample of…

Promoting Positive Family Interactions: Evaluating a Free Early Childhood Book Distribution Program

ERIC Educational Resources Information Center

Funge, Simon P.; Sullivan, Dana J.; Tarter, Kirsten

2017-01-01

The Dolly Parton Imagination Library (DPIL) program encourages reading among families of preschool children by mailing age-appropriate books, once per month, until the child reaches the age of five. An evaluation of a DPIL program in a southern state in the U.S. was conducted to assess the impact on enrolled children. Focus groups were conducted…

Methods for evaluating a mature substance abuse prevention/early intervention program.

PubMed

Becker, L R; Hall, M; Fisher, D A; Miller, T R

2000-05-01

The authors describe methods for work in progress to evaluate four workplace prevention and/or early intervention programs designed to change occupational norms and reduce substance abuse at a major U.S. transportation company. The four programs are an employee assistance program, random drug testing, managed behavioral health care, and a peer-led intervention program. An elaborate mixed-methods evaluation combines data collection and analysis techniques from several traditions. A process-improvement evaluation focuses on the peer-led component to describe its evolution, document the implementation process for those interested in replicating it, and provide information for program improvement. An outcome-assessment evaluation examines impacts of the four programs on job performance measures (e.g., absenteeism, turnover, injury, and disability rates) and includes a cost-offset and employer cost-savings analysis. Issues related to using archival data, combining qualitative and quantitative designs, and working in a corporate environment are discussed.

Evaluation of a community-based, family focused healthy weights initiative using the RE-AIM framework.

PubMed

Jung, Mary E; Bourne, Jessica E; Gainforth, Heather L

2018-01-26

Childhood overweight and obesity is a major public health concern. Community-based interventions have the potential to reach caregivers and children. However, the overall health impact of these programs is rarely comprehensively assessed. This study evaluated a physical activity and healthy eating family program (Healthy Together; HT) using the RE-AIM framework. Ten sites implemented the 5-week program. Thirty-nine staff members and 277 program participants (126 caregivers [M age  = 35.6] and 151 children [M age  = 13]) participated in the evaluation. Each RE-AIM dimension was assessed independently using a mixed-methods approach. Sources of data included archival records, interviews and surveys. Effectiveness outcome variables were assessed at pre- and post-intervention and 6-month follow-up. Reach: HT participants were almost entirely recruited from existing programs within sites. Effectiveness: Caregivers' nutrition related efficacy beliefs increased following HT (ps < .03). Participation in HT was not associated with significant changes in physical activity or nutrition behaviour or perceived social support (ps > .05). Knowledge surrounding healthy diets and physical activity increased in children and caregivers (ps < .05). Adoption: Thirty-five percent of sites approached to implement HT expressed interest. The 10 sites selected recruited existing staff members to implement HT. Program objectives were met 72.8% of the time and 71 adaptations were made. HT was finance- and time-dependent. Maintenance: Two sites fully implemented HT in the follow-up year and 5 sites incorporated aspects of HT into other programs. Working alongside organizations that develop community programs to conduct comprehensive, arms-length evaluations can systematically highlight areas of success and challenges. Overall HT represents a feasible community-based intervention; however further support is required in order to ensure the program is effective at positively targeting the desired outcomes. As a result of this evaluation, modifications are currently being implemented to HT.

Residents-as-Teachers Publications: What Can Programs Learn From the Literature When Starting a New or Refining an Established Curriculum?

PubMed

Bree, Kelly K; Whicker, Shari A; Fromme, H Barrett; Paik, Steve; Greenberg, Larrie

2014-06-01

Teaching residents how to teach is a critical part of resident education because residents are often the major teachers of medical students. The importance of formal residents-as-teachers (RAT) curricula has been emphasized throughout the literature, yet not all residency programs have such a curriculum in place. The purpose of our study was to (1) review the medical education literature for established RAT curricula, (2) assess published curricula's reproducibility, (3) evaluate the type of outcomes achieved using the Kirkpatrick model of evaluation, and (4) identify curricula that training programs could feasibly adopt. We performed a literature review using PubMed, Medline, Scopus, PsycINFO, ERIC, and Embase. Key search words included residents, residents as teachers, teaching, internship and residency, and curriculum. In addition, a search of MedEdPORTAL was performed using the same key terms. Articles were evaluated based on the reproducibility of curricula and the assessment tools. Evaluation of educational outcomes was performed using the Kirkpatrick model. Thirty-nine articles were deemed appropriate for review. Interventions and evaluation techniques varied greatly. Only 1 article from the literature was deemed to have both curricula and assessments that would be fully reproducible by other programs. A literature review on RAT curricula found few articles that would be easily reproduced for residency programs that want to start or improve their own RAT curricula. It also demonstrated the difficulty and lack of rigorous outcome measurements for most curricula.

Assessment of the NASA AvSTAR Project Plan

NASA Technical Reports Server (NTRS)

Ulrey, Michael L.; Haraldsdottir, Aslaug; Berge, Matthew E.; Hopperstad, Craig A.; Schwab, Robert W.

2004-01-01

This report is a preliminary evaluation of NASA's proposed Aviation System Technology Advanced Research (AvSTAR) Program during the early stages of its definition, in the first half of the year 2001. This evaluation focuses on how well the program goals address the needs of the U.S. National Airspace System, the technical feasibility of the program goals, and the logistical feasibility of the program plan. This report also provides recommendations on how the AvSTAR program could be strengthened and improved. This document has two appendices.

Assessing the effects of regional payment for watershed services program on water quality using an intervention analysis model.

PubMed

Lu, Yan; He, Tian

2014-09-15

Much attention has been recently paid to ex-post assessments of socioeconomic and environmental benefits of payment for ecosystem services (PES) programs on poverty reduction, water quality, and forest protection. To evaluate the effects of a regional PES program on water quality, we selected chemical oxygen demand (COD) and ammonia-nitrogen (NH3-N) as indicators of water quality. Statistical methods and an intervention analysis model were employed to assess whether the PES program produced substantial changes in water quality at 10 water-quality sampling stations in the Shaying River watershed, China during 2006-2011. Statistical results from paired-sample t-tests and box plots of COD and NH3-N concentrations at the 10 stations showed that the PES program has played a positive role in improving water quality and reducing trans-boundary water pollution in the Shaying River watershed. Using the intervention analysis model, we quantitatively evaluated the effects of the intervention policy, i.e., the watershed PES program, on water quality at the 10 stations. The results suggest that this method could be used to assess the environmental benefits of watershed or water-related PES programs, such as improvements in water quality, seasonal flow regulation, erosion and sedimentation, and aquatic habitat. Copyright © 2014 Elsevier B.V. All rights reserved.

Are we making the grade? Practices and reported efficacy measures of primate conservation education programs.

PubMed

Kling, Katherine J; Hopkins, Mariah E

2015-04-01

Conservation education is often employed alongside primate conservation efforts with the aim of changing knowledge, attitudes, and behaviors toward non-human primates. Recommended best-use practices include longevity, use of program incentives, collaboration among educators, and adaptive program assessment, among others. This study surveys primate conservation education programs (PCEPs) to assess the frequency of suggested best-use practices, and to investigate impacts on program efficacy. Online surveys were collected from PCEPs in 2013-2014 (N = 43). The majority of programs reported lengths of 5-10 years, with participant involvement ranging widely from a day to several years. Non-economic and economic incentives were distributed by approximately half of all programs, with programs that provided economic incentives reporting positive participant attitude changes more frequently than those that did not (P = 0.03). While >70% of PCEPs consulted with community leaders, local teachers, and research scientists, only 45.9% collaborated with other conservation educators and only 27% collaborated with cultural experts such as cultural anthropologists. Programs that collaborated with other conservation educators were more likely to report reductions in threats to primates, specifically to bushmeat hunting and capture of primates for the pet trade (P = 0.07). Formal program evaluations were employed by 72.1% of all programs, with the majority of programs using surveys to assess changes to participant attitudes and knowledge. Formal evaluations of participant behavior, community attitudes and behaviors, and threats to primate populations were less common. While results indicate that PCEPs follow many suggested best-use practices, program impacts may be enhanced by greater discussion of economic incentivization, increased collaboration between conservation educators, and improved commitment to adaptive evaluation of changes to behaviors in addition to attitudes and knowledge. © 2014 Wiley Periodicals, Inc.

Developing a framework for assessing responsible conduct of research education programs.

PubMed

Olson, Lynne E

2010-03-01

Education in the responsible conduct of research (RCR) in the United States has evolved over the past decade from targeting trainees to including educational efforts aimed at faculty and staff. In addition RCR education has become more focused as federal agencies have moved to recommend specific content and to mandate education in certain areas. RCR education has therefore become a research-compliance issue necessitating the development of policies and the commitment of resources to develop or expand systems for educating faculty and staff and for assuring compliance. These changes implied the need to develop a program evaluation model that could be applied to institutional RCR education programs, which were expected to differ from traditional academic credit-bearing courses targeting trainees. Information gleaned from the examination of corporate compliance models was analyzed in order to create a program evaluation module that could be used to document and assess educational programs focused on teaching RCR. A programmed series of questions for each of the nine RCR content areas identified by the United States Office of Research Integrity was created based on a performance-monitoring evaluation model. The questions focus on educational goals, resources provided to support the educational efforts, educational content, content delivery, educational outcomes, compliance requirements and feedback. Answers collected in response to the questions could be used to both document and continually improve the quality of RCR educational programs through on-going formative assessment and feedback.

10 CFR 850.21 - Hazard assessment.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 10 Energy 4 2010-01-01 2010-01-01 false Hazard assessment. 850.21 Section 850.21 Energy DEPARTMENT OF ENERGY CHRONIC BERYLLIUM DISEASE PREVENTION PROGRAM Specific Program Requirements § 850.21 Hazard... with the greatest risks of exposure are evaluated first. (b) The responsible employer must ensure that...

Interpersonal Skills Training: Evaluation of a Program with Adult Male Offenders.

ERIC Educational Resources Information Center

Bornstein, Philip H.; And Others

1979-01-01

To assess the efficacy of an interpersonal skill training program, adult offenders were randomly assigned to either interpersonal effectiveness training or waiting-list control. Results indicated interpersonal effectiveness training group superiority on Interpersonal Behavior Role-Play Test training and generalization assessment items. Findings…

76 FR 39394 - Notice of Proposed Information Collection Requests

Federal Register 2010, 2011, 2012, 2013, 2014

2011-07-06

... within the U.S. Department of Education (ED) has contracted with Decision Information Resources, Inc. and Mathematica Policy Research, Inc. to assess the procedures for collecting and reporting program performance and evaluation data for eleven ED grant programs. These audits and assessments will provide ED with...

IMPROVING ESTUARINE EVALUATION THROUGH OUTREACH AND TECHNOLOGY TRANSFER TO STATES, TRIBES AND OTHER PARTNERS: EPA'S NATIONAL COASTAL ASSESSMENT

EPA Science Inventory

The Environmental Protection Agency's (EPA's) National Coastal Assessment (NCA) is a large-scale, comprehensive environmental monitoring program designed to characterize the ecological condition of the Nation's coastal resources. A key to this successful program is the developmen...

Clinical considerations for an infant oral health care program.

PubMed

Ramos-Gomez, Francisco J

2005-05-01

The American Academy of Pediatric Dentistry and the American Association of Pediatrics recommend dental assessments and evaluations for children during their first year of life. Early dental intervention evaluates a child's risk status based on parental interviews and oral examinations. These early screenings present an opportunity to educate parents about the medical, dental, and cost benefits of preventive--rather than restorative-care and may be more effective in reducing early childhood caries than traditional infectious disease models. A comprehensive infant oral care program includes: (1) risk assessments at regularly scheduled dental visits; (2) preventive treatments such as fluoride varnishes or sealants; (3) parental education on the correct methods to clean the baby's mouth; and (4) incentives to encourage participation in ongoing educational programming. Recruiting mothers during pregnancy improves the likelihood that they will participate in the assessment program. To maximize interest, trust, and success among participating parents, educational and treatment programs must be tailored to the social and cultural norms within the community being served.

An Evaluation of the Effectiveness of U.S. Naval Aviation Crew Resource Management Training Programs: A Reassessment for the Twenty-First Century Operating Environment

DTIC Science & Technology

2009-06-01

3. Previous Navy CRM Assessments ....................................................24 4. Applying Kirkpatrick’s Topology of Evaluation...development within each aviation community. Kirkpatrick’s (1976) hierarchy of training evaluation technique was applied to examine three levels of... Applying methods and techniques used in previous CRM evaluation research, this thesis provided an updated evaluation of the Naval CRM program to fill

Evaluating Psychiatry Residents as Physician-Managers: Development of an Assessment Tool

ERIC Educational Resources Information Center

Sockalingam, Sanjeev; Stergiopoulos, Vicky; Maggi, Julie D.; Zaretsky, Ari; Stovel, Laura; Hodges, Brian

2013-01-01

Objectives: With the emergence of physician-manager (PM) curricula in medical education, more effective assessment tools are needed to evaluate psychiatry trainees in this role. The aim of this study was to determine psychiatry residents', program directors', and PM educators' perceptions about PM role-assessment. Methods: Psychiatry residents at…

Evaluation of a preschool nutrition education program based on the theory of multiple intelligences.

PubMed

Cason, K L

2001-01-01

This report describes the evaluation of a preschool nutrition education program based on the theory of multiple intelligences. Forty-six nutrition educators provided a series of 12 lessons to 6102 preschool-age children. The program was evaluated using a pretest/post-test design to assess differences in fruit and vegetable identification, healthy snack choices, willingness to taste foods, and eating behaviors. Subjects showed significant improvement in food identification and recognition, healthy snack identification, willingness to taste foods, and frequency of fruit, vegetable, meat, and dairy consumption. The evaluation indicates that the program was an effective approach for educating preschool children about nutrition.

Evaluating the Implementation of an Olympic Education Program in Greece

NASA Astrophysics Data System (ADS)

Grammatikopoulos, Vasilios; Tsigilis, Nikolaos; Koustelios, Athanasios; Theodorakis, Yannis

2005-11-01

The aim of this study was to develop an instrument for evaluating how an education program has been implemented. Such evaluation can provide insight into the effectiveness of a program. Examined here was the Olympic Education Program used in Greek schools since 2000. In it, students learn the history of the Olympic games and the importance of exercise for health along with the principles and values of sports and volunteerism. The evaluation instrument underlying this study addressed the following six factors: `facilities', `administration', `educational material', `student-teacher relationships', `educational procedures', and `training'. Results indicate that the instrument, while adequate for assessing effectiveness, should be combined with advanced statistical methods.

Evaluation of an Educational Program to Improve School Nursing Staff Perceptions of Bullying In Pinellas County, Florida.

PubMed

Salmeron, Patricia A; Christian, Becky J

2016-01-01

The purpose of this project was to determine if a bullying educational program for school nurses and certified nursing assistants/health technicians (CNAs/HTs) would increase knowledge of bullying, probability of reporting a bully, and probability of assisting a bullied victim. This educational program and evaluation employed a retrospective, post-then-pre-test design. Instruments used included a 17-item demographic questionnaire and the 12-item Reduced Aggression/ Victimization Scale Bullying Assessment Tool (BAT), a 5-point Likert Scale de - signed to assess school nurses’ and CNAs’/HTs’ understanding of bullying, the probability of reporting bullies, and the probability of assisting bullied victims before and after the educational presentation. Findings of this educational evaluation program indicated that the majority of school nurses and CNAs/HTs had an increased understanding of bullying, higher probability of reporting a bully, and assisting a bullied victim after the presentation.

Evaluation of Athletic Training Students' Clinical Proficiencies

PubMed Central

Walker, Stacy E; Weidner, Thomas G; Armstrong, Kirk J

2008-01-01

Context: Appropriate methods for evaluating clinical proficiencies are essential in ensuring entry-level competence. Objective: To investigate the common methods athletic training education programs use to evaluate student performance of clinical proficiencies. Design: Cross-sectional design. Setting: Public and private institutions nationwide. Patients or Other Participants: All program directors of athletic training education programs accredited by the Commission on Accreditation of Allied Health Education Programs as of January 2006 (n  =  337); 201 (59.6%) program directors responded. Data Collection and Analysis: The institutional survey consisted of 11 items regarding institutional and program demographics. The 14-item Methods of Clinical Proficiency Evaluation in Athletic Training survey consisted of respondents' demographic characteristics and Likert-scale items regarding clinical proficiency evaluation methods and barriers, educational content areas, and clinical experience settings. We used analyses of variance and independent t tests to assess differences among athletic training education program characteristics and the barriers, methods, content areas, and settings regarding clinical proficiency evaluation. Results: Of the 3 methods investigated, simulations (n  =  191, 95.0%) were the most prevalent method of clinical proficiency evaluation. An independent-samples t test revealed that more opportunities existed for real-time evaluations in the college or high school athletic training room (t189  =  2.866, P  =  .037) than in other settings. Orthopaedic clinical examination and diagnosis (4.37 ± 0.826) and therapeutic modalities (4.36 ± 0.738) content areas were scored the highest in sufficient opportunities for real-time clinical proficiency evaluations. An inadequate volume of injuries or conditions (3.99 ± 1.033) and injury/condition occurrence not coinciding with the clinical proficiency assessment timetable (4.06 ± 0.995) were barriers to real-time evaluation. One-way analyses of variance revealed no difference between athletic training education program characteristics and the opportunities for and barriers to real-time evaluations among the various clinical experience settings. Conclusions: No one primary barrier hindered real-time clinical proficiency evaluation. To determine athletic training students' clinical proficiency for entry-level employment, athletic training education programs must incorporate standardized patients or take a disciplined approach to using simulation for instruction and evaluation. PMID:18668172

Evaluation of Arizona Health Care Cost Containment System, 1984-85

PubMed Central

McCall, Nelda; Henton, Douglas; Haber, Susan; Paringer, Lynn; Crane, Michael; Wrightson, William; Freund, Deborah

1987-01-01

In this article, we describe the evaluation of the Arizona Health Care Cost Containment System (AHCCCS), Arizona's alternative to the acute care portion of Medicaid. We provide an assessment of implementation of the program's innovative features during its second 18 months of operation, from April 1984 through September 1985. Included in the evaluation are assessments of the administration of the program, provider relations, eligibility, enrollment and marketing, information systems, quality assurance and member satisfaction activities, the relationship of the county governments to AHCCCS, the competitive bidding process, and the plans and their financial status. PMID:10312395

A Descriptive Study of Nursing Peer-Review Programs in US Magnet® Hospitals.

PubMed

Roberts, Holli; Cronin, Sherill Nones

2017-04-01

The goal of this study was to assess the types of nursing peer review (NPR) programs in US Magnet® organizations. The 2 most predominant models of NPR programs in the literature are performance evaluation and clinical peer review. The literature on clinical peer review is primarily descriptive, outlining structures and anecdotal outcomes. Participants from hospitals holding Magnet recognition were selected using a stratified random-sampling method. A survey developed by the researchers assessed the presence of NPR. If clinical NPR was in place, program design, evaluation measurements, and barriers were explored. Findings suggest wide variability in NPR models. More than one-third of the respondents conduct peer evaluation as the only mechanism of NPR. Most hospitals with a clinical peer-review program reported a case review structure and process measurements not supported by data. The variations noted in this study suggest more research is needed to measure the effectiveness of NPR models and associated outcomes.

The Ergonomic Program Implementation Continuum (EPIC): integration of health and safety--a process evaluation in the healthcare sector.

PubMed

Baumann, Andrea; Holness, D Linn; Norman, Patrica; Idriss-Wheeler, Dina; Boucher, Patricia

2012-07-01

This article presents a health and safety intervention model and the use of process evaluation to assess a participatory ergonomic intervention. The effectiveness of the Ergonomic Program Implementation Continuum (EPIC) was assessed at six healthcare pilot sites in Ontario, Canada. The model provided a framework to demonstrate evaluation findings. Participants reported that EPIC was thorough and identified improvements related to its use. Participants believed the program contributed to advancing an organizational culture of safety (COS). Main barriers to program uptake included resistance to change and need for adequate funding and resources. The dedication of organizational leaders and consultant coaches was identified as essential to the program's success. In terms of impact on industry, findings contribute to the evidence-based knowledge of health and safety interventions and support use of the framework for creating a robust infrastructure to advance organizational COS and link staff safety and wellness with patient safety in healthcare. Copyright © 2012 National Safety Council and Elsevier Ltd. All rights reserved.

Evaluating a complex, multi-site, community-based program to improve healthcare quality: the summative research design for the Aligning Forces for Quality initiative.

PubMed

Scanlon, Dennis P; Wolf, Laura J; Alexander, Jeffrey A; Christianson, Jon B; Greene, Jessica; Jean-Jacques, Muriel; McHugh, Megan; Shi, Yunfeng; Leitzell, Brigitt; Vanderbrink, Jocelyn M

2016-08-01

The Aligning Forces for Quality (AF4Q) initiative was the Robert Wood Johnson Foundation's (RWJF's) signature effort to increase the overall quality of healthcare in targeted communities throughout the country. In addition to sponsoring this 16-site complex program, RWJF funded an independent scientific evaluation to support objective research on the initiative's effectiveness and contributions to basic knowledge in 5 core programmatic areas. The research design, data, and challenges faced during the summative evaluation phase of this near decade-long program are discussed. A descriptive overview of the summative research design and its development for a multi-site, community-based, healthcare quality improvement initiative is provided. The summative research design employed by the evaluation team is discussed. The evaluation team's summative research design involved a data-driven assessment of the effectiveness of the AF4Q program at large, assessments of the impact of AF4Q in the specific programmatic areas, and an assessment of how the AF4Q alliances were positioned for the future at the end of the program. The AF4Q initiative was the largest privately funded community-based healthcare improvement initiative in the United States to date and was implemented at a time of rapid change in national healthcare policy. The implementation of large-scale, multi-site initiatives is becoming an increasingly common approach for addressing problems in healthcare. The summative evaluation research design for the AF4Q initiative, and the lessons learned from its approach, may be valuable to others tasked with evaluating similarly complex community-based initiatives.

Comprehensive Monitoring Program: Air Quality Data Assessment Report for FY90. Volume 2. Version 3.1

DTIC Science & Technology

1991-09-01

91311R01 If VERSION 3.10) VOLUME II Comm 2ND COPY COMPREHENSIVE MONITORING PROGRAM Contract Number DAAAI5-87-0095 AIR QUALITY DATA ASSESSMENT REPORT...MONITORING PROGRAM. FINAL AIR QUALITY DATA ASSESSMENT REPORT FOR FY90, VERSION 3.1 NONE 6. AUTHOR(S) 7. PERFORMING ORGANIZATION NAME(S) AND ADDRES.S(S) 8...RELEASE; DISTRIBUTION IS UNLIMITED 13. ABSTRACT (Maximum 200 words) THE OBJECTIVE OF THIS CMP IS TO: VERIFY AND EVALUATE POTENTIAL AIR QUALITY HEALTH

Evaluation of Student Engagement Assessment in Colorado State University's Warner College of Natural Resources

ERIC Educational Resources Information Center

Holman, Debra Kaye

2013-01-01

The purpose of this mixed methods study was to conduct a participatory program evaluation of student engagement assessment in Colorado State University's (CSU) Warner College of Natural Resources (WCNR). The college requested the evaluation after completing two pilot studies of undergraduate engagement which led them to consider establishing the…

Formative Evaluation of the ACSC Distance Learning Program: A Status Study.

ERIC Educational Resources Information Center

MacCuish, Donald A.

2001-01-01

Describes a formative evaluation of the Air Command and Staff College (ACSC) distance learning program, which offers professional military education (PME). Highlights include use of the status study methodology; curriculum development; course design; learning theories; instructional systems design; best practices; and student assessment. (LRW)

Evaluating WIC.

ERIC Educational Resources Information Center

Germanis, Peter; Besharov, Douglas J.

2000-01-01

Describes the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) and reviews and assesses research on its impact. Recommends policy reforms and randomized experiments to evaluate the program and any changes made to it. Weaknesses in research to date make the real impacts of WIC unclear. (Contains 79 references.) (SLD)

Training oncology and palliative care clinical nurse specialists in psychological skills: evaluation of a pilot study.

PubMed

Clark, Jane E; Aitken, Susan; Watson, Nina; McVey, Joanne; Helbert, Jan; Wraith, Anita; Taylor, Vanessa; Catesby, Sarah

2015-06-01

National guidelines in the United Kingdom recommend training Clinical Nurse Specialists in psychological skills to improve the assessment and intervention with psychological problems experienced by people with a cancer diagnosis (National Institute for Health and Clinical Excellence, 2004). This pilot study evaluated a three-day training program combined with supervision sessions from Clinical Psychologists that focused on developing skills in psychological assessment and intervention for common problems experienced by people with cancer. Questionnaires were developed to measure participants' levels of confidence in 15 competencies of psychological skills. Participants completed these prior to the program and on completion of the program. Summative evaluation was undertaken and results were compared. In addition, a focus group interview provided qualitative data of participants' experiences of the structure, process, and outcomes of the program. Following the program, participants rated their confidence in psychological assessment and skills associated with providing psychological support as having increased in all areas. This included improved knowledge of psychological theories, skills in assessment and intervention and accessing and using supervision appropriately. The largest increase was in providing psycho-education to support the coping strategies of patients and carers. Thematic analysis of interview data identified two main themes including learning experiences and program enhancements. The significance of the clinical supervision sessions as key learning opportunities, achieved through the development of a community of practice, emerged. Although this pilot study has limitations, the results suggest that a combined teaching and supervision program is effective in improving Clinical Nurse Specialists' confidence level in specific psychological skills. Participants' experiences highlighted suggestions for refinement and development of the program. Opportunities for further research and developments in this area are discussed.

Building an evidence base for community health: a review of the quality of program evaluations.

PubMed

Jolley, Gwyn M; Lawless, Angela P; Baum, Fran E; Hurley, Catherine J; Fry, Denise

2007-11-01

An assessment of the quality of program evaluations conducted in South Australian community health services investigated how effective evaluation reporting is in producing an evidence base for community health. Evaluation reports were assessed by a team of reviewers. Practitioner workshops allowed an understanding of the uses of evaluation and what promotes or acts as a barrier to undertaking evaluations. Community health services do undertake a good deal of evaluation. However, reports were not generally explicit in dealing with the principles that underpin community health. Few engaged with program theory or rationale. Typically, reports were of short-term projects with uncertain futures so there may seem little point in considering issues of long-term health outcomes and transferability to other settings. The most important issue from our study is the lack of investment in applied health services research of the sort that will be required to produce the evidence for practice that policy makers desire. The current lack of evidence for community health reflects failure of the system to invest in research and evaluation that is adequately resourced and designed for complex community settings.

Employer evaluations of nurse graduates: a critical program assessment element.

PubMed

Ryan, M E; Hodson, K E

1992-05-01

Accountability in higher education dictates implementation of a comprehensive evaluation plan. Employer evaluation of graduates is an important component of program evaluation and contributes a different view that is rarely reported in the literature. The purpose of this study was to establish a database by surveying employers of baccalaureate-prepared nurses, postgraduation, over a five-year period. Employer surveys measured perceptions of graduates' functioning. Findings indicated that graduates function above expected levels for leadership skills, nursing skills, communication skills, and professionalism. Systematic program evaluation by employers is recommended at one and five years after graduation. A tool for employer evaluation of baccalaureate graduates is discussed.

Satellite power system concept development and evaluation program system definition technical assessment report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

1980-12-01

The results of the system definition studies conducted by NASA as a part of the Department of Energy/National Aeronautics and Space Administration SPS Concept Development and Evaluation Program are summarized. The purpose of the system definition efforts was to identify and define candidate SPS concepts and to evaluate the concepts in terms of technical and cost factors. Although the system definition efforts consisted primarily of evaluation and assessment of alternative technical approaches, a reference system was also defined to facilitate economic, environmental, and societal assessments by the Department of Energy. This reference system was designed to deliver 5 GW ofmore » electrical power to the utility grid. Topics covered include system definition; energy conversion and power management; power transmission and reception; structures, controls, and materials; construction and operations; and space transportation.« less

Evaluation of Demonstrations of National School Lunch Program and School Breakfast Program Direct Certification of Children Receiving Medicaid Benefits: Access Evaluation Report

ERIC Educational Resources Information Center

Hulsey, Lara; Gordon, Anne; Leftin, Joshua; Beyler, Nicholas; Schirm, Allen; Smither-Wulsin, Claire; Crumbley, Will

2015-01-01

This report presents findings from the Access Evaluation, a study component that is designed to assess the potential impacts of direct certification-Medicaid (DC-M) on students' access to free school meals by conducting retrospective simulations of DC-M in school year 2011-2012, the year before the demonstration began. For the Access Evaluation,…

A framework for assessment and monitoring of arthropods in a lowland tropical forest.

PubMed

Finnamore, Albert; Alonso, Alfonso; Santisteban, Jose; Cordova, Saida; Valencia, Gorky; de la Cruz, Alicia; Polo, Roberto

2002-05-01

By applying principles of adaptive management, and by using the valuable information that arthropods provide from assessment and monitoring programs, managers can identify and reduce possible impacts on biodiversity in development projects. In 1996, the Smithsonian Institution's Monitoring and Assessment of Biodiversity program worked together with Shell Prospecting and Development Peru to establish an adaptive management program to protect biodiversity in a natural gas exploration project in a Peruvian rainforest. In this paper, we outlined the conceptual steps involved in establishing an assessment and monitoring program for arthropods, including setting objectives, evaluating the results and making decisions. We also present the results of the assessment using some of groups of arthropods, and summarize the steps taken to identify appropriate groups for monitoring.

State Assessment (Accountability): Potential for Becoming a Friend or Foe?

ERIC Educational Resources Information Center

Buchmiller, Archie A.

A summary of legislation concerning educational accountability reveals that 13 States hope to find greater accountability in testing or assessment programs, 7 in planning, programming, budgeting systems, 4 in uniform accounting system, 8 in the evaluation of professional employees, 2 in management information systems, and 1 in performance…

Literacy Assessment in the Service of Literacy Policy.

ERIC Educational Resources Information Center

Venezky, Richard L.

Literacy policy has often developed independently of other social and employment programs. As a consequence, literacy tends to become an end unto itself, and assessment is directed more toward academic, archival ends than toward policy evaluation. Many justifications given for large-scale literacy programs are not based upon empirical data.…

Practitioner Assessment of Conflict Resolution Programs. ERIC Digest Number 163.

ERIC Educational Resources Information Center

Deutsch, Morton

There are many ways to assess the effectiveness of school conflict resolution training (CRT) programs. Some methods require extensive resources, but others, conducted by CRT practitioners themselves, also provide useful information. This digest presents a framework for CRT evaluation by practitioners that enables them to reflect productively on…

The Impact of Diagnosing Skill Deficiencies and Assessment-Based Communication Training on Managerial Performance.

ERIC Educational Resources Information Center

Papa, Michael J.; Graham, Elizabeth E.

1991-01-01

Evaluates an organizational diagnosis program that assesses managerial communication skills and provides the frame for follow-up training programs. Finds that managers participating in follow-up communication skills training performed significantly higher on interpersonal skills, problem-solving ability, and productivity over three long-term…

An evaluation capacity building toolkit for principal investigators of undergraduate research experiences: A demonstration of transforming theory into practice.

PubMed

Rorrer, Audrey S

2016-04-01

This paper describes the approach and process undertaken to develop evaluation capacity among the leaders of a federally funded undergraduate research program. An evaluation toolkit was developed for Computer and Information Sciences and Engineering(1) Research Experiences for Undergraduates(2) (CISE REU) programs to address the ongoing need for evaluation capacity among principal investigators who manage program evaluation. The toolkit was the result of collaboration within the CISE REU community with the purpose being to provide targeted instructional resources and tools for quality program evaluation. Challenges were to balance the desire for standardized assessment with the responsibility to account for individual program contexts. Toolkit contents included instructional materials about evaluation practice, a standardized applicant management tool, and a modulated outcomes measure. Resulting benefits from toolkit deployment were having cost effective, sustainable evaluation tools, a community evaluation forum, and aggregate measurement of key program outcomes for the national program. Lessons learned included the imperative of understanding the evaluation context, engaging stakeholders, and building stakeholder trust. Results from project measures are presented along with a discussion of guidelines for facilitating evaluation capacity building that will serve a variety of contexts. Copyright © 2016. Published by Elsevier Ltd.

A Comparison of Methodological Approaches for Evaluating the Quality of Basic Skills Programs. AIR Forum 1982 Paper.

ERIC Educational Resources Information Center

Dumont, Richard G.; And Others

A multiple-indicator and multiple-design approach for evaluating the quality of basic skills programs in reading, writing, and mathematics that has been implemented within a statewide system of higher education in the southeastern United States was assessed. Several of the evaluation methodologies are compared in order to determine whether results…

Evaluating Realized Impacts of DOE/EERE R&D Programs. Standard impact evaluation method

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ruegg, Rosalie; O'Connor, Alan C.; Loomis, Ross J.

2014-08-01

This document provides guidance for evaluators who conduct impact assessments of research and development (R&D) programs for the U.S. Department of Energy’s (DOE) Office of Energy Efficiency and Renewable Energy (EERE). It is also targeted at EERE program staff responsible for initiating and managing commissioned impact studies. The guide specifies how to estimate economic benefits and costs, energy saved and installed or generated, environmental impacts, energy security impacts, and knowledge impacts of R&D investments in advanced energy technologies.

Evaluating postdoctoral dental candidates: assessing the need and recommendations for a national qualifying examination.

PubMed

Brodie, Abby J; Crow, Heidi C; Eber, Robert M; Handysides, Robert; Holexa, Roy; Kiat-amnuay, Sudarat; Spallek, Heiko

2011-06-01

Increasingly, U.S. dental schools report pass/fail grades and do not rank students. In addition, the Joint Commission on National Dental Examinations will report National Board Dental Examination (NBDE) scores as pass/fail after January 1, 2012. This article discusses how these changes will force postdoctoral dental program directors to modify how they assess candidates and how noncognitive evaluations might enhance those assessments. The authors propose developing a national qualifying examination for postdoctoral dental programs that will measure knowledge, decision making, and noncognitive traits including empathy, self-confidence, integrity, and emotional intelligence. Without NBDE scores, class rank, and GPA as a basis for decision making, a single national qualifying examination would assist postdoctoral programs in selecting high-quality candidates based on knowledge, critical thinking skills, and noncognitive traits.

A High School-Based Evaluation of TakeCARE, a Video Bystander Program to Prevent Adolescent Relationship Violence.

PubMed

Sargent, Kelli S; Jouriles, Ernest N; Rosenfield, David; McDonald, Renee

2017-03-01

Although bystander programs to prevent relationship and sexual violence have been evaluated with college students, few evaluations have been conducted with high school students. This study evaluated the effectiveness of TakeCARE, a brief video bystander program designed to promote helpful bystander behavior in situations involving relationship violence among high school students. Students (N = 1295; 52.5% female; 72.3% Hispanic) reported their bystander behavior at a baseline assessment. Classrooms (N = 66) were randomized to view TakeCARE or to a control condition, and high school counselors administered the video in the classrooms assigned to view TakeCARE. Students again reported their bystander behavior at a follow-up assessment approximately 3 months afterward. Results indicate that students who viewed TakeCARE reported more helpful bystander behavior at the follow-up assessment than students in the control condition. Results of exploratory analyses of the likelihood of encountering and intervening upon specific situations calling for bystander behavior are also reported. TakeCARE is efficacious when implemented in an urban high school by high school counselors.

Growing Our Own: A Longitudinal Evaluation of a Professional Development Program for Early-Career 4-H Professionals

ERIC Educational Resources Information Center

Varrella, Gary F.; Luckey, Brian P.; Baca, Jacqueline S.; Peters, Curt

2016-01-01

We present the results of a longitudinal evaluation of the Western Region 4-H Institute, a 5-day training program designed to enhance the skill sets of early-career Extension professionals organized around the 4-H professional research, knowledge, and competencies model. Programs such as this often are assessed for their short-term relevance and…

Assessing the effectiveness of Montana's vehicle occupant protection program.

DOT National Transportation Integrated Search

2015-02-01

The purpose of this project was to quantitatively evaluate the relationships between MDT's occupant protection program : activities and seat restraint usage throughout Montana, in an effort to clarify how MDT's occupant protection programs may : affe...

Space transportation system shuttle turnabout analysis report

NASA Technical Reports Server (NTRS)

Reedy, R. E.

1979-01-01

The progress made and the problems encountered by the various program elements of the shuttle program in achieving the 160 hour ground turnaround goal are presented and evaluated. Task assessment time is measured against the program allocation time.

Multidimensional outcome considerations in assessing the efficacy of medical educational programs.

PubMed

Blumberg, Phyllis

2003-01-01

To be accredited, schools must evaluate the effectiveness of their programs. Educators are looking for specific indicators beyond the traditional measures. Data from multiple methods, including indicators of student performance and of the institutional environment, can be used to determine if educational program goals are met. This article outlines specific ways to consider three types of efficacy outcomes that are consistent with accreditation guidelines: educational, clinical career, and environmental outcomes. Specific measurable outcomes for each of these categories are derived from higher education literature: for example, learning approaches and information acquisition for education; professional behaviors and interpersonal dimensions for clinical career, and scholarship of teaching and teaching itself as a shared and valued activity for environmental outcomes. These outcomes address student assessment and program evaluation. Data from problem-based learning programs illustrate these outcomes. Educators can determine if educational program innovations have met their goals by collecting efficacy outcome data.

A novel clinical evaluation method using maximum angular velocity during knee extension to assess lower extremity muscle function of older adults.

PubMed

Arai, Takeshi; Obuchi, Shuichi; Shiba, Yoshitaka

2017-11-01

The purpose of this study is to examine the utilities of maximum angular velocity (AV) assessment during knee extension (KE) using a gyroscope for clinical evaluation of exercise program for older adults. Two hundred and 4 community-dwelling older adults underwent a 3-month exercise intervention program. Outcome measures included AV during KE and other physical functions (isometric strength (IS), walking abilities, and balance functions). A correlation coefficient was used to evaluate the relationships between AV and other physical functions at baseline. The differences of physical functions before and after intervention were evaluated and the effect size of each measurement was calculated after the program. The AV measurement was significantly correlated with IS during KE (r=0.303, P<0.01) and other physical functions. Most correlation coefficients of angular velocity were greater than that of IS. All of physical assessments were significantly improved. Also, effect size of AV was greater than that of IS (d=0.45 vs. 0.42). AV of the lower extremities is useful to evaluate the effects of exercise intervention in the elderly. Copyright © 2017 Elsevier B.V. All rights reserved.

What Works? Common Practices in High Functioning Afterschool Programs across the Nation in Math, Reading, Science, Arts, Technology, and Homework--A Study by the National Partnership. The Afterschool Program Assessment Guide. CRESST Report 768

ERIC Educational Resources Information Center

Huang, Denise; Cho, Jamie; Mostafavi, Sima; Nam, Hannah H.; Oh, Christine; Harven, Aletha; Leon, Seth

2010-01-01

In an effort to identify and incorporate exemplary practices into existing and future afterschool programs, the U.S. Department of Education commissioned a large-scale evaluation of the 21st Century Community Learning Center (CCLC) program. The purpose of this evaluation project was to develop resources and professional development that addresses…

Maintenance of a smoking cessation program in public health clinics beyond the experimental evaluation period.

PubMed Central

Manfredi, C.; Crittenden, K.; Cho, Y. I.; Engler, J.; Warnecke, R.

2001-01-01

OBJECTIVES: As phase 3 of a study to evaluate a smoking cessation program in public health practice, the authors assess the maintenance and impact of the It's Time smoking cessation program in seven public maternal and child health clinics in Chicago. METHODS: The authors interviewed 404 clinic patients in the study's baseline phase (prior to introduction of the It's Time intervention program), and 610 in the program maintenance phase (in the year after experimental evaluation had ended) to assess exposure to smoking cessation interventions offered at the clinic, and smoking cessation outcomes (quit, actions toward quitting, scores on action, motivation, readiness, and confidence scales). The authors controlled for clustering of smokers within clinics, smokers' characteristics prior to clinic visit, and type of clinic service. They compared outcomes by study group (control or intervention) to which each clinic had been assigned in the earlier experimental phase. RESULTS: Compared to baseline, smokers in the maintenance phase had greater exposure to posters, provider advice and booklet, and better outcomes on seven of eight smoking cessation measures, including quitting. These improvements were larger for clinics with prior experience implementing It's Time. CONCLUSION: Participation in the experimental evaluation of the It's Time program prepared and possibly motivated the clinics to continue the program. Continuing the program resulted in greater delivery of interventions and improved smoking cessation outcomes for smokers in the clinics. PMID:11889280

A National Survey of Undergraduate Teaching in Obstetrics and Gynecology.

ERIC Educational Resources Information Center

And Others; Stenchever, Morton A.

1979-01-01

A survey of academic departments of obstetrics and gynecology was designed to assess undergraduate educational programs and the impact of efforts made to improve teaching in the specialty. It focuses on instructional patterns, the clinical clerkship, student evaluation, and program administration and evaluation. Prior surveys are noted.…

Pilot evaluation of a walking school bus program in a low-income, urban community

USDA-ARS?s Scientific Manuscript database

Our objective was to evaluate the impact of a walking school bus (WSB) program on student transport in a low-income, urban neighborhood. The design was a controlled, quasi-experimental trial with consecutive cross-sectional assessments. The setting was three urban, socioeconomically disadvantaged, p...

Consultation in Bullying Prevention: An Elementary School Case Study

ERIC Educational Resources Information Center

Morrow, Michael T.; Hooker, Steven D.; Cate, Rebecca Lynne

2015-01-01

This manuscript outlines a consultation with a public elementary school that was aimed at assessing and strengthening the school's antibullying programs. We gathered consultation data through interviews and observations and also reviewed existing program evaluation data. We evaluated these data in light of current research on bullying prevention…

Use of Propensity Score Matching for Training Research with Observational Data

ERIC Educational Resources Information Center

Gemici, Sinan; Rojewski, Jay W.; Lee, In Heok

2012-01-01

Evaluations of vocational education and training (VET) programs play a key role in informing training policy in Australia and elsewhere. Increasingly, such evaluations use observational data from surveys or administrative collections to assess the effectiveness of VET programs and interventions. The difficulty associated with using observational…

A Final Report for the Evaluation of the Achieve3000 Programs

ERIC Educational Resources Information Center

Shannon, Lisa; Grant, Billie-Jo

2015-01-01

Achieve3000 publishes a number of online literacy programs that differentiate lessons and activities based on student performance. Magnolia Consulting--an independent, third party evaluation firm--assessed Achieve3000 Solutions' impact on student learning during the 2014-15 school year. The researchers used a randomized control trial, where…

Computer-Assisted Learning in Elementary Reading: A Randomized Control Trial

ERIC Educational Resources Information Center

Shannon, Lisa Cassidy; Styers, Mary Koenig; Wilkerson, Stephanie Baird; Peery, Elizabeth

2015-01-01

This study evaluated the efficacy of Accelerated Reader, a computer-based learning program, at improving student reading. Accelerated Reader is a progress-monitoring, assessment, and practice tool that supports classroom instruction and guides independent reading. Researchers used a randomized controlled trial to evaluate the program with 344…

A Multicultural Assessment of Adolescent Connectedness: Testing Measurement Invariance across Gender and Ethnicity

ERIC Educational Resources Information Center

Karcher, Michael J.; Sass, Daniel

2010-01-01

Counselors, psychologists, and evaluators of intervention programs for youth increasingly view the promotion of connectedness as an important intervention outcome. When evaluating these programs, researchers frequently test whether the treatment effects differ across gender and ethnic or racial groups. Doing so necessitates the availability of…

45 CFR 2516.820 - What types of internal evaluation activities are required of programs?

Code of Federal Regulations, 2011 CFR

2011-10-01

... required to: (a) Continuously assess management effectiveness, the quality of services provided, and the... evaluation methods, such as community advisory councils, participant advisory councils, peer reviews, quality... following data: (1) The total number of participants in each program and basic demographic characteristics...

Intensive training program for ultrasound diagnosis of adnexal masses: protocol and preliminary results.

PubMed

Alcázar, J L; Díaz, L; Flórez, P; Guerriero, S; Jurado, M

2013-08-01

To assess the feasibility of a specific training program for ultrasound diagnosis of adnexal masses. A 2-month intensive training program was developed. The program protocol consisted of a 1-day intensive theoretical course focused on clinical and sonographic issues related to adnexal masses and ovarian cancer, followed by a 4-week real-time ultrasound training program in a tertiary center (25-30 adnexal masses evaluated per month) and a final 4-week period for offline assessment of three-dimensional (3D) volumes from adnexal masses. In this final period, each trainee evaluated five sets of 100 3D volumes. 3D volumes contained gray-scale and power Doppler information, and the trainee was provided with clinical data for each case (patient age, menopausal status and reported symptoms). 3D volumes were obtained from surgically removed masses that had undergone histological diagnosis or from masses that had been followed up until resolution. After assessment of each set, the trainee's diagnostic performance was calculated (sensitivity and specificity) and each incorrectly classified mass was evaluated with the trainer. The objective was to achieve a sensitivity of > 95% and a specificity of > 90%. Learning curve cumulative summation (LC-CUSUM) graphs were plotted to assess the learning curve for the trainees. One trainer and two trainees with little experience in gynecological ultrasound (one gynecologist and one radiologist) participated in this study. LC-CUSUM graphs showed that competence was achieved after 170 or 185 examinations. The objectives for diagnostic performance were achieved after assessment of the second set of 3D volumes (200 cases) for each trainee. The proposed training program appears to be feasible. High diagnostic performance can be achieved after analysis of 200 cases and maintained thereafter. Copyright © 2013 ISUOG. Published by John Wiley & Sons Ltd.

Clinical Trial Assessment of Infrastructure Matrix Tool to Improve the Quality of Research Conduct in the Community.

PubMed

Dimond, Eileen P; Zon, Robin T; Weiner, Bryan J; St Germain, Diane; Denicoff, Andrea M; Dempsey, Kandie; Carrigan, Angela C; Teal, Randall W; Good, Marjorie J; McCaskill-Stevens, Worta; Grubbs, Stephen S; Dimond, Eileen P; Zon, Robin T; Weiner, Bryan J; St Germain, Diane; Denicoff, Andrea M; Dempsey, Kandie; Carrigan, Angela C; Teal, Randall W; Good, Marjorie J; McCaskill-Stevens, Worta; Grubbs, Stephen S

2016-01-01

Several publications have described minimum standards and exemplary attributes for clinical trial sites to improve research quality. The National Cancer Institute (NCI) Community Cancer Centers Program (NCCCP) developed the clinical trial Best Practice Matrix tool to facilitate research program improvements through annual self-assessments and benchmarking. The tool identified nine attributes, each with three progressive levels, to score clinical trial infrastructural elements from less to more exemplary. The NCCCP sites correlated tool use with research program improvements, and the NCI pursued a formative evaluation to refine the interpretability and measurability of the tool. From 2011 to 2013, 21 NCCCP sites self-assessed their programs with the tool annually. During 2013 to 2014, NCI collaborators conducted a five-step formative evaluation of the matrix tool. Sites reported significant increases in level-three scores across the original nine attributes combined (P<.001). Two specific attributes exhibited significant change: clinical trial portfolio diversity and management (P=.0228) and clinical trial communication (P=.0281). The formative evaluation led to revisions, including renaming the Best Practice Matrix as the Clinical Trial Assessment of Infrastructure Matrix (CT AIM), expanding infrastructural attributes from nine to 11, clarifying metrics, and developing a new scoring tool. Broad community input, cognitive interviews, and pilot testing improved the usability and functionality of the tool. Research programs are encouraged to use the CT AIM to assess and improve site infrastructure. Experience within the NCCCP suggests that the CT AIM is useful for improving quality, benchmarking research performance, reporting progress, and communicating program needs with institutional leaders. The tool model may also be useful in disciplines beyond oncology.

Evaluating and giving feedback to mentors: new evidence-based approaches.

PubMed

Anderson, Lauren; Silet, Karin; Fleming, Michael

2012-02-01

A comprehensive mentoring program includes a variety of components. One of the most important is the ongoing assessment of and feedback to mentors. Scholars need strong active mentors who have the expertise, disposition, motivation, skills, and the ability to accept feedback and to adjust their mentoring style. Assessing the effectiveness of a given mentor is no easy task. Variability in learning needs and academic goals among scholars makes it difficult to develop a single evaluation instrument or a standardized procedure for evaluating mentors. Scholars, mentors, and program leaders are often reluctant to conduct formal evaluations, as there are no commonly accepted measures. The process of giving feedback is often difficult and there is limited empirical data on efficacy. This article presents a new and innovative six-component approach to mentor evaluation that includes the assessment of mentee training and empowerment, peer learning and mentor training, scholar advocacy, mentee-mentor expectations, mentor self-reflection, and mentee evaluation of their mentor. © 2012 Wiley Periodicals, Inc.

What does a Performance Measurement System Tell Us about the National Comprehensive Cancer Control Program?

PubMed Central

Townsend, Julie S.; Moore, Angela R.; Mulder, Tiffani N.; Boyd, Mary

2015-01-01

Context The National Comprehensive Cancer Control Program (NCCCP) performance measurement system seeks to understand both the processes that funded programs undertake with their respective coalitions to implement the objectives of their cancer plans and outcomes of those efforts. Objective To identify areas of achievement and technical assistance needs of NCCCP awardees. Design Program performance was assessed through surveys completed by program directors on performance indicators in 2009 and 2010 and queries from a web-based management information system in 2011 and 2012. Setting Programs funded by CDC’s NCCCP. Participants 69 programs. Main Outcome Measure(s) The key performance measures assessed were: inclusion of diverse partners and key sectors in cancer coalitions; partners’ involvement in activities; receiving in-kind resources from partners; using evidence-based interventions and data for setting priorities; conducting program evaluation; using community- or organization-level strategies to address cancer control efforts; and demonstrating progress toward achieving health outcomes. Results Most programs reported having active coalitions that represent diverse organizational sectors. Nearly all programs routinely assess the burden of cancer. In-kind resources to implement activities peaked at $64,716 in the second year of a five year funding cycle, and declined in subsequent project years. By year 3, over 70% of programs reported having an evaluation plan. While programs reported that nearly two-thirds of their interventions were evidence-based, some programs implemented non-evidence-based interventions. A majority of programs successfully used at least one community- or organization-level change strategy. However, many programs did not incorporate objectives linked to health outcomes as they reported progress in implementing interventions. Conclusions: While NCCCP programs were strong at building and maintaining infrastructure, some programs may need additional technical assistance to increase the adoption of evidence-based interventions, develop solid and responsive evaluation plans, and better link efforts to population-based measures that demonstrate impact toward reducing the burden of cancer. PMID:25136936

Building an Assessment Argument to Design and Use Next Generation Science Assessments in Efficacy Studies of Curriculum Interventions

ERIC Educational Resources Information Center

DeBarger, Angela Haydel; Penuel, William R.; Harris, Christopher J.; Kennedy, Cathleen A.

2016-01-01

Evaluators must employ research designs that generate compelling evidence related to the worth or value of programs, of which assessment data often play a critical role. This article focuses on assessment design in the context of evaluation. It describes the process of using the Framework for K-12 Science Education and Next Generation Science…

Disease management interventions: what's in the black box?

PubMed

Linden, Ariel; Roberts, Nancy

2004-01-01

In discussing evaluation techniques to assess disease management (DM) program outcomes, it is often assumed that DM program interventions are premised on sound clinical judgment, an understanding of the disease process, and knowledge of the psychosocial models of behavioral change that must be used to effect those processes and ultimately improve the health outcomes that are being evaluated. This paper describes eight commonly used behavioral change models applied in the healthcare industry today. They represent programs designed to address individual, interpersonal, and community level factors as well as "packaged" comprehensive approaches. These models illustrate the breadth of approaches to consider when designing or assessing DM program interventions. Careful consideration of the type of behavioral change desired and the theories of how to effect such change should be an integral part of designing disease management program interventions.

Can outcome-based continuing medical education improve performance of immigrant physicians?

PubMed

Castel, Orit Cohen; Ezra, Vered; Alperin, Mordechai; Nave, Rachel; Porat, Tamar; Golan, Avivit Cohen; Vinker, Shlomo; Karkabi, Khaled

2011-01-01

Immigrant physicians are a valued resource for physician workforces in many countries. Few studies have explored the education and training needs of immigrant physicians and ways to facilitate their integration into the health care system in which they work. Using an educational program developed for immigrant civilian physicians working in military primary care clinics at the Israel Defence Force, we illustrate how an outcome-based CME program can address practicing physicians' needs for military-specific primary care education and improve patient care. Following an extensive needs assessment, a 3-year curriculum was developed. The curriculum was delivered by a multidisciplinary educational team. Pre/post multiple-choice examinations, objective structured clinical examinations (OSCE), and end-of-program evaluations were administered for curriculum evaluation. To evaluate change in learners' performance, data from the 2003 (before-program) and 2006 (after-program) work-based assessments were retrieved retrospectively. Change in the performance of program participants was compared with that of immigrant physicians who did not participate in the program. Out of 28 learners, 23 (82%) completed the program. Learners did significantly better in the annual post-tests compared with the pretests (p <.01) and improved their OSCE scores (p <.001). Most program graduates (90%) rated overall satisfaction as very good or excellent. In comparison with nonparticipants, program graduates performed better on work-based assessments (Cohen's d =.63). Our intensive, outcome-based, longitudinal CME program has yielded encouraging results. Other medical educators, facing the challenge of integrating immigrant physicians to fit their health care system, may consider adapting our approach. Copyright © 2011 The Alliance for Continuing Medical Education, the Society for Academic Continuing Medical Education, and the Council on CME, Association for Hospital Medical Education.

NDE detectability of fatigue-type cracks in high-strength alloys: NDI reliability assessments

NASA Technical Reports Server (NTRS)

Christner, Brent K.; Long, Donald L.; Rummel, Ward D.

1988-01-01

This program was conducted to generate quantitative flaw detection capability data for the nondestructive evaluation (NDE) techniques typically practiced by aerospace contractors. Inconel 718 and Haynes 188 alloy test specimens containing fatigue flaws with a wide distribution of sizes were used to assess the flaw detection capabilities at a number of contractor and government facilities. During this program 85 inspection sequences were completed presenting a total of 20,994 fatigue cracks to 53 different inspectors. The inspection sequences completed included 78 liquid penetrant, 4 eddy current, and 3 ultrasonic evaluations. The results of the assessment inspections are presented and discussed. In generating the flaw detection capability data base, procedures for data collection, data analysis, and specimen care and maintenance were developed, demonstrated, and validated. The data collection procedures and methods that evolved during this program for the measurement of flaw detection capabilities and the effects of inspection variables on performance are discussed. The Inconel 718 and Haynes 188 test specimens that were used in conducting this program and the NDE assessment procedures that were demonstrated, provide NASA with the capability to accurately assess the flaw detection capabilities of specific inspection procedures being applied or proposed for use on current and future fracture control hardware program.

Interdisciplinary Studies and the Real World: A Practical Rationale for and Guide to Postgraduation Evaluation and Assessment

ERIC Educational Resources Information Center

Brooks, Benjamin; Widders, Evan

2012-01-01

This article is an outgrowth of the evaluation and assessment process in a large interdisciplinary studies (IDS) program at West Virginia University. We argue for the importance of collecting and assessing quantitative and qualitative data concerning learning outcomes, student satisfaction, career choice and development, and graduate education…

A Critical Examination of the Technical Adequacy of a Curriculum-Based Assessment Using Rasch Analyses

ERIC Educational Resources Information Center

Winchell, Brooke

2011-01-01

The purpose of the study was to (a) examine the psychometric properties of The Assessment, Evaluation, and Programming System for Infants and Children (AEPS Test); (b) provide a process for establishing psychometric properties for other Curriculum Based Assessments (CBAs); and (c) identify and guide evaluation and subsequent revisions of the AEPS…

34 CFR 303.321 - Evaluation of the child and assessment of the child and family.

Code of Federal Regulations, 2013 CFR

2013-07-01

... INTERVENTION PROGRAM FOR INFANTS AND TODDLERS WITH DISABILITIES Child Find, Evaluations and Assessments, and... (a)(3)(i) of this section; and (ii) If the child is determined eligible as an infant or toddler with... needs of that infant or toddler. The assessments of the child and family are described in paragraph (c...

34 CFR 303.321 - Evaluation of the child and assessment of the child and family.

Code of Federal Regulations, 2014 CFR

2014-07-01

... INTERVENTION PROGRAM FOR INFANTS AND TODDLERS WITH DISABILITIES Child Find, Evaluations and Assessments, and... (a)(3)(i) of this section; and (ii) If the child is determined eligible as an infant or toddler with... needs of that infant or toddler. The assessments of the child and family are described in paragraph (c...

34 CFR 303.321 - Evaluation of the child and assessment of the child and family.

Code of Federal Regulations, 2012 CFR

2012-07-01

... INTERVENTION PROGRAM FOR INFANTS AND TODDLERS WITH DISABILITIES Child Find, Evaluations and Assessments, and... (a)(3)(i) of this section; and (ii) If the child is determined eligible as an infant or toddler with... needs of that infant or toddler. The assessments of the child and family are described in paragraph (c...

Experimental Evaluation of Computer Assisted Self-Assessment of Reading Comprehension: Effects on Reading Achievement and Attitude.

ERIC Educational Resources Information Center

Vollands, Stacy R.; And Others

A study evaluated the effect software for self-assessment and management of reading practice had on reading achievement and motivation in two primary schools in Aberdeen, Scotland. The program utilized was The Accelerated Reader (AR) which was designed to enable curriculum based assessment of reading comprehension within the classroom. Students…

Assessment of Students’ Critical-Thinking and Problem-Solving Abilities Across a 6-Year Doctor of Pharmacy Program

PubMed Central

Gaebelein, Claude J.; Grice, Gloria R.; Crannage, Andrew J.; Weck, Margaret A.; Hurd, Peter; Walter, Brenda; Duncan, Wendy

2013-01-01

Objective. To determine the feasibility of using a validated set of assessment rubrics to assess students’ critical-thinking and problem-solving abilities across a doctor of pharmacy (PharmD) curriculum. Methods. Trained faculty assessors used validated rubrics to assess student work samples for critical-thinking and problem-solving abilities. Assessment scores were collected and analyzed to determine student achievement of these 2 ability outcomes across the curriculum. Feasibility of the process was evaluated in terms of time and resources used. Results. One hundred sixty-one samples were assessed for critical thinking, and 159 samples were assessed for problem-solving. Rubric scoring allowed assessors to evaluate four 5- to 7-page work samples per hour. The analysis indicated that overall critical-thinking scores improved over the curriculum. Although low yield for problem-solving samples precluded meaningful data analysis, it was informative for identifying potentially needed curricular improvements. Conclusions. Use of assessment rubrics for program ability outcomes was deemed authentic and feasible. Problem-solving was identified as a curricular area that may need improving. This assessment method has great potential to inform continuous quality improvement of a PharmD program. PMID:24159207

Assessment of students' critical-thinking and problem-solving abilities across a 6-year doctor of pharmacy program.

PubMed

Gleason, Brenda L; Gaebelein, Claude J; Grice, Gloria R; Crannage, Andrew J; Weck, Margaret A; Hurd, Peter; Walter, Brenda; Duncan, Wendy

2013-10-14

To determine the feasibility of using a validated set of assessment rubrics to assess students' critical-thinking and problem-solving abilities across a doctor of pharmacy (PharmD) curriculum. Trained faculty assessors used validated rubrics to assess student work samples for critical-thinking and problem-solving abilities. Assessment scores were collected and analyzed to determine student achievement of these 2 ability outcomes across the curriculum. Feasibility of the process was evaluated in terms of time and resources used. One hundred sixty-one samples were assessed for critical thinking, and 159 samples were assessed for problem-solving. Rubric scoring allowed assessors to evaluate four 5- to 7-page work samples per hour. The analysis indicated that overall critical-thinking scores improved over the curriculum. Although low yield for problem-solving samples precluded meaningful data analysis, it was informative for identifying potentially needed curricular improvements. Use of assessment rubrics for program ability outcomes was deemed authentic and feasible. Problem-solving was identified as a curricular area that may need improving. This assessment method has great potential to inform continuous quality improvement of a PharmD program.

Assessing the Value of Workforce Training. A Guide for Small and Mid-sized Companies and the Providers that Serve Them.

ERIC Educational Resources Information Center

Askov, Eunice N.; Hoops, John; Alamprese, Judith

This booklet provides an introduction to evaluating a work force training program, both to assess its impact and to improve its effectiveness. The guide provides instructions for assessing a single training program, rather than a training department in a company or a training provider. It is targeted at small and midsized companies and the…

Program Evaluation of Outcomes Based Orthotic and Prosthetic Education

DTIC Science & Technology

2008-12-01

the Student Learning Outcomes Assessment group (SLOAC) at California State University Dominguez Hills. She is an assessment expert and consultant...identified in the program-level outcomes (Essential Elements of Learning Outcomes Assessment, Indicators 4 & 5, University Student Learning Outcomes...A short 70-mile trip is changing the lives of O&P students and impacting future care of returning military service members. By Jennifer Hoydicz

A multidimensional evaluation of a nursing information-literacy program.

PubMed Central

Fox, L M; Richter, J M; White, N E

1996-01-01

The goal of an information-literacy program is to develop student skills in locating, evaluating, and applying information for use in critical thinking and problem solving. This paper describes a multidimensional evaluation process for determining nursing students' growth in cognitive and affective domains. Results indicate improvement in student skills as a result of a nursing information-literacy program. Multidimensional evaluation produces a well-rounded picture of student progress based on formal measurement as well as informal feedback. Developing new educational programs can be a time-consuming challenge. It is important, when expending so much effort, to ensure that the goals of the new program are achieved and benefits to students demonstrated. A multidimensional approach to evaluation can help to accomplish those ends. In 1988, The University of Northern Colorado School of Nursing began working with a librarian to integrate an information-literacy component, entitled Pathways to Information Literacy, into the curriculum. This article describes the program and discusses how a multidimensional evaluation process was used to assess program effectiveness. The evaluation process not only helped to measure the effectiveness of the program but also allowed the instructors to use several different approaches to evaluation. PMID:8826621

Environmental assessment for the satellite power system concept development and evaluation program: nonmicrowave health and ecological effects

DOE Office of Scientific and Technical Information (OSTI.GOV)

White, M R

1980-11-01

A Concept Development and Evaluation Program is being carried out for a proposed Satellite Power System (SPS). For purposes of this evaluation, a preliminary reference system has been developed. SPS, as described in the reference system, would collect solar energy on satellites in geosychronous orbit in space. The energy would be converted to microwaves and beamed to an earth-receiving antenna (rectenna). One task in the environmental part of the program is the assessment of the nonmicrowave effects on health and the environment. These effects would result from all phases of SPS development and operation. This report covers the current knowledgemore » regarding these effects, and is based on the reference system. The assessment is summarized as to scope, methodology, impacts of terrestrial development, launch and recovery of spacecraft, space activities (including health effects of the space environment, ionizing radiation, electromagnetic exposure, spacecraft charging and environmental interactions, occupational hazards, etc.) and construction and operation of rectenna (ground receiving station).« less

Using evaluation methods to guide the development of a tobacco-use prevention curriculum for youth: a case study.

PubMed

Bridge, P D; Gallagher, R E; Berry-Bobovski, L C

2000-01-01

Fundamental to the development of educational programs and curricula is the evaluation of processes and outcomes. Unfortunately, many otherwise well-designed programs do not incorporate stringent evaluation methods and are limited in measuring program development and effectiveness. Using an advertising lesson in a school-based tobacco-use prevention curriculum as a case study, the authors examine the role of evaluation in the development, implementation, and enhancement of the curricular lesson. A four-phase formative and summative evaluation design was developed to divide the program-evaluation continuum into a structured process that would aid in the management of the evaluation, as well as assess curricular components. Formative and summative evaluation can provide important guidance in the development, implementation, and enhancement of educational curricula. Evaluation strategies identified unexpected barriers and allowed the project team to make necessary "time-relevant" curricular adjustments during each stage of the process.

Aggressive driving enforcement : evaluation of two demonstration programs

DOT National Transportation Integrated Search

2004-03-24

This report presents the results of a study conducted to assess the effects of two programs that were implemented to reduce the incidence of aggressive driving. The programs were conducted by the Marion County Traffic Safety Partnership (a consortium...

Aggressive driving enforcement : evaluations of two demonstration programs

DOT National Transportation Integrated Search

2004-03-01

This report presents the results of a study conducted to assess the effects of two programs that were implemented to reduce the incidence of aggressive driving. The programs were conducted by the Marion County Traffic Safety Partnership (a consortium...

Evaluation of staff performance and interpretation of the screening program for prevention of thalassemia.

PubMed

Prommetta, Simaporn; Sanchaisuriya, Kanokwan; Fucharoen, Goonnapa; Yamsri, Supawadee; Chaiboonroeng, Attawut; Fucharoen, Supan

2017-06-15

Thalassemia screening program has been implemented for years in Southeast Asia, but no external quality assessment program has been established. We have developed and initiated the proficiency testing (PT) program for the first time in Thailand with the aim to assess the screening performance of laboratory staff and their competency in interpretation of the screening results. Three PT cycles per year were organized. From the first to the third cycle of the PT scheme, a total number of participant laboratories increased from 59 to 67. In each cycle, 2 PT items (assigned as blood samples of the couple) were provided. Performance evaluation was based on the accuracy of screening results, i.e . mean corpuscular volume (MCV), mean corpuscular haemoglobin (MCH) and the dichlorophenolindophenol (DCIP) test for haemoglobin E, including the competency in interpretation of screening results and assessment of foetal risk. Performance was assessed by comparing the participants' result against the assigned value. Of all 3 cycles, most laboratories reported acceptable MCV and MCH values. From the first to the third cycle, incorrect DCIP test and misinterpretation rates were decreased while incorrect risk assessment varied by cycle to cycle. Combining the accuracy of thalassemia screening and the competency in interpretation and risk assessment, approximately half of participants showed excellent performance. Improved performance observed in many laboratories reflects the achievement and benefit of the PT program which should be regularly provided.

Assessment in Online Courses: How Are Counseling Skills Evaluated?

ERIC Educational Resources Information Center

Cicco, Gina

2011-01-01

Online courses are a necessary addition to most graduate education programs. Offering students the option of completing program requirements online makes the program more competitive, convenient, and attractive. Responsible online instructors and program administrators must consider whether or not specific courses that are offered in the…

Utility of the Living (Well Through) Intergenerational Fitness and Exercise Program as a County-Delivered Extension Program

ERIC Educational Resources Information Center

Sowle, Ashleigh J.; Francis, Sarah L.; Margrett, Jennifer A.; Franke, Warren D.

2016-01-01

Rural-residing older adults are not participating in regular physical activity. Extension is in an excellent position to fill this programming void through transdisciplinary programming such as the Living (well through) Intergenerational Fitness and Exercise (LIFE) program. Qualitative evaluation was conducted to assess the LIFE program's utility…

Using insects for STEM outreach: Development and evaluation of the UA Insect Discovery Program

NASA Astrophysics Data System (ADS)

Beal, Benjamin D.

Science and technology impact most aspects of modern daily life. It is therefore important to create a scientifically literate society. Since the majority of Americans do not take college-level science courses, strong K-12 science education is essential. At the K-5 level, however, many teachers lack the time, resources and background for effective science teaching. Elementary teachers and students may benefit from scientist-led outreach programs created by Cooperative Extension or other institutions. One example is the University of Arizona Insect Discovery Program, which provides short-duration programing that uses insects to support science content learning, teach critical thinking and spark interest in science. We conducted evaluations of the Insect Discovery programming to determine whether the activities offered were accomplishing program goals. Pre-post tests, post program questionnaires for teachers, and novel assessments of children's drawings were used as assessment tools. Assessments were complicated by the short duration of the program interactions with the children as well as their limited literacy. In spite of these difficulties, results of the pre-post tests indicated a significant impact on content knowledge and critical thinking skills. Based on post-program teacher questionnaires, positive impacts on interest in science learning were noted as much as a month after the children participated in the program. New programming and resources developed to widen the potential for impact are also described.

A Prospective Policy Evaluation of the Michigan Merit Award Program.

ERIC Educational Resources Information Center

Bishop, John H.

In 1999 Michigan implemented the Michigan Merit Award program, a program to motivate high school students to take their studies more seriously. The program offers 1-year college scholarships to students who meet or exceed state standards on Michigan Educational Assessment Program (MEAP) tests in reading, mathematics, science, and writing. This…

Developing Leadership for Life: Outcomes from a Collegiate Student-Alumni Mentoring Program

ERIC Educational Resources Information Center

Priest, Kerry L.; Donley, Sarah

2014-01-01

This application brief describes the exploratory assessment of a mentoring program between current students and alumni of a leadership studies minor program. We connect leadership education research and practice in two ways: first, we describe a process of qualitative program evaluation to inform program best practices and improvement. In doing…

Factor Analysis of the HEW National Strategy for Youth Development Model's Community Program Impact Scales.

ERIC Educational Resources Information Center

Truckenmiller, James L.

The former HEW (Health, Education, and Welfare) National Strategy for Youth Development Model proposed a community-based program to promote positive youth development and to prevent delinquency through a sequence of youth needs assessments, needs-targeted programs, and program impact evaluation. HEW Community Program Impact Scales data obtained…

Independence Community College Program Review Process.

ERIC Educational Resources Information Center

Romine, Robert J.

Program review at Independence Community College (ICC) is structured so that every program of study will undergo review within a 5-year period. The purpose of the program review effort is to assess the relationship of inputs to outcomes, so that actual outcomes can be compared to expected outcomes. Each program of study is evaluated according to…

An Effective Model for Improving Global Health Nursing Competence.

PubMed

Kang, Sun-Joo

2016-01-01

This paper proposed an effective model for improving global health nursing competence among undergraduate students. A descriptive case study was conducted by evaluation of four implemented programs by the author. All programs were conducted with students majoring in nursing and healthcare, where the researcher was a program director, professor, or facilitator. These programs were analyzed in terms of students' needs assessment, program design, and implementation and evaluation factors. The concept and composition of global nursing competence, identified within previous studies, were deemed appropriate in all of our programs. Program composition varied from curricular to extracurricular domains. During the implementation phase, some of the programs included non-Korean students to improve cultural diversity and overcome language barriers. Qualitative and quantitative surveys were conducted to assess program efficacy. Data triangulation from students' reflective journals was examined. Additionally, students' awareness regarding changes within global health nursing, improved critical thinking, cultural understanding, and global leadership skills were investigated pre- and post-program implementation. The importance of identifying students' needs regarding global nursing competence when developing appropriate curricula is discussed.

An evaluation of client satisfaction with training programs and technical assistance provided by Florida's Coordinated School Health Program Office.

PubMed

Weiler, R M; Pigg, R M

2000-11-01

Client or customer satisfaction surveys assess the perceived quality of programs, products, services, and employee performance. Such assessments prove beneficial for evaluation and planning purposes. This survey examined the satisfaction of clients using the programs, services, and technical assistance provided through the Coordinated School Health Program Office (CSHPO) in the Florida Department of Education. Using the 42-item Client Satisfaction Survey, data were collected in summer 1999 from 300 of 574 clients (52.3%) who attended training sessions or sought technical assistance from CSHPO during 1996-1999. More than two-thirds (67.2%) of clients rated the training program as "very good" or "excellent" at increasing their understanding about the concept of a coordinated school health program. Overall, 69.7% of clients rated the training programs they attended as "very good" or "excellent." Resource materials and staff effectiveness rated positively as well. Findings confirmed client satisfaction with CSHPO's training programs, technical assistance, and staff. Information obtained through the client satisfaction survey can be used by CSHPO to assist in future program planning and resource allocations.

Criteria for Evaluating Alternative Network and Link Layer Protocols for the NASA Constellation Program Communication Architecture

NASA Technical Reports Server (NTRS)

Benbenek, Daniel; Soloff, Jason; Lieb, Erica

2010-01-01

Selecting a communications and network architecture for future manned space flight requires an evaluation of the varying goals and objectives of the program, development of communications and network architecture evaluation criteria, and assessment of critical architecture trades. This paper uses Cx Program proposed exploration activities as a guideline; lunar sortie, outpost, Mars, and flexible path options are described. A set of proposed communications network architecture criteria are proposed and described. They include: interoperability, security, reliability, and ease of automating topology changes. Finally a key set of architecture options are traded including (1) multiplexing data at a common network layer vs. at the data link layer, (2) implementing multiple network layers vs. a single network layer, and (3) the use of a particular network layer protocol, primarily IPv6 vs. Delay Tolerant Networking (DTN). In summary, the protocol options are evaluated against the proposed exploration activities and their relative performance with respect to the criteria are assessed. An architectural approach which includes (a) the capability of multiplexing at both the network layer and the data link layer and (b) a single network layer for operations at each program phase, as these solutions are best suited to respond to the widest array of program needs and meet each of the evaluation criteria.

Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: practical outcomes of an assessment of program director needs.

PubMed

Puddester, Derek; MacDonald, Colla J; Clements, Debbie; Gaffney, Jane; Wiesenfeld, Lorne

2015-06-05

The Royal College of Physicians and Surgeons of Canada and the College of Family Physicians of Canada mandate that faculty members demonstrate they are evaluating residents on all CanMEDS (Canadian Medical Education Directions for Specialists) roles as part of the accreditation process. Postgraduate Medical Education at the University of Ottawa initiated a 5-year project to develop and implement a comprehensive system to assess the full spectrum of CanMEDS roles. This paper presents the findings from a needs assessment with Program Directors, in order to determine how postgraduate medical faculty can be motivated and supported to evaluate residents on the intrinsic CanMEDS roles. Semi-structured individual interviews were conducted with 60 Postgraduate Program Directors in the Faculty of Medicine. Transcribed interviews were analyzed using qualitative analysis. Once the researchers were satisfied the identified themes reflected the views of the participants, the data was assigned to categories to provide rich, detailed, and comprehensive information that would indicate what faculty need in order to effectively evaluate their residents on the intrinsic roles. Findings indicated faculty members need faculty development and shared point of care resources to support them with how to not only evaluate, but also teach, the intrinsic roles. Program Directors expressed the need to collaborate and share resources across departments and national specialty programs. Based on our findings, we designed and delivered workshops with companion eBooks to teach and evaluate residents at the point of care (Developing the Professional, Health Advocate and Scholar). Identifying stakeholder needs is essential for designing effective faculty development. By sharing resources, faculties can prevent 'reinventing the wheel' and collaborate to meet the Colleges' accreditation requirements more efficiently.

Assessing Student Achievement in the States. The First Report of the National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1990 Trial State Assessment.

ERIC Educational Resources Information Center

National Academy of Education, Stanford, CA.

This report presents findings of the National Academy of Education Panel on the Evaluation of the Trial State Assessment (TSA) concerning the 1990 National Assessment of Educational Progress (NAEP) mathematics TSA program. The Panel's recommendations regarding the continuation of TSAs for review by the Congress, states, and Executive Branch of the…

Evaluation of the Florida coordinated school health program pilot schools project.

PubMed

Weiler, Robert M; Pigg, R Morgan; McDermott, Robert J

2003-01-01

The Florida Department of Education, with CDC funding, designed the Florida Coordinated School Health Program Pilot Schools Project (PSP) to encourage innovative approaches to promote coordinated school health programs (CSHP) in Florida schools. Each of eight pilot schools received $15,000 in project funding, three years of technical assistance including on-site and off-site assistance, a project office resource center, mailings of resource materials, needs assessment and evaluation assistance, and three PSP Summer Institutes. Project evaluators created a context evaluation, approaching each school independently as a "case study" to measure the school's progress in meeting goals established at baseline. Data were collected using the How Healthy is Your School? needs assessment instrument, a School Health Portfolio constructed by each school team, a Pilot Schools Project Team Member Survey instrument, midcourse team interviews, final team interviews, and performance indicator data obtained from pilot and control schools. The PSP posed two fundamental questions: "Can financial resources, professional training, and technical assistance enable individual schools to create and sustain a coordinated school health program?" and "What outcomes reasonably can one expect from a coordinated school health program, assuming programs receive adequate support over time?" First, activities at the eight schools confirmed that a coordinated school health programs can be established and sustained. Program strength and sustainability depend on long-term resources, qualified personnel, and administrative support. Second, though coordinated school health programs may improve school performance indicators, the PSP yielded insufficient evidence to support that belief. Future projects should include robust measurement and evaluation designs, thereby producing conclusive evidence about the influence of a coordinated school health program on such outcomes.

Developing a comprehensive resident education evaluation system in the era of milestone assessment.

PubMed

Gardner, Aimee K; Scott, Daniel J; Choti, Michael A; Mansour, John C

2015-01-01

In an effort to move training programs toward competency-based education, the Accreditation Council for Graduate Medical Education (ACGME) introduced the Next Accreditation System (NAS), which organizes specific milestones regarding resident skills, knowledge, and abilities along a continuum. In order to foster innovation and creativity, the ACGME has provided programs with minimal guidelines regarding the optimal way to approach these milestones. The education team at UT Southwestern embraced the milestones and developed a process in which performance assessment methods were critically evaluated, mapped onto an extrapolated performance list corresponding to the areas required by the ACGME milestones, and filled gaps in the previous system by modifying evaluation tools and creating new program components. Although the authors are early in the evolution of applying the new milestones system, this approach has thus far allowed them to comprehensively evaluate the residents and the program in an efficient and effective fashion, with notable improvements compared to the prior approach. The authors hope that these experiences can inform others embarking upon similar journeys with the milestones. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Design and Assessment of an Associate Degree-Level Plant Operations Technical Education Program

NASA Astrophysics Data System (ADS)

Selwitz, Jason Lawrence

Research was undertaken to develop and evaluate an associate degree-level technical education program in Plant Operations oriented towards training students in applied science, technology, engineering, and mathematics (STEM) skills and knowledge relevant to a spectrum of processing industries. This work focuses on four aspects of the curriculum and course development and evaluation research. First, the context of, and impetus for, what was formerly called vocational education, now referred to as technical or workforce education, is provided. Second, the research that was undertaken to design and evaluate an associate degree-level STEM workforce education program is described. Third, the adaptation of a student self-assessment of learning gains instrument is reviewed, and an analysis of the resulting data using an adapted logic model is provided, to evaluate the extent to which instructional approaches, in two process control/improvement-focused courses, were effective in meeting course-level intended learning outcomes. Finally, eight integrative multiscale exercises were designed from two example process systems, wastewater treatment and fast pyrolysis. The integrative exercises are intended for use as tools to accelerate the formation of an operator-technician's multiscale vision of systems, unit operations, underlying processes, and fundamental reactions relevant to multiple industries. Community and technical colleges serve a vital function in STEM education by training workers for medium- and high-skilled technical careers and providing employers the labor necessary to operate and maintain thriving business ventures. Through development of the curricular, course, and assessment-related instruments and tools, this research helps ensure associate degree-level technical education programs can engage in a continual process of program evaluation and improvement.

Assessment of Program Impact Through First Grade, Volume II: Impact on Institutions. An Evaluation of Project Developmental Continuity. Interim Report X.

ERIC Educational Resources Information Center

Rosario, Jose; And Others

As part of a longitudinal study evaluating program effects, this report, the second in a series of six, describes the impact of Project Developmental Continuity (PDC) on the institutional policies and procedures of participating Head Start centers and elementary schools up to the time the evaluation study's cohort of children had completed grade…

More, but Not Yet Better: An Evaluation of USAID's Programs and Policies To Improve Girls' Education. USAID Program and Operations Assessment Report No. 25.

ERIC Educational Resources Information Center

O'Gara, Chloe; Benoliel, Sharon; Sutton, Margaret; Tietjen, Karen

This report evaluates U.S. Agency for International Development (USAID) efforts to improve basic education for girls. The evaluation drew on field studies in: Guatemala, Guinea, Malawi, Nepal, and Pakistan; a country desk study of Egypt; issue-oriented research on Bolivia and Thailand; and a literature review. Findings indicate that effective…

A Program Evaluation of the Basic Supervisory Development Seminar

DTIC Science & Technology

1985-08-01

evaluation. - Hoyle , English, and Steffy (1985) describe various methods that can be used to accomplisn the appropriate objectives of the evaluation...Maynihan Figure 2 Alternative Methods of Assessing Student and Program Effectiveness ( Hoyle , English, and Steffy, 1985) 21 CHAPTER III METHODCLOGY S Design...Fiedler, Fred E. "Responses to Sergiovanni." Educational Leadership (March 1979), 394-96. Fiedler, Fred E., and Martin M. Chemers. Leadership and

Evidence of the Adoption and Implementation of a Statewide Childhood Obesity Prevention Initiative in the New York State WIC Program: The "NY Fit WIC" Process Evaluation

ERIC Educational Resources Information Center

Sekhobo, Jackson P.; Egglefield, Katherine; Edmunds, Lynn S.; Shackman, Gene

2012-01-01

Process evaluations are critical in determining whether outcome evaluations are warranted. This study assessed the extent to which a childhood obesity prevention initiative, "NY Fit WIC", was adopted and implemented by the New York State Supplemental Nutrition Program for Women, Infants and Children (WIC). Process data came from…

Preparing for Evaluation: Lessons from the Evaluability Assessment of the Teagle Foundation's College-Community Connections Initiative. Report

ERIC Educational Resources Information Center

Black, Kristin

2016-01-01

Funders, policymakers, and program leaders recognize the value of high-quality evidence. To make good use of a program evaluation, initiatives must contend with a set of fundamental questions first. Some of these are about the initiative itself: What outcomes does it seek to affect? Are daily activities in line with long-term goals? Others are…

Primary prevention research: a preliminary review of program outcome studies.

PubMed

Schaps, E; Churgin, S; Palley, C S; Takata, B; Cohen, A Y

1980-07-01

This article reviews 35 drug abuse prevention program evaluations employing drug-specific outcome measures. Many of these evaluations assessed the effects of "new generation" prevention strategies: affective, peer-oriented, and multidimensional approaches. Only 14 studies evaluated purely informational programs. Evaluations were analyzed to ascertain (1) characteristics of the programs under study, (2) characteristics of the research designs, and (3) patterns among findings. This review provides some evidence that the newer prevention strategies may produce more positive and fewer negative outcomes than did older drug information approaches. Over 70% of the programs using the newer strategies produced some positive effects; only 29% showed negative effects. In contrast, 46% of informational programs showed positive effects; 46% showed negative effects. These findings must be approached with great caution, since the research was frequently scientifically inadequate, and since rigor of research was negatively correlated with intensity and duration of program services.

Evaluation of the impact of collaborative work by teams from the National Medical Residency Committee and the Brazilian Society of Neurosurgery. Retrospective and prospective study.

PubMed

Santos, Renato Antunes Dos; Snell, Linda; Nunes, Maria do Patrocínio Tenório

2016-04-01

Training for specialist physicians in Brazil can take place in different ways. Closer liaison between institutions providing this training and assessment and health care services may improve qualifications. This article analyzes the impact of closer links and joint work by teams from the National Medical Residency Committee (Comissão Nacional de Residência Médica, CNRM) and the Brazilian Society of Neurosurgery (Sociedade Brasileira de Neurocirurgia, SBN) towards evaluating these programs. Retrospective and prospective study, conducted in a public university on a pilot project developed between CNRM and SBN for joint assessment of training programs across Brazil. The literature in the most relevant databases was reviewed. Documents and legislation produced by official government bodies were evaluated. Training locations were visited. Reports produced about residency programs were analyzed. Only 26% of the programs were immediately approved. The joint assessments found problems relating to teaching and to functioning of clinical service in 35% of the programs. The distribution of programs in this country has a strong relationship with the Human Development Index (HDI) of the regions and is very similar to the distribution of specialists. Closer collaboration between the SBN and CNRM had a positive impact on assessment of neurosurgery medical residency across the country. The low rates of direct approval have produced modifications and improvements to the quality of teaching and care (services). Closer links between the CNRM and other medical specialties have the capability to positively change the structure and function of specialty training in Brazil.

UTAH MANPOWER DEVELOPMENT AND TRAINING ACT (MDTA) SCHOOLS ASSESSMENT.

ERIC Educational Resources Information Center

GADDIS, G. WARREN

BECAUSE LITTLE RELATED EXPERIENCE WAS AVAILABLE UPON WHICH TO FABRICATE THE MOST EFFICIENT PROGRAMS OF THE TWO FIRST MANPOWER DEVELOPMENT AND TRAINING ACT (MDTA) SPONSORED SCHOOLS OPENED IN UTAH IN 1965, THIS STUDY PROPOSED TO ASSESS THE PROGRAMS TO DETERMINE SIGNIFICANT DIFFERENCES, AND TO MAKE A SUBJECTIVE EVALUATION OF THE CURRICULAR STRENGTHS…

Administration and Research of Competency-/Performance-Based Teacher Education Programs.

ERIC Educational Resources Information Center

Trzasko, Joseph A.

This paper describes the proposed Assessment Center at Mercy College in Dobbs Ferry, New York. The center is intended to provide statistical and technical support for the Mercy College elementary education, special education, and speech and hearing departments in the areas of student assessment, student guidance, and program evaluation. Evaluation…

Estimating the Effectiveness of Special Education Using Large-Scale Assessment Data

ERIC Educational Resources Information Center

Ewing, Katherine Anne

2009-01-01

The inclusion of students with disabilities in large scale assessment and accountability programs has provided new opportunities to examine the impact of special education services on student achievement. Hanushek, Kain, and Rivkin (1998, 2002) evaluated the effectiveness of special education programs by examining students' gains on a large-scale…

Using LISREL to Evaluate Measurement Models and Scale Reliability.

ERIC Educational Resources Information Center

Fleishman, John; Benson, Jeri

1987-01-01

LISREL program was used to examine measurement model assumptions and to assess reliability of Coopersmith Self-Esteem Inventory for Children, Form B. Data on 722 third-sixth graders from over 70 schools in large urban school district were used. LISREL program assessed (1) nature of basic measurement model for scale, (2) scale invariance across…

Assessment of Critical Business Skill Development by MBA Alumni

ERIC Educational Resources Information Center

Glynn, Joseph G.; Wood, Gregory R.

2008-01-01

Six years of survey data were analyzed to assess, among other things, the degree to which an AACSB accredited graduate business program successfully developed student skills in a variety of areas deemed important for career success. The study illustrates a methodology institutions can use to respond to increasing demands for program evaluation and…

Assessing paratransit eligibility under the Americans With Disabilities Act in the rehabilitation setting.

PubMed

Griffin, Jeanne; Priddy, David A

2005-06-01

To assess the usefulness of a rehabilitation-based assessment program designed to determine the eligibility, according to Americans With Disabilities Act criteria, of applicants for paratransit bus services. Retrospective summary statistics on 500 consecutive paratransit evaluations. Outpatient physical medicine and rehabilitation center. Applicants for a community paratransit bus service. Not applicable. Clinical assessment of each applicants functional physical and cognitive ability to ride a fixed-route or paratransit bus system. Of the 500 applicants for specialized paratransit services, 38 (8%) were found to be ineligible, based on rehabilitation professionals evaluations of their physical and cognitive abilities. Mass transit organizations must adjust to the rapidly growing demand for paratransit services. Rehabilitation-based assessment programs, because of the expertise they provide in assessing functional abilities, are uniquely qualified to provide objective determinations of paratransit eligibility.

Internal dosimetry technical basis manual

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

1990-12-20

The internal dosimetry program at the Savannah River Site (SRS) consists of radiation protection programs and activities used to detect and evaluate intakes of radioactive material by radiation workers. Examples of such programs are: air monitoring; surface contamination monitoring; personal contamination surveys; radiobioassay; and dose assessment. The objectives of the internal dosimetry program are to demonstrate that the workplace is under control and that workers are not being exposed to radioactive material, and to detect and assess inadvertent intakes in the workplace. The Savannah River Site Internal Dosimetry Technical Basis Manual (TBM) is intended to provide a technical and philosophicalmore » discussion of the radiobioassay and dose assessment aspects of the internal dosimetry program. Detailed information on air, surface, and personal contamination surveillance programs is not given in this manual except for how these programs interface with routine and special bioassay programs.« less

A three-year reflective writing program as part of introductory pharmacy practice experiences.

PubMed

Nuffer, Wesley; Vaughn, Jessica; Kerr, Kevin; Zielenski, Christopher; Toppel, Brianna; Johnson, Lauren; McCauley, Patrina; Turner, Christopher J

2013-06-12

To implement and evaluate a 3-year reflective writing program incorporated into introductory pharmacy practice experiences (IPPEs) in the first- through third-year of a doctor of pharmacy (PharmD) program. Reflective writing was integrated into 6 IPPE courses to develop students' lifelong learning skills. In their writing, students were required to self-assess their performance in patient care activities, identify and describe how they would incorporate learning opportunities, and then evaluate their progress. Practitioners, faculty members, and fourth-year PharmD students served as writing preceptors. The success of the writing program was assessed by reviewing class performance and surveying writing preceptor's opinions regarding the student's achievement of program objectives. Class pass rates averaged greater than 99% over the 8 years of the program and the large majority of the writing preceptors reported that student learning objectives were met. A support pool of 99 writing preceptors was created. A 3-year reflective writing program improved pharmacy students' reflection and reflective writing skills.

Increasing the use of evaluation data collection in an EPO program

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dordevic, M.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D.; Welti, R.; Davis, H. B.

2017-12-01

Over the past two years, the Incorporated Research Institutions for Seismology Education and Public Outreach (EPO) program has sought to increase the evaluation rigor of its programs and products. Specifically we sought to make evaluation an integral part of our work; enabling staff to demonstrate why we do the activities we do, enhancing the impact or our products/programs, and empowering staff to make evidence-based claims. The Collaborative Impact Analysis Method (Davis and Scalice, 2015) was selected as it allowed us to combine staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make improvements to the programs over time. A key part of the initial action plans has been the collection and analysis of new evaluation data. The most frequently used tools were surveys as they were relatively straightforward to implement and analyze, and could be adapted for different situations. Examples include: brand awareness, value of booth interactions, assessing community interest in a data app, and user surveys of social media and specific web pages. Other evaluation activities included beta testing of new software, and interviews with students and faculty involved in summer field experiences. The surveys have allowed us to document increased impact in some areas, to improve the usability of products and activities, and to provide baseline impact data. The direct involvement of staff in the process has helped staff appreciate the value of evaluation, but there are also challenges to this approach. Since many of the surveys are developed and conducted by EPO staff, rather than being primarily handled by the evaluator, the process takes considerably more staff time to implement. We are still determining how to best manage and present the data and analysis; our current approach is to post evaluation reports on our EPO website so that other groups may be able to benefit from our evaluation results. Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis Method. Abstract ED53D-0871, 2015 Fall Meeting, AGU.

Shortening the Miles to the Milestones: Connecting EPA-Based Evaluations to ACGME Milestone Reports for Internal Medicine Residency Programs.

PubMed

Choe, John H; Knight, Christopher L; Stiling, Rebekah; Corning, Kelli; Lock, Keli; Steinberg, Kenneth P

2016-07-01

The Next Accreditation System requires internal medicine training programs to provide the Accreditation Council for Graduate Medical Education (ACGME) with semiannual information about each resident's progress in 22 subcompetency domains. Evaluation of resident "trustworthiness" in performing entrustable professional activities (EPAs) may offer a more tangible assessment construct than evaluations based on expectations of usual progression toward competence. However, translating results from EPA-based evaluations into ACGME milestone progress reports has proven to be challenging because the constructs that underlay these two systems differ.The authors describe a process to bridge the gap between rotation-specific EPA-based evaluations and ACGME milestone reporting. Developed at the University of Washington in 2012 and 2013, this method involves mapping EPA-based evaluation responses to "milestone elements," the narrative descriptions within the columns of each of the 22 internal medicine subcompetencies. As faculty members complete EPA-based evaluations, the mapped milestone elements are automatically marked as "confirmed." Programs can maintain a database that tallies the number of times each milestone element is confirmed for a resident; these data can be used to produce graphical displays of resident progress along the internal medicine milestones.Using this count of milestone elements allows programs to bridge the gap between faculty assessments of residents based on rotation-specific observed activities and semiannual ACGME reports based on the internal medicine milestones. Although potentially useful for all programs, this method is especially beneficial to large programs where clinical competency committee members may not have the opportunity for direct observation of all residents.

Assessment of the quality and applicability of an e-portfolio capstone assessment item within a bachelor of midwifery program.

PubMed

Baird, Kathleen; Gamble, Jenny; Sidebotham, Mary

2016-09-01

Education programs leading to professional licencing need to ensure assessments throughout the program are constructively aligned and mapped to the specific professional expectations. Within the final year of an undergraduate degree, a student is required to transform and prepare for professional practice. Establishing assessment items that are authentic and able to reflect this transformation is a challenge for universities. This paper both describes the considerations around the design of a capstone assessment and evaluates, from an academics perspective, the quality and applicability of an e-portfolio as a capstone assessment item for undergraduate courses leading to a professional qualification. The e-portfolio was seen to meet nine quality indicators for assessment. Academics evaluated the e-portfolio as an authentic assessment item that would engage the students and provide them with a platform for ongoing professional development and lifelong learning. The processes of reflection on strengths, weaknesses, opportunities and threats, comparison of clinical experiences with national statistics, preparation of professional philosophy and development of a curriculum vitae, whilst recognised as comprehensive and challenging were seen as highly valuable to the student transforming into the profession. Copyright © 2016 Elsevier Ltd. All rights reserved.

Opening the Conversation on REU Assessment and Evaluation

NASA Astrophysics Data System (ADS)

Pressley, S. N.; LeBeau, J. E.

2015-12-01

Project evaluation is a key component to ensuring success of any Research Experience for Undergraduates (REU) program. The Washington State University (WSU) Regional Atmospheric Chemistry: State-of-the-art Measurement and Modeling in the Pacific Northwest REU Site employs a mixed method approach to determine what is working well and what can use improvement (formative evaluation) and to determine impact and effectiveness of the project in reaching the stated goals (summative evaluation). Quantitative data is collected via a pre-/post-test measuring participants' research self-efficacy (RSE), motivation, background information, extent of socialization, and their interpretation of the value of the REU experience. Qualitative data is gathered through individual interviews with the REU students (at the beginning and end of the program) and faculty mentors (at the end). Beginning interviews focus on expectations for the REU program and student backgrounds. End interviews focus on student RSE development, interpretations of their experience, and the value of the experience. Faculty mentors are interviewed to gather insight on student performance in the program and perspectives on the overall success of the program in meeting the proposed goals. Students are provided an opportunity to comment on the strengths and weaknesses of workshops, providing critical feedback to the particular instructor and enabling the faculty to modify the workshop content and activities in future years. Finally, research results are evaluated during the final poster presentation, and faculty are interviewed to report on their perception of how each student learned and gained knowledge during the program. To evaluate the retention of students in engineering and science and identify chosen career paths, a longitudinal survey was created and it is administered via email each year. Many REU programs also employ the Undergraduate Research Students Self-Assessment (URSSA) online tool designed for evaluating REU experiences (University of Colorado at Boulder). Information presented in this presentation will include a discussion of the current tools used as described above as well as a discussion of the pros/cons of using URSSA. One of the concerns with using URSSA in conjunction with the other assessment tools is "survey fatigue".