Evaluating Educational Programs. ERIC Digest Series Number EA 54.

ERIC Educational Resources Information Center

Beswick, Richard

In this digest, readers are introduced to the scope of instructional program evaluation and evaluators' changing roles in school districts. A program evaluation measures outcomes based on student-attainment goals, implementation levels, and external factors such as budgetary restraints and community support. Instructional program evaluation may be…

Measurement issues in the evaluation of chronic disease self-management programs.

PubMed

Nolte, Sandra; Elsworth, Gerald R; Newman, Stanton; Osborne, Richard H

2013-09-01

To provide an in-depth analysis of outcome measures used in the evaluation of chronic disease self-management programs consistent with the Stanford curricula. Based on a systematic review on self-management programs, effect sizes derived from reported outcome measures are categorized according to the quality of life appraisal model developed by Schwartz and Rapkin which classifies outcomes from performance-based measures (e.g., clinical outcomes) to evaluation-based measures (e.g., emotional well-being). The majority of outcomes assessed in self-management trials are based on evaluation-based methods. Overall, effects on knowledge--the only performance-based measure observed in selected trials--are generally medium to large. In contrast, substantially more inconsistent results are found for both perception- and evaluation-based measures that mostly range between nil and small positive effects. Effectiveness of self-management interventions and resulting recommendations for health policy makers are most frequently derived from highly variable evaluation-based measures, that is, types of outcomes that potentially carry a substantial amount of measurement error and/or bias such as response shift. Therefore, decisions regarding the value and efficacy of chronic disease self-management programs need to be interpreted with care. More research, especially qualitative studies, is needed to unravel cognitive processes and the role of response shift bias in the measurement of change.

The Relationship of Explicit-Implicit Evaluative Discrepancy to Exercise Dropout in Middle-Aged Adults.

PubMed

Berry, Tanya R; Rodgers, Wendy M; Divine, Alison; Hall, Craig

2018-06-19

Discrepancies between automatically activated associations (i.e., implicit evaluations) and explicit evaluations of motives (measured with a questionnaire) could lead to greater information processing to resolve discrepancies or self-regulatory failures that may affect behavior. This research examined the relationship of health and appearance exercise-related explicit-implicit evaluative discrepancies, the interaction between implicit and explicit evaluations, and the combined value of explicit and implicit evaluations (i.e., the summed scores) to dropout from a yearlong exercise program. Participants (N = 253) completed implicit health and appearance measures and explicit health and appearance motives at baseline, prior to starting the exercise program. The sum of implicit and explicit appearance measures was positively related to weeks in the program, and discrepancy between the implicit and explicit health measures was negatively related to length of time in the program. Implicit exercise evaluations and their relationships to oft-cited motives such as appearance and health may inform exercise dropout.

From Implementation to Outcomes to Impacts: Designing a Comprehensive Program Evaluation

NASA Astrophysics Data System (ADS)

Shebby, S.

2015-12-01

Funders are often interested in learning about the impact of program activities, yet before the impacts are determined, educational evaluations should first examine program implementation and outcomes. Implementation evaluation examines how and the extent to which program activities are delivered as intended, including the extent to which activities reached the targeted participants. Outcome evaluation is comprised of a systematic examination of the effects that a program has on program participants, such as changes in knowledge, attitudes, beliefs, values, and behaviors. In this presentation, presenters will share insights on evaluating the implementation, outcomes, and impacts associated with an online science curriculum for K-2 students. The science curriculum was designed to provide students with access to science concepts and skills in an interactive and innovative environment, and teachers with embedded, aligned, and on-demand professional development. One of the most important—and challenging—steps in this evaluation was to select outcomes that were well-defined, measurable, and aligned to program activities, as well as relevant to program stakeholders. An additional challenge was to measure implementation given limited access to the classroom environment. This presentation will include a discussion of the process evaluators used to select appropriate implementation indicators and outcomes (teacher and student), design an evaluation approach, and craft data collection instruments. Although examples provided are specific to the K-2 science intervention, the best practices discussed are pertinent to all program and event evaluations. Impact evaluation goes beyond implementation and outcome evaluation to inform whether a program is working or not. It requires a comparison group to inform what outcomes would have been in the absence of the intervention. As such, this presentation will also include a discussion of impacts, including how impacts are defined and measured, and some common challenges in evaluating program impact.

Outcome Evaluation: Student Development Program, Special Studies Division, Cleveland State University.

ERIC Educational Resources Information Center

Pasch, Marvin

Techniques and procedures used to evaluate the outcomes of the student development program, and to use the evaluation results, are presented. Specific evaluation questions are posed that address overall outcomes, not individual student outcomes, and quantitative measures are suggested to accompany the questions. The measures include statistical…

Measuring Success: Evaluating Educational Programs

ERIC Educational Resources Information Center

Fisher, Yael

2010-01-01

This paper reveals a new evaluation model, which enables educational program and project managers to evaluate their programs with a simple and easy to understand approach. The "index of success model" is comprised of five parameters that enable to focus on and evaluate both the implementation and results of an educational program. The…

Continuous Evaluation in Ethics Education: A Case Study.

PubMed

McIntosh, Tristan; Higgs, Cory; Mumford, Michael; Connelly, Shane; DuBois, James

2018-04-01

A great need for systematic evaluation of ethics training programs exists. Those tasked with developing an ethics training program may be quick to dismiss the value of training evaluation in continuous process improvement. In the present effort, we use a case study approach to delineate how to leverage formative and summative evaluation measures to create a high-quality ethics education program. With regard to formative evaluation, information bearing on trainee reactions, qualitative data from the comments of trainees, in addition to empirical findings, can ensure that the training program operates smoothly. Regarding summative evaluation, measures examining trainee cognition, behavior, and organization-level results provide information about how much trainees have changed as a result of taking the ethics training. The implications of effective training program evaluation are discussed.

Evaluating the potential of group singing to enhance the well-being of older people.

PubMed

Davidson, Jane W; McNamara, Beverley; Rosenwax, Lorna; Lange, Andrea; Jenkins, Sue; Lewin, Gill

2014-06-01

To evaluate the effect of a singing program developed specifically for older community-dwelling people on measures of health and well-being. An eight-week singing program was developed and evaluated using standardised measures of health and well-being, measures designed to examine specific singing program outcomes, and semi-structured interviews. Participants aged 70 years and older were recruited through a home care service provider (n = 17) and an advertisement in a community newspaper (n = 19). Standard outcome measures indicated that the program had little effect on health and well-being. However, study-specific measures indicated that many participants had positive gains. Those in the home care group required more assistance to attend and continue in the program than those in the general community. Participants reported that the community-based singing facilitator was essential to the program's success. Well-structured community-based singing programs have the potential to impact positively upon the well-being of older people, but program viability depends on support with recruitment, transport and funding. © 2013 The Authors. Australasian Journal on Ageing © 2013 ACOTA.

The State of Evaluation in Internal Medicine Residency

PubMed Central

Holmboe, Eric; Beasley, Brent W.

2008-01-01

Background There are no nationwide data on the methods residency programs are using to assess trainee competence. The Accreditation Council for Graduate Medical Education (ACGME) has recommended tools that programs can use to evaluate their trainees. It is unknown if programs are adhering to these recommendations. Objective To describe evaluation methods used by our nation’s internal medicine residency programs and assess adherence to ACGME methodological recommendations for evaluation. Design Nationwide survey. Participants All internal medicine programs registered with the Association of Program Directors of Internal Medicine (APDIM). Measurements Descriptive statistics of programs and tools used to evaluate competence; compliance with ACGME recommended evaluative methods. Results The response rate was 70%. Programs were using an average of 4.2–6.0 tools to evaluate their trainees with heavy reliance on rating forms. Direct observation and practice and data-based tools were used much less frequently. Most programs were using at least 1 of the Accreditation Council for Graduate Medical Education (ACGME)’s “most desirable” methods of evaluation for all 6 measures of trainee competence. These programs had higher support staff to resident ratios than programs using less desirable evaluative methods. Conclusions Residency programs are using a large number and variety of tools for evaluating the competence of their trainees. Most are complying with ACGME recommended methods of evaluation especially if the support staff to resident ratio is high. PMID:18612734

Effective Practices for Evaluating Education and Public Outreach Programs

NASA Astrophysics Data System (ADS)

Wilkerson, S.

2013-12-01

Stephanie Baird Wilkerson, PhD Carol Haden EdD Magnolia Consulting,LLC Education and public outreach (EPO) program developers and providers seeking insights regarding effective practices for evaluating EPO activities programs benefit from understanding why evaluation is critical to the success of EPO activities and programs, what data collection methods are appropriate, and how to effectively communicate and report findings. Based on our extensive experience evaluating EPO programs, we will share lessons learned and examples of how these practices play out in actual evaluation studies. EPO program developers, providers, and evaluators must consider several factors that influence which evaluation designs and data collection methods will be most appropriate, given the nature of EPO programs. Effective evaluation practices of EPO programs take into account a program's phase of development, duration, and budget as well as a program's intended outcomes. EPO programs that are just beginning development will have different evaluation needs and priorities than will well-established programs. Effective evaluation practices consider the 'life' of a program with an evaluation design that supports a program's growth through various phases including development, revision and refinement, and completion. It would be premature and inappropriate to expect the attainment of longer-term outcomes of activities during program development phases or early stages of implementation. During program development, EPO providers should clearly define program outcomes that are feasible and appropriate given a program's scope and expected reach. In many respects, this directly relates to the amount of time, or duration, intended audiences participate in EPO programs. As program duration increases so does the likelihood that the program can achieve longer-term outcomes. When choosing which outcomes are reasonable to impact and measure, program duration should be considered. Effective evaluation practices include selecting appropriate data collection methods given a program's duration and corresponding intended outcomes. Data collection methods for programs of short duration might involve simple evaluation activities, whereas programs of longer duration might involve ongoing data collection measures including longitudinal student surveys, implementation logs, student journals, and student achievement measures. During our presentation, we will share examples from our own experience to illustrate how effective evaluation practices can be applied to various EPO programs based on program duration. Irrespective of duration, we find that EPO program developers and providers want both formative feedback to guide improvements and summative feedback on outcomes. More often than not, evaluation budgets for EPO programs are meager at best, yet come with the same information needs and priorities as programs with larger evaluation budgets. So how do program providers get the information they need given their limited funds for evaluation? We will offer several recommendations for helping EPO program providers work with evaluators to become better-informed consumers of evaluation by maximizing evaluation offerings and minimizing costs. During our presentation we also will share examples of communicating and reporting results for EPO program developers, EPO facilitators and practitioners, and funders.

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2013 CFR

2013-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2014 CFR

2014-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2012 CFR

2012-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2011 CFR

2011-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... services from your program who increase their reading ability from “below grade level” to “at or above grade level”. This measure indicates something good is happening to your program's service beneficiaries...

45 CFR 2522.700 - How does evaluation differ from performance measurement?

Code of Federal Regulations, 2010 CFR

2010-10-01

... progress, evaluation uses scientifically-based research methods to assess the effectiveness of programs by... the reading ability of students in a program over time to a similar group of students not... example, a performance measure for a literacy program may include the percentage of students receiving...

Measuring the Impact of Employment-Related Social Programs. A Primer on the Evaluation of Employment and Training, Vocational Education, Vocational Rehabilitation, and Other Job-Oriented Programs.

ERIC Educational Resources Information Center

Borus, Michael E.

An approach and methodology for the systematic measurement of the impact of employment-related social programs is presented in this primer. Chapter 1 focuses on evaluation as the third step (the first two being planning and operation) in the process of program implementation. Chapter 2 examines the impacts of social programs. Topics include…

An Examination of the Concept and Role of Program Monitoring and Evaluation.

ERIC Educational Resources Information Center

Sherwood-Fabre, Liese

This paper examines the concepts of program monitoring and program evaluation in the literature, and offers working definitions based on two dimensions of measurement: focus (what questions are addressed) and timing (how often the measures are taken). Focus can be on inputs to the program or outcomes from it; timing can be one-shot or continuous.…

Methodology for the evaluation of the Stephanie Alexander Kitchen Garden program.

PubMed

Gibbs, L; Staiger, P K; Townsend, M; Macfarlane, S; Gold, L; Block, K; Johnson, B; Kulas, J; Waters, E

2013-04-01

Community and school cooking and gardening programs have recently increased internationally. However, despite promising indications, there is limited evidence of their effectiveness. This paper presents the evaluation framework and methods negotiated and developed to meet the information needs of all stakeholders for the Stephanie Alexander Kitchen Garden (SAKG) program, a combined cooking and gardening program implemented in selectively funded primary schools across Australia. The evaluation used multiple aligned theoretical frameworks and models, including a public health ecological approach, principles of effective health promotion and models of experiential learning. The evaluation is a non-randomised comparison of six schools receiving the program (intervention) and six comparison schools (all government-funded primary schools) in urban and rural areas of Victoria, Australia. A mixed-methods approach was used, relying on qualitative measures to understand changes in school cultures and the experiential impacts on children, families, teachers, parents and volunteers, and quantitative measures at baseline and 1 year follow up to provide supporting information regarding patterns of change. The evaluation study design addressed the limitations of many existing evaluation studies of cooking or garden programs. The multistrand approach to the mixed methodology maintained the rigour of the respective methods and provided an opportunity to explore complexity in the findings. Limited sensitivity of some of the quantitative measures was identified, as well as the potential for bias in the coding of the open-ended questions. The SAKG evaluation methodology will address the need for appropriate evaluation approaches for school-based kitchen garden programs. It demonstrates the feasibility of a meaningful, comprehensive evaluation of school-based programs and also demonstrates the central role qualitative methods can have in a mixed-method evaluation. So what? This paper contributes to debate about appropriate evaluation approaches to meet the information needs of all stakeholders and will support the sharing of measures and potential comparisons between program outcomes for comparable population groups and settings.

1999 commuter assistance program evaluation manual

DOT National Transportation Integrated Search

2001-01-01

This manual was developed to assist Florida's Commuter Assistance Programs (CAP) to measure and evaluate their performance. It provides information necessary for a CAP to create and implement its own evaluation program. It discusses performance measu...

Chapter 13: Assessing Persistence and Other Evaluation Issues Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Violette, Daniel M.

Addressing other evaluation issues that have been raised in the context of energy efficiency programs, this chapter focuses on methods used to address the persistence of energy savings, which is an important input to the benefit/cost analysis of energy efficiency programs and portfolios. In addition to discussing 'persistence' (which refers to the stream of benefits over time from an energy efficiency measure or program), this chapter provides a summary treatment of these issues -Synergies across programs -Rebound -Dual baselines -Errors in variables (the measurement and/or accuracy of input variables to the evaluation).

Using Curriculum-Based Measurements for Program Evaluation: Expanding Roles for School Psychologists

ERIC Educational Resources Information Center

Tusing, Mary E.; Breikjern, Nicholle A.

2017-01-01

Educators increasingly need to evaluate schoolwide reform efforts; however, complex program evaluations often are not feasible in schools. Through a case example, we provide a heuristic for program evaluation that is easily replicated in schools. Criterion-referenced interpretations of schoolwide screening data were used to evaluate outcomes…

Review of Evaluation, Measurement and Verification Approaches Used to Estimate the Load Impacts and Effectiveness of Energy Efficiency Programs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Messenger, Mike; Bharvirkar, Ranjit; Golemboski, Bill

Public and private funding for end-use energy efficiency actions is expected to increase significantly in the United States over the next decade. For example, Barbose et al (2009) estimate that spending on ratepayer-funded energy efficiency programs in the U.S. could increase frommore » $3.1 billion in 2008 to $$7.5 and 12.4 billion by 2020 under their medium and high scenarios. This increase in spending could yield annual electric energy savings ranging from 0.58% - 0.93% of total U.S. retail sales in 2020, up from 0.34% of retail sales in 2008. Interest in and support for energy efficiency has broadened among national and state policymakers. Prominent examples include {approx}$$18 billion in new funding for energy efficiency programs (e.g., State Energy Program, Weatherization, and Energy Efficiency and Conservation Block Grants) in the 2009 American Recovery and Reinvestment Act (ARRA). Increased funding for energy efficiency should result in more benefits as well as more scrutiny of these results. As energy efficiency becomes a more prominent component of the U.S. national energy strategy and policies, assessing the effectiveness and energy saving impacts of energy efficiency programs is likely to become increasingly important for policymakers and private and public funders of efficiency actions. Thus, it is critical that evaluation, measurement, and verification (EM&V) is carried out effectively and efficiently, which implies that: (1) Effective program evaluation, measurement, and verification (EM&V) methodologies and tools are available to key stakeholders (e.g., regulatory agencies, program administrators, consumers, and evaluation consultants); and (2) Capacity (people and infrastructure resources) is available to conduct EM&V activities and report results in ways that support program improvement and provide data that reliably compares achieved results against goals and similar programs in other jurisdictions (benchmarking). The National Action Plan for Energy Efficiency (2007) presented commonly used definitions for EM&V in the context of energy efficiency programs: (1) Evaluation (E) - The performance of studies and activities aimed at determining the effects and effectiveness of EE programs; (2) Measurement and Verification (M&V) - Data collection, monitoring, and analysis associated with the calculation of gross energy and demand savings from individual measures, sites or projects. M&V can be a subset of program evaluation; and (3) Evaluation, Measurement, and Verification (EM&V) - This term is frequently seen in evaluation literature. EM&V is a catchall acronym for determining both the effectiveness of program designs and estimates of load impacts at the portfolio, program and project level. This report is a scoping study that assesses current practices and methods in the evaluation, measurement and verification (EM&V) of ratepayer-funded energy efficiency programs, with a focus on methods and practices currently used for determining whether projected (ex-ante) energy and demand savings have been achieved (ex-post). M&V practices for privately-funded energy efficiency projects (e.g., ESCO projects) or programs where the primary focus is greenhouse gas reductions were not part of the scope of this study. We identify and discuss key purposes and uses of current evaluations of end-use energy efficiency programs, methods used to evaluate these programs, processes used to determine those methods; and key issues that need to be addressed now and in the future, based on discussions with regulatory agencies, policymakers, program administrators, and evaluation practitioners in 14 states and national experts in the evaluation field. We also explore how EM&V may evolve in a future in which efficiency funding increases significantly, innovative mechanisms for rewarding program performance are adopted, the role of efficiency in greenhouse gas mitigation is more closely linked, and programs are increasingly funded from multiple sources often with multiple program administrators and intended to meet multiple purposes.« less

The Efficacy of Stuttering Measurement Training: Evaluating Two Training Programs

ERIC Educational Resources Information Center

Bainbridge, Lauren A.; Stavros, Candace; Ebrahimian, Mineh; Wang, Yuedong; Ingham, Roger J.

2015-01-01

Purpose: Two stuttering measurement training programs currently used for training clinicians were evaluated for their efficacy in improving the accuracy of total stuttering event counting. Method: Four groups, each with 12 randomly allocated participants, completed a pretest-posttest design training study. They were evaluated by their counts of…

An Evaluation of the Technical Adequacy of a Revised Measure of Quality Indicators of Transition

ERIC Educational Resources Information Center

Morningstar, Mary E.; Lee, Hyunjoo; Lattin, Dana L.; Murray, Angela K.

2016-01-01

This study confirmed the reliability and validity of the Quality Indicators of Exemplary Transition Programs Needs Assessment-2 (QI-2). Quality transition program indicators were identified through a systematic synthesis of transition research, policies, and program evaluation measures. To verify reliability and validity of the QI-2, we…

SUPERFUND INNOVATIVE TECHNOLOGY EVALUATION PROGRAM TECHNOLOGY PROFILES: SIXTH EDITION

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program evaluates new and promising treatment and monitoring and measurement technologies for cleanup of hazardous waste sites. The program was created to encourage the development and routine use of innovative treatment techn...

SUPERFUND INNOVATIVE TECHNOLOGY EVALUATION PROGRAM - TECHNOLOGY PROFILES - SEVENTH EDITION

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program evaluates new and promising treatment and monitoring and measurement technologies for cleanup of hazardous waste sites. The program was created to encourage the development and routine use of innovative treatment techn...

An evaluation toolkit for Florida's Commuter Assistance Programs (CAP) : a companion to the 1999 CAP evaluation manual

DOT National Transportation Integrated Search

2001-01-01

This manual is a companion piece to the Commuter Assistance Program Evaluation Manual that was developed to assist Florida's Commuter Assistance Programs (CAP) in their efforts to measure and evaluate their performance. This manual is intended to pro...

An evaluation of Teaching-Family (Achievement Place) group homes for juvenile offenders.

PubMed

Kirigin, K A; Braukmann, C J; Atwater, J D; Wolf, M M

1982-01-01

Juvenile crime is a serious problem for which treatment approach has been found to be reliably effective. This outcome evaluation assessed during and posttreatment effectiveness of Teaching-Family group home treatment programs for juvenile offenders. The evaluation included the original Achievement Place program, which was the prototype for the development of the Teaching-Family treatment approach, 12 replications of Achievement Place, and 9 comparison group home programs. Primary dependent measures were retrieved from court and police files and included number of alleged offenses, percentage of youths involved in those alleged offenses, and percentage of youths institutionalized. Other dependent measures were subjective ratings of effectiveness obtained from the program consumers, including the group home residents. The results showed difference during treatment favoring the Teaching-Family programs on rate of alleged criminal offenses, percentage of youths involved in those offenses, and consumer ratings of the programs. The consumer ratings provided by the youths and their school teachers were found to be inversely and significantly correlated with the reduction of criminal offenses during treatment. There were no significant differences during treatment on measures of noncriminal offenses (e.g., truancy, runaway, and curfew violations). In the posttreatment year, none of the differences between the groups was significant on any of the outcome measures. The results are discussed in terms of measurement and design issues in the evaluation of delinquency treatment programs and in relation to the evaluation; of Teaching-Family group homes by Richard Jones and his colleagues.

An evaluation of Teaching-Family (Achievement Place) group homes for juvenile offenders.

PubMed Central

Kirigin, K A; Braukmann, C J; Atwater, J D; Wolf, M M

1982-01-01

Juvenile crime is a serious problem for which treatment approach has been found to be reliably effective. This outcome evaluation assessed during and posttreatment effectiveness of Teaching-Family group home treatment programs for juvenile offenders. The evaluation included the original Achievement Place program, which was the prototype for the development of the Teaching-Family treatment approach, 12 replications of Achievement Place, and 9 comparison group home programs. Primary dependent measures were retrieved from court and police files and included number of alleged offenses, percentage of youths involved in those alleged offenses, and percentage of youths institutionalized. Other dependent measures were subjective ratings of effectiveness obtained from the program consumers, including the group home residents. The results showed difference during treatment favoring the Teaching-Family programs on rate of alleged criminal offenses, percentage of youths involved in those offenses, and consumer ratings of the programs. The consumer ratings provided by the youths and their school teachers were found to be inversely and significantly correlated with the reduction of criminal offenses during treatment. There were no significant differences during treatment on measures of noncriminal offenses (e.g., truancy, runaway, and curfew violations). In the posttreatment year, none of the differences between the groups was significant on any of the outcome measures. The results are discussed in terms of measurement and design issues in the evaluation of delinquency treatment programs and in relation to the evaluation; of Teaching-Family group homes by Richard Jones and his colleagues. PMID:7096223

Evaluation Techniques for the Sandy Point Discovery Center, Great Bay National Estuarine Research Reserve.

ERIC Educational Resources Information Center

Heffernan, Bernadette M.

1998-01-01

Describes work done to provide staff of the Sandy Point Discovery Center with methods for evaluating exhibits and interpretive programming. Quantitative and qualitative evaluation measures were designed to assess the program's objective of estuary education. Pretest-posttest questionnaires and interviews are used to measure subjects' knowledge and…

Evaluating disease management program effectiveness: an introduction to time-series analysis.

PubMed

Linden, Ariel; Adams, John L; Roberts, Nancy

2003-01-01

Currently, the most widely used method in the disease management (DM) industry for evaluating program effectiveness is referred to as the "total population approach." This model is a pretest-posttest design, with the most basic limitation being that without a control group, there may be sources of bias and/or competing extraneous confounding factors that offer a plausible rationale explaining the change from baseline. Furthermore, with the current inclination of DM programs to use financial indicators rather than program-specific utilization indicators as the principal measure of program success, additional biases are introduced that may cloud evaluation results. This paper presents a non-technical introduction to time-series analysis (using disease-specific utilization measures) as an alternative, and more appropriate, approach to evaluating DM program effectiveness than the current total population approach.

A multidimensional evaluation of a nursing information-literacy program.

PubMed Central

Fox, L M; Richter, J M; White, N E

1996-01-01

The goal of an information-literacy program is to develop student skills in locating, evaluating, and applying information for use in critical thinking and problem solving. This paper describes a multidimensional evaluation process for determining nursing students' growth in cognitive and affective domains. Results indicate improvement in student skills as a result of a nursing information-literacy program. Multidimensional evaluation produces a well-rounded picture of student progress based on formal measurement as well as informal feedback. Developing new educational programs can be a time-consuming challenge. It is important, when expending so much effort, to ensure that the goals of the new program are achieved and benefits to students demonstrated. A multidimensional approach to evaluation can help to accomplish those ends. In 1988, The University of Northern Colorado School of Nursing began working with a librarian to integrate an information-literacy component, entitled Pathways to Information Literacy, into the curriculum. This article describes the program and discusses how a multidimensional evaluation process was used to assess program effectiveness. The evaluation process not only helped to measure the effectiveness of the program but also allowed the instructors to use several different approaches to evaluation. PMID:8826621

Formative Evaluation of the Canada Student Loans Program. Final Report

ERIC Educational Resources Information Center

Human Resources and Skills Development Canada, 2004

2004-01-01

The "Formative Evaluation of the Canada Student Loans Program" was undertaken to assess issues of program relevance, design and delivery and for the purposes of examining the early impacts of changes made to the program since 1998. The evaluation also reviewed the Performance Measurement Strategy contained in the July 2002 Results-Based…

Chapter 8: Whole-Building Retrofit with Consumption Data Analysis Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W.; Agnew, Ken; Goldberg, Mimi

Whole-building retrofits involve the installation of multiple measures. Whole-building retrofit programs take many forms. With a focus on overall building performance, these programs usually begin with an energy audit to identify cost-effective energy efficiency measures for the home. Measures are then installed, either at no cost to the homeowner or partially paid for by rebates and/or financing. The methods described here may also be applied to evaluation of single-measure retrofit programs. Related methods exist for replace-on-failure programs and for new construction, but are not the subject of this chapter.

75 FR 42760 - Statement of Organization, Functions, and Delegations of Authority

Federal Register 2010, 2011, 2012, 2013, 2014

2010-07-22

... accounting reports and invoices, and monitoring all spending. The Team develops, defends and executes the... results; performance measurement; research and evaluation methodologies; demonstration testing and model... ACF programs; strategic planning; performance measurement; program and policy evaluation; research and...

Hanford radiological protection support services annual report for 1994

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lyon, M.; Bihl, D.E.; Fix, J.J.

1995-06-01

Various Hanford Site radiation protection services provided by the Pacific Northwest Laboratory for the US Department of Energy Richland Operations Office and Hanford contractors are described in this annual report for the calendar year 1994. These activities include external dosimetry measurements and evaluations, internal dosimetry measurements and evaluations, in vivo measurements, radiological record keeping, radiation source calibration, and instrument calibration and evaluation. For each of these activities, the routine program and any program changes or enhancements are described, as well as associated tasks, investigations, and studies. Program- related publications, presentations, and other staff professional activities are also described.

Hanford radiological protection support services. Annual report for 1995

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lyon, M.; Bihl, D.E.; Carbaugh, E.H.

1996-05-01

Various Hanford Site radiation protection services provided by the Pacific Northwest National Laboratory for the U.S. Department of Energy Richland Operations Office and Hanford contractors are described in this annual report for calendar year 1995. These activities include external dosimetry measurements and evaluations, internal dosimetry measurements and evaluations, in vivo measurements, radiological record keeping, radiation source calibration, and instrument calibration and evaluation. For each of these activities, the routine program and any program changes or enhancements are described, as well as associated tasks, investigations, and studies. Program-related publications, presentations, and other staff professional activities are also described.

Posture and gait abilities in patients with myotonic dystrophy (Steinert disease). Evaluation on the short-term of a rehabilitation program.

PubMed

Missaoui, B; Rakotovao, E; Bendaya, S; Mane, M; Pichon, B; Faucher, M; Thoumie, P

2010-01-01

To evaluate the effects of a rehabilitation program in terms of balance, gait and muscle strength in a population of patients with myotonic dystrophy. Twenty patients benefited, as outpatients in a hospital setting, from a rehabilitation program with clinical and instrumental evaluations. The evaluation focused on quantitative balance measurement by clinical and stabilometer tests, gait assessed by Locometre and extensors and flexors knee muscle strength measured in isokinetic concentric mode at 60°/s. After the rehabilitation program, we observed a significant improvement in the patients' balance capacities measured with the Berg Balance Scale (BBS), fast gait speed and muscle strength. However, the instrumental evaluation did not report any gains for static balance and spontaneous gait speed after the training program. No correlation was found between the various improvements. A rehabilitation program focused on strength, gait and balance allowed for significant improvements in some parameters of myotonic dystrophy. These results attest to the relevance of a short-term rehabilitation protocol for these patients in the framework of a multidisciplinary therapeutic care. The disparity observed in the results measured for these patients suggest the contribution of cognitive involvement in the limitations felt by patients with myotonic dystrophy in the areas of gait and balance. Copyright 2010 Elsevier Masson SAS. All rights reserved.

Pilot Program on Common Status Measures Objective-Referenced Tests. Colorado Evaluation Project, Report No. 1.

ERIC Educational Resources Information Center

Colorado State Dept. of Education, Denver.

The purpose of the Colorado Evaluation Project was to field test the Common Status Measures at grades four and eleven in conjunction with a statewide assessment program based on objective-referenced testing instruments developed by the Colorado Department of Education for grades kindergarten, three, six, nine, and twelve. The evaluation was…

Are You Making an Impact? Evaluating the Population Health Impact of Community Benefit Programs.

PubMed

Rains, Catherine M; Todd, Greta; Kozma, Nicole; Goodman, Melody S

The Patient Protection and Affordable Care Act includes a change to the IRS 990 Schedule H, requiring nonprofit hospitals to submit a community health needs assessment every 3 years. Such health care entities are challenged to evaluate the effectiveness of community benefit programs addressing the health needs identified. In an effort to determine the population health impact of community benefit programs in 1 hospital outreach department, researchers and staff conducted an impact evaluation to develop priority areas and overarching goals along with program- and department-level objectives. The longitudinal impact evaluation study design consists of retrospective and prospective secondary data analyses. As an urban pediatric hospital, St Louis Children's Hospital provides an array of community benefit programs to the surrounding community. Hospital staff and researchers came together to form an evaluation team. Data from program evaluation and administrative data for analysis were provided by hospital staff. Impact scores were calculated by scoring objectives as met or unmet and averaged across goals to create impact scores that measure how closely programs meet the overarching departmental mission and goals. Over the 4-year period, there is an increasing trend in program-specific impact scores across all programs except one, Healthy Kids Express Asthma, which had a slight decrease in year 4 only. Current work in measuring and assessing the population health impact of community benefit programs is mostly focused on quantifying dollars invested into community benefit work rather than measuring the quality and impact of services. This article provides a methodology for measuring population health impact of community benefit programs that can be used to evaluate the effort of hospitals in providing community benefit. This is particularly relevant in our changing health care climate, as hospitals are being asked to justify community benefit and make meaningful contributions to population health. The Patient Protection and Affordable Care Act includes a change to the IRS 990 Schedule H, requiring nonprofit hospitals to submit a community health needs assessment every 3 years, and requires evaluation of program effectiveness; yet, it does not require any quantification of the impact of community benefit programs. The IRS Schedule H 990 policies could be strengthened by requiring an impact evaluation such as outlined in this article. As hospitals are being asked to justify community benefit and make meaningful contributions to population health, impact evaluations can be utilized to demonstrate the cumulative community benefit of programs and assess population health impact of community benefit programs.

Participants' perceptions of the 1997-1998 Missouri State Parks Passport Program

Treesearch

Yi-Jin Ye; Jaclyn Card

2002-01-01

Service quality is increasingly important to park managers. Recreation and park evaluation measures the implementation and outcome of programs for decision-making. Decisions based on evaluations are often concerned with improving the quality of the program for participants. The purpose of the study was to evaluate the Missouri State Parks Passport Program (MSPPP) by...

List randomization for soliciting experience of intimate partner violence: Application to the evaluation of Zambia's unconditional child grant program.

PubMed

Peterman, Amber; Palermo, Tia M; Handa, Sudhanshu; Seidenfeld, David

2018-03-01

Social scientists have increasingly invested in understanding how to improve data quality and measurement of sensitive topics in household surveys. We utilize the technique of list randomization to collect measures of physical intimate partner violence in an experimental impact evaluation of the Government of Zambia's Child Grant Program. The Child Grant Program is an unconditional cash transfer, which targeted female caregivers of children under the age of 5 in rural areas to receive the equivalent of US $24 as a bimonthly stipend. The implementation results show that the list randomization methodology functioned as planned, with approximately 15% of the sample identifying 12-month prevalence of physical intimate partner violence. According to this measure, after 4 years, the program had no measurable effect on partner violence. List randomization is a promising approach to incorporate sensitive measures into multitopic evaluations; however, more research is needed to improve upon methodology for application to measurement of violence. Copyright © 2017 John Wiley & Sons, Ltd.

Critical Measurement Issues in Translational Research

ERIC Educational Resources Information Center

Glasgow, Russell E.

2009-01-01

This article summarizes critical evaluation needs, challenges, and lessons learned in translational research. Evaluation can play a key role in enhancing successful application of research-based programs and tools as well as informing program refinement and future research. Discussion centers on what is unique about evaluating programs and…

Connected Vehicle Pilot Deployment Program phase 1 : performance measurement and evaluation support plan : Tampa (THEA) : final report.

DOT National Transportation Integrated Search

2016-03-14

The Performance Measurement and Evaluation Support Plan for the Connected Vehicle Pilot Deployment Program Phase 1, Tampa Hillsborough Expressway Authority, outlines the goals and objectives for the Pilot as well as the proposed performance metrics. ...

Performance Measures in Out-of-School Time Evaluation. Out-of-School Time Evaluation Snapshot, Number 3.

ERIC Educational Resources Information Center

Harvard University Harvard Family Research Project, 2004

2004-01-01

Increasingly, out-of-school time programs are being asked to provide data that documents their progress and demonstrates their results. But for many OST programs and their over-worked staff this added responsibility is the "straw that broke the camel?s back." Across the country there is a cry from programs for help in deciding what to measure and…

Barriers to Evaluation

ERIC Educational Resources Information Center

Charley, Lois

2007-01-01

According to the Department of Health and Human Services, "good program evaluations assess programs performance, measure impacts on families and communities, and document program successes." With this information, programs are able to direct limited resources to where they are most needed and most effective in their communities. This article is…

HISTORY AND ACCOMPLISHMENTS OF THE US EPA'S SUPERFUND INNOVATIVE TECHNOLOGY EVALUATION (SITE) MONITORING AND MEASUREMENT (MMT) PROGRAM

EPA Science Inventory

This manuscript presents the history and evolution of the U.S. Environmental Protection Agency's (EPA) Superfund Innovative Technology Evaluation (SITE) Monitoring and Measurement Technology (MMT) Program. This includes a discussion of how the fundamental concepts of a performanc...

The Efficacy of Stuttering Measurement Training: Evaluating Two Training Programs

PubMed Central

Bainbridge, Lauren A.; Stavros, Candace; Ebrahimian, Mineh; Wang, Yuedong

2015-01-01

Purpose Two stuttering measurement training programs currently used for training clinicians were evaluated for their efficacy in improving the accuracy of total stuttering event counting. Method Four groups, each with 12 randomly allocated participants, completed a pretest–posttest design training study. They were evaluated by their counts of stuttering events on eight 3-min audiovisual speech samples from adults and children who stutter. Stuttering judgment training involved use of either the Stuttering Measurement System (SMS), Stuttering Measurement Assessment and Training (SMAAT) programs, or no training. To test for the reliability of any training effect, SMS training was repeated with the 4th group. Results Both SMS-trained groups produced approximately 34% improvement, significantly better than no training or the SMAAT program. The SMAAT program produced a mixed result. Conclusions The SMS program was shown to produce a “medium” effect size improvement in the accuracy of stuttering event counts, and this improvement was almost perfectly replicated in a 2nd group. Half of the SMAAT judges produced a 36% improvement in accuracy, but the other half showed no improvement. Additional studies are needed to demonstrate the durability of the reported improvements, but these positive effects justify the importance of stuttering measurement training. PMID:25629956

The efficacy of stuttering measurement training: evaluating two training programs.

PubMed

Bainbridge, Lauren A; Stavros, Candace; Ebrahimian, Mineh; Wang, Yuedong; Ingham, Roger J

2015-04-01

Two stuttering measurement training programs currently used for training clinicians were evaluated for their efficacy in improving the accuracy of total stuttering event counting. Four groups, each with 12 randomly allocated participants, completed a pretest-posttest design training study. They were evaluated by their counts of stuttering events on eight 3-min audiovisual speech samples from adults and children who stutter. Stuttering judgment training involved use of either the Stuttering Measurement System (SMS), Stuttering Measurement Assessment and Training (SMAAT) programs, or no training. To test for the reliability of any training effect, SMS training was repeated with the 4th group. Both SMS-trained groups produced approximately 34% improvement, significantly better than no training or the SMAAT program. The SMAAT program produced a mixed result. The SMS program was shown to produce a "medium" effect size improvement in the accuracy of stuttering event counts, and this improvement was almost perfectly replicated in a 2nd group. Half of the SMAAT judges produced a 36% improvement in accuracy, but the other half showed no improvement. Additional studies are needed to demonstrate the durability of the reported improvements, but these positive effects justify the importance of stuttering measurement training.

An Evaluation of a Mixed-Gender Sexual Assault Prevention Program

ERIC Educational Resources Information Center

Bradley, April R.; Yeater, Elizabeth A.; O'Donohue, William

2009-01-01

This study evaluated the short-term effectiveness of a mixed-gender sexual assault prevention program developed for college students. Program participants (n = 177) were compared to non-program participants (n = 132) prior to the program and during a 2-week follow-up period on measures of rape myths, victim empathy, perceived negative consequences…

What does a Performance Measurement System Tell Us about the National Comprehensive Cancer Control Program?

PubMed Central

Townsend, Julie S.; Moore, Angela R.; Mulder, Tiffani N.; Boyd, Mary

2015-01-01

Context The National Comprehensive Cancer Control Program (NCCCP) performance measurement system seeks to understand both the processes that funded programs undertake with their respective coalitions to implement the objectives of their cancer plans and outcomes of those efforts. Objective To identify areas of achievement and technical assistance needs of NCCCP awardees. Design Program performance was assessed through surveys completed by program directors on performance indicators in 2009 and 2010 and queries from a web-based management information system in 2011 and 2012. Setting Programs funded by CDC’s NCCCP. Participants 69 programs. Main Outcome Measure(s) The key performance measures assessed were: inclusion of diverse partners and key sectors in cancer coalitions; partners’ involvement in activities; receiving in-kind resources from partners; using evidence-based interventions and data for setting priorities; conducting program evaluation; using community- or organization-level strategies to address cancer control efforts; and demonstrating progress toward achieving health outcomes. Results Most programs reported having active coalitions that represent diverse organizational sectors. Nearly all programs routinely assess the burden of cancer. In-kind resources to implement activities peaked at $64,716 in the second year of a five year funding cycle, and declined in subsequent project years. By year 3, over 70% of programs reported having an evaluation plan. While programs reported that nearly two-thirds of their interventions were evidence-based, some programs implemented non-evidence-based interventions. A majority of programs successfully used at least one community- or organization-level change strategy. However, many programs did not incorporate objectives linked to health outcomes as they reported progress in implementing interventions. Conclusions: While NCCCP programs were strong at building and maintaining infrastructure, some programs may need additional technical assistance to increase the adoption of evidence-based interventions, develop solid and responsive evaluation plans, and better link efforts to population-based measures that demonstrate impact toward reducing the burden of cancer. PMID:25136936

Sustained Implementation Support Scale: Validation of a Measure of Program Characteristics and Workplace Functioning for Sustained Program Implementation.

PubMed

Hodge, Lauren M; Turner, Karen M T; Sanders, Matthew R; Filus, Ania

2017-07-01

An evaluation measure of enablers and inhibitors to sustained evidence-based program (EBP) implementation may provide a useful tool to enhance organizations' capacity. This paper outlines preliminary validation of such a measure. An expert informant and consumer feedback approach was used to tailor constructs from two existing measures assessing key domains associated with sustained implementation. Validity and reliability were evaluated for an inventory composed of five subscales: Program benefits, Program burden, Workplace support, Workplace cohesion, and Leadership style. Exploratory and confirmatory factor analysis with a sample of 593 Triple P-Positive Parenting Program-practitioners led to a 28-item scale with good reliability and good convergent, discriminant, and predictive validity. Practitioners sustaining implementation at least 3 years post-training were more likely to have supervision/peer support, reported higher levels of program benefit, workplace support, and positive leadership style, and lower program burden compared to practitioners who were non-sustainers.

General test plan redundant sensor strapdown IMU evaluation program

NASA Technical Reports Server (NTRS)

Hartwell, T.; Irwin, H. A.; Miyatake, Y.; Wedekind, D. E.

1971-01-01

The general test plan for a redundant sensor strapdown inertial measuring unit evaluation program is presented. The inertial unit contains six gyros and three orthogonal accelerometers. The software incorporates failure detection and correction logic and a land vehicle navigation program. The principal objective of the test is a demonstration of the practicability, reliability, and performance of the inertial measuring unit with failure detection and correction in operational environments.

Personal Benefits of a Health Evaluation and Enhancement Program

NASA Technical Reports Server (NTRS)

Heinzelmann, F.; Durbeck, D. C.

1970-01-01

A study was made of the benefits reported by participants in a health evaluation and enhancement program dealing with physical activity. Program benefits were identified and defined in regard to three major areas: program effects on work; program effects on health; and program effects on habits and behavior. A strong positive and consistent relationship was found between reported benefits in each of these areas and measures of improvement in cardiovascular functioning based on treadmill performance. Significant differences in these measures of improvement were also found between participants who reported program benefits and those persons who did not. These findings provide a meaningful profile of the pattern of benefits generated by this kind of health program.

Evaluation of Supplemental Nutrition Assistance Program Education: Application of Behavioral Theory and Survey Validation

ERIC Educational Resources Information Center

Wyker, Brett A.; Jordan, Patricia; Quigley, Danielle L.

2012-01-01

Objective: Application of the Transtheoretical Model (TTM) to Supplemental Nutrition Assistance Program Education (SNAP-Ed) evaluation and development and validation of an evaluation tool used to measure TTM constructs is described. Methods: Surveys were collected from parents of children receiving food at Summer Food Service Program sites prior…

Summative Evaluation of the Foreign Credential Recognition Program. Final Report

ERIC Educational Resources Information Center

Human Resources and Skills Development Canada, 2010

2010-01-01

A summative evaluation of the Foreign Credential Recognition Program (FCRP) funded by Human Resources and Skills Development Canada (HRSDC) was conducted during the spring, summer and fall of 2008. The main objective of the evaluation was to measure the relevance, impacts, and cost-effectiveness of the program. Given the timing of the evaluation…

Evaluation of a Group-Based Trauma Recovery Program in Gaza: Students' Subjective Experiences

ERIC Educational Resources Information Center

Barron, Ian; Abdullah, Ghassan

2012-01-01

Internationally, evaluation of group-based trauma recovery programs has relied upon normative outcome measures, with no studies systematically analyzing children's subjective experience for program development. In contrast, the current study explored children's experience of a Gazan recovery program "in their own words." Twenty-four…

Evaluation of Carrying Capacity : Measure 7.1A of the Northwest Power Planning Council`s 1994 Fish and Wildlife Program : Report 1 of 4, Final Report.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Neitzel, Duane A.; Johnson, Gary E.

This report is one of four that the Pacific Northwest National Laboratory (PNNL) staff prepared to address Measure 7.1A in the Northwest Power Planning Council's (Council) Fish and Wildlife Program (Program) dated december 1994 (NPPC 1994). Measure 7.1A calls for the Bonneville Power Administration (BPA) to fund an evaluation of salmon survival, ecology, carrying capacity, and limiting factors in freshwater, estuarine, and marine habitats. Additionally, the Measure asks for development of a study plan based on critical uncertainties and research needs identified during the evaluation. This report deals with the evaluation of carrying capacity. It describes the analysis of differentmore » views of capacity as it relates to salmon survival and abundance. The report ends with conclusions and recommendations for studying carrying capacity.« less

Chapter 11: Sample Design Cross-Cutting Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Khawaja, M. Sami; Rushton, Josh

Evaluating an energy efficiency program requires assessing the total energy and demand saved through all of the energy efficiency measures provided by the program. For large programs, the direct assessment of savings for each participant would be cost-prohibitive. Even if a program is small enough that a full census could be managed, such an undertaking would almost always be an inefficient use of evaluation resources. The bulk of this chapter describes methods for minimizing and quantifying sampling error. Measurement error and regression error are discussed in various contexts in other chapters.

7 CFR 1709.216 - Evaluation criteria and weights.

Code of Federal Regulations, 2013 CFR

2013-01-01

... announcement. (a) Program Design. Reviewers will consider the financial viability of the applicant's revolving... less severe physical and economic challenges. (c) Program evaluation and performance measures...

7 CFR 1709.216 - Evaluation criteria and weights.

Code of Federal Regulations, 2010 CFR

2010-01-01

... announcement. (a) Program Design. Reviewers will consider the financial viability of the applicant's revolving... less severe physical and economic challenges. (c) Program evaluation and performance measures...

7 CFR 1709.216 - Evaluation criteria and weights.

Code of Federal Regulations, 2012 CFR

2012-01-01

... announcement. (a) Program Design. Reviewers will consider the financial viability of the applicant's revolving... less severe physical and economic challenges. (c) Program evaluation and performance measures...

7 CFR 1709.216 - Evaluation criteria and weights.

Code of Federal Regulations, 2011 CFR

2011-01-01

... announcement. (a) Program Design. Reviewers will consider the financial viability of the applicant's revolving... less severe physical and economic challenges. (c) Program evaluation and performance measures...

7 CFR 1709.216 - Evaluation criteria and weights.

Code of Federal Regulations, 2014 CFR

2014-01-01

... announcement. (a) Program Design. Reviewers will consider the financial viability of the applicant's revolving... less severe physical and economic challenges. (c) Program evaluation and performance measures...

Tailored program evaluation: Past, present, future.

PubMed

Suggs, L Suzanne; Cowdery, Joan E; Carroll, Jennifer B

2006-11-01

This paper discusses measurement issues related to the evaluation of computer-tailored health behavior change programs. As the first generation of commercially available tailored products is utilized in health promotion programming, programmers and researchers are becoming aware of the unique challenges that the evaluation of these programs presents. A project is presented that used an online tailored health behavior assessment (HBA) in a worksite setting. Process and outcome evaluation methods are described and include the challenges faced, and strategies proposed and implemented, for meeting them. Implications for future research in tailored program development, implementation, and evaluation are also discussed.

Local/State Bilingual Project. 1981-82 Final Technical Report. Appendixes.

ERIC Educational Resources Information Center

Austin Independent School District, TX. Office of Research and Evaluation.

The 1981-82 Local/State Bilingual Program Technical Report addresses the evaluation questions of the Local/State Bilingual Program Evaluation Design. It is organized into six appendixes. Each appendix reports the information collected by a specific measure. Each appendix consists of (1) an instrument description, (2) purpose of the measure, (3)…

Determining Appropriate Criteria in the Evaluation of Correctional Mental Health Treatment for Inmates.

ERIC Educational Resources Information Center

Alexander, Rudolph, Jr.

1992-01-01

Notes that, despite changed goals of mental health treatment for inmates, some prison treatment programs still evaluate effects in terms of adjustment indicators. Discusses and critiques proposals in Ohio to use adjustment indicators as outcome measures for new treatment program for mentally ill inmates. Discusses proper outcome measures for…

Changing Preschoolers' Attitudes toward Animals: A Zoo Program and an Evaluation.

ERIC Educational Resources Information Center

Reames, Judi; Rajecki, D. W.

A zoo outreach program for preschoolers was evaluated by assessing the reactions of the children themselves. Children's attitudes toward certain animals were measured before and after live exposure to those animals in regular preschool settings. The attitudinal measure was a nonverbal expression of affect as elicited by pictures. Additionally,…

Emergency radiobioassay preparedness exercises through the NIST radiochemistry intercomparison program.

PubMed

Nour, Svetlana; LaRosa, Jerry; Inn, Kenneth G W

2011-08-01

The present challenge for the international emergency radiobioassay community is to analyze contaminated samples rapidly while maintaining high quality results. The National Institute of Standards and Technology (NIST) runs a radiobioassay measurement traceability testing program to evaluate the radioanalytical capabilities of participating laboratories. The NIST Radiochemistry Intercomparison Program (NRIP) started more than 10 years ago, and emergency performance testing was added to the program seven years ago. Radiobioassay turnaround times under the NRIP program for routine production and under emergency response scenarios are 60 d and 8 h, respectively. Because measurement accuracy and sample turnaround time are very critical in a radiological emergency, response laboratories' analytical systems are best evaluated and improved through traceable Performance Testing (PT) programs. The NRIP provides participant laboratories with metrology tools to evaluate their performance and to improve it. The program motivates the laboratories to optimize their methodologies and minimize the turnaround time of their results. Likewise, NIST has to make adjustments and periodical changes in the bioassay test samples in order to challenge the participating laboratories continually. With practice, radioanalytical measurements turnaround time can be reduced to 3-4 h.

Monitoring and evaluating transition and sustainability of donor-funded programs: Reflections on the Avahan experience.

PubMed

Bennett, Sara; Ozawa, Sachiko; Rodriguez, Daniela; Paul, Amy; Singh, Kriti; Singh, Suneeta

2015-10-01

In low and middle-income countries, programs funded and implemented by international donors frequently transition to local funding and management, yet such processes are rarely evaluated. We reflect upon experience evaluating the transition of a large scale HIV/AIDS prevention program in India, known as Avahan, in order to draw lessons about transition evaluation approaches and implementation challenges. In terms of conceptualizing the transition theory, the evaluation team identified tensions between the idea of institutionalizing key features of the Avahan program, and ensuring program flexibility to promote sustainability. The transition was planned in three rounds allowing for adaptations to transition intervention and program design during the transition period. The assessment team found it important to track these changes in order to understand which strategies and contextual features supported transition. A mixed methods evaluation was employed, combining semi-structured surveys of transitioning entities (conducted pre and post transition), with longitudinal case studies. Qualitative data helped explain quantitative findings. Measures of transition readiness appeared robust, but we were uncertain of the robustness of institutionalization measures. Finally, challenges to the implementation of such an evaluation are discussed. Given the scarceness of transition evaluations, the lessons from this evaluation may have widespread relevance. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Evaluating Cross-Cutting Approaches to Chronic Disease Prevention and Management: Developing a Comprehensive Evaluation

PubMed Central

Jernigan, Jan; Barnes, Seraphine Pitt; Shea, Pat; Davis, Rachel; Rutledge, Stephanie

2017-01-01

We provide an overview of the comprehensive evaluation of State Public Health Actions to Prevent and Control Diabetes, Heart Disease, Obesity and Associated Risk Factors and Promote School Health (State Public Health Actions). State Public Health Actions is a program funded by the Centers for Disease Control and Prevention to support the statewide implementation of cross-cutting approaches to promote health and prevent and control chronic diseases. The evaluation addresses the relevance, quality, and impact of the program by using 4 components: a national evaluation, performance measures, state evaluations, and evaluation technical assistance to states. Challenges of the evaluation included assessing the extent to which the program contributed to changes in the outcomes of interest and the variability in the states’ capacity to conduct evaluations and track performance measures. Given the investment in implementing collaborative approaches at both the state and national level, achieving meaningful findings from the evaluation is critical. PMID:29215974

Implementing and Evaluating School-Based Primary Prevention Programs and the Importance of Differential Effects on Outcomes

ERIC Educational Resources Information Center

Hussey, David L.; Flannery, Daniel J.

2007-01-01

In 2004, Second Step (Committee for Children, 2002), a violence prevention program, was implemented in the Cleveland Heights-University Heights school district for 1,416 K through second grade students. Both process and outcome measures were used to evaluate program impact and examine issues related to the implementation and evaluation of…

Annual Evaluation: Elementary and Secondary Act of 1965 - Title I, Maine, FY 1970.

ERIC Educational Resources Information Center

Maine State Dept. of Education, Augusta.

This evaluation attempts to measure the extent and effectiveness of ESEA Title I programs designed to meet the needs of disadvantaged children and apprizes the public and the legislature of program outcomes. In keeping with USOE requirements for evaluating Title I programs, this document is constructed of (1) responses to USOE probes by…

Cancer Education Program Evaluation: A Responsive Approach to Planning an Evaluation and Initial Results.

ERIC Educational Resources Information Center

Pearsol, James A.

This paper describes evaluation planning for the Cancer Education Program (CEP) at Ohio State University (OSU). The three-year OSU CEP project was designed as a multidisciplinary cancer education program. A responsive method, which trades off some measurement precision in order to increase the usefulness of the findings, was employed in the…

The development of a streamlined, coordinated and sustainable evaluation methodology for a diverse chronic disease management program.

PubMed

Berlowitz, David J; Graco, Marnie

2010-05-01

The Northern Alliance Hospital Admission Risk Program-Chronic Disease Management comprises 13 services delivering care to those with chronic disease and older people with complex care needs, who are frequent hospital users. To develop and implement a system-wide approach to the evaluation of this existing program. The Northern Clinical Research Centre audited all existing, routinely collected administrative data within the program and then met with each service to develop service specific outcome measures. The evaluators then developed and implemented a system-wide evaluation approach to measure performance in terms of: client profile; access and entry; service efficiency; client outcomes; and hospital demand. Data are collected electronically and more than 80% are derived from existing, administrative datasets, minimising staff and client burden. Additional data include client outcomes and a health related quality of life measure. The preliminary twelve month data suggest that clients have the equivalent of 'fair' or 'poor' self-reported health status (n = 862) and the average health utility scores are significantly (P < 0.05) worse than population control data. These analyses reveal, for the first time, that the Northern Alliance Hospital Admission Risk Program-Chronic Disease Management program is targeting appropriate clients. This methodology will enable many prospective assessments to be performed including; client outcome evaluation, service model comparisons, and cost-utility analyses. This evaluation approach demonstrates the feasibility of a highly coordinated 'whole of system' evaluation. Such an approach may ultimately contribute to the development of evidence-based policy.

Evaluating School Violence Programs. Urban Diversity Series.

ERIC Educational Resources Information Center

Flaxman, Erwin, Ed.

The essays in this collection review information, resources, and guidelines on the basic principles of internal evaluations of school violence programs. This information can help staff with program definition, planning, implementation, continuous improvement, and measurement of outcomes. "Introduction" (Erwin Flaxman) examines types of…

Evaluation readiness: improved evaluation planning using a data inventory framework.

PubMed

Cohen, A B; Hall, K C; Cohodes, D R

1985-01-01

Factors intrinsic to many programs, such as ambiguously stated objectives, inadequately defined performance measures, and incomplete or unreliable databases, often conspire to limit the evaluability of these programs. Current evaluation planning approaches are somewhat constrained in their ability to overcome these obstacles and to achieve full preparedness for evaluation. In this paper, the concept of evaluation readiness is introduced as a complement to other evaluation planning approaches, most notably that of evaluability assessment. The basic products of evaluation readiness--the formal program definition and the data inventory framework--are described, along with a guide for assuring more timely and appropriate evaluation response capability to support the decision making needs of program managers. The utility of evaluation readiness for program planning, as well as for effective management, is also discussed.

Evaluation of the Meaning of Life Program in Israel

ERIC Educational Resources Information Center

Kasler, Jonathan; White, Gwyne W.; Elias, Maurice J.

2013-01-01

During the 2009-2010 academic year, 10 schools participated in the Meaning of Life educational program, an adaption of the popular U.S. Laws of Life program. The program sought to encourage each participant to develop a personal approach to finding meaning in life. To evaluate the success of the program, we conducted a study to compare measures of…

Measuring techniques in the measuring program for the windpowered unit GROWIAN

NASA Astrophysics Data System (ADS)

Koerber, F.

1984-02-01

The measuring strategy in the GROWIAN program and the measuring systems are presented. Power, load, and behavior during operation were checked. The determining physical characteristics, mainly mechanical and electrical, are obtained with 200 measuring points; they are recorded and evaluated by a data processing system.

Broadening measures of success: results of a behavioral health translational research training program.

PubMed

Baldwin, Julie A; Williamson, Heather J; Eaves, Emery R; Levin, Bruce L; Burton, Donna L; Massey, Oliver T

2017-07-24

While some research training programs have considered the importance of mentoring in inspiring professionals to engage in translational research, most evaluations emphasize outcomes specific to academic productivity as primary measures of training program success. The impact of such training or mentoring programs on stakeholders and local community organizations engaged in translational research efforts has received little attention. The purpose of this evaluation is to explore outcomes other than traditional academic productivity in a translational research graduate certificate program designed to pair graduate students and behavioral health professionals in collaborative service-learning projects. Semi-structured qualitative interviews with scholars, community mentors, and academic mentors were conducted regarding a translational research program to identify programmatic impacts. Interviews were transcribed and coded by the research team to identify salient themes related to programmatic outcomes. Results are framed using the Translational Research Impact Scale which is organized into three overarching domains of potential impact: (1) research-related impacts, (2) translational impacts, and (3) societal impacts. This evaluation demonstrates the program's impact in all three domains of the TRIS evaluation framework. Graduate certificate participants (scholars) reported that gaining experience in applied behavioral health settings added useful skills and expertise to their present careers and increased their interest in pursuing translational research. Scholars also described benefits resulting from networks gained through participation in the program, including valuable ties between the university and community behavioral health organizations. This evaluation of the outcomes of a graduate certificate program providing training in translational research highlights the need for more community-oriented and practice-based measures of success. Encouraging practitioner involvement in translational research is vital to translate knowledge into practice and to enable practice-based needs to inform research and policy. A more flexible approach to measuring programmatic success in research training programs can help bridge the knowledge translation gap.

The Definition and Measurement of Evaluation Quality as a Management Tool.

ERIC Educational Resources Information Center

Chelimsky, Eleanor

1983-01-01

Evaluation management requires a system for measuring, improving, and accounting for quality. The Institute for Program Evaluation in the United States General Accounting Office is developing such a system. (Author)

Measurement Program for Evaluation of Concrete Ties and Fastenings in Transit Track

DOT National Transportation Integrated Search

1979-03-01

This report outlines a measurement program to obtain data on the performance of standard tie designs and associated fastening systems under field service conditions. In addition, the program identifies limited data to be obtained from a wood tie trac...

Writing objectives and evaluating learning in the affective domain.

PubMed

Maier-Lorentz, M M

1999-01-01

Staff educators recognize the importance of affective competency for effective nursing practice. Inservice programs must include affective learning with objectives stated in measurable terms. Staff educators often express frustration in developing affective objectives and evaluating the learning outcome because attitudes and feelings are usually inferred from observations. This article presents affective learning objectives for a gerontological nursing inservice program and a rating scale that measures attitudes to evaluate the learning outcome.

Promoting sustainability in quality improvement: an evaluation of a web-based continuing education program in blood pressure measurement.

PubMed

Block, Lauren; Flynn, Sarah J; Cooper, Lisa A; Lentz, Caroline; Hull, Tammie; Dietz, Katherine B; Boonyasai, Romsai T

2018-01-10

The accuracy of blood pressure measurement is variable in office-based settings. Even when staff training programs are effective, knowledge and skills decay over time, supporting the need for ongoing staff training. We evaluated whether a web-based continuing education program in blood pressure measurement reinforced knowledge and skills among clinical staff and promoted sustainability of an existing quality improvement program. Medical assistants and nurses at six primary care clinics within a health system enrolled in a 30-min online educational program designed to refresh their knowledge of blood pressure measurement. A 20-question pre- and post-intervention survey addressed learners' knowledge and attitudes. Direct observation of blood pressure measurement technique before and after the intervention was performed. Differences in responses to pre- and post-module knowledge and attitudes questions and in observation data were analyzed using chi-square tests and simple logistic regression. All 88 clinical staff members participated in the program and completed the evaluation survey. Participants answered 80.6% of questions correctly before the module and 93.4% afterwards (p < 0.01). Scores improved significantly among staff from all job types. Licensed practical nurses and staff who had been in their current job at least a year were more likely to answer questions correctly than registered nurses and those in their current job less than a year. Attitudes toward correct blood pressure measurement were high at baseline and did not improve significantly. Prior to the intervention, staff adhered to 9 of 18 elements of the recommended technique during at least 90% of observations. Following the program, staff was more likely to explain the protocol, provide a rest period, measure an average blood pressure, and record the average blood pressure, but less likely to measure blood pressure with the arm at heart level and use the right arm. We designed, implemented, and evaluated a web-based educational program to improve knowledge, skills, and attitudes in blood pressure measurement and use of an automated device among nurses and medical assistants in ambulatory care. The program reinforced knowledge related to recommended blood pressure measurement technique. Retrospectively registered with ClincalTrials.gov on March 22, 2012; registration number NCT01566864 .

A Measure of the Child Care Ecology: Day Care Program Compliance with State Regulations.

ERIC Educational Resources Information Center

Fiene, Richard

Between July 1978 and June 1980 a program evaluation was undertaken in Pennsylvania in order to measure compliance with state day care licensing regulations. The evaluation involved approximately 1000 licensed/approved child care centers and 50,000 children. Statistical data indicate that by the period April to June 1980 the statewide compliance…

The ambiguities of performance-based governance reforms in Italy: Reviving the fortunes of evaluation and performance measurement.

PubMed

Marra, Mita

2018-08-01

Over the past two decades, Italy's administrative reforms have institutionalized evaluation to improve program effectiveness, staff productivity, and results-driven accountability against waste and corruption. Across ministries, regional governments, universities, schools and environmental protection agencies, seemingly unexpected consequences have emerged out of the implementation of performance measurement and evaluation regimes within public organizations. Formal compliance to legally binding evaluation procedures, judicially-sanctioned managerial accountability and lack of cross-agency coordination coupled with long-standing cultural separations among evaluators are some of the ambiguities associated with a performance-based governance system within Italian public administration. Building upon the 'new governane theory,' and qualitative fieldwork, I explore the political consequences of evaluation and performance measurement for possible improvements. From a normative perspective, greater integration between program evaluation and performance measurement can support organizational learning and democratic accountability both at the central and local level. Copyright © 2017 Elsevier Ltd. All rights reserved.

Faculty performance evaluation in accredited U.S. public health graduate schools and programs: a national study.

PubMed

Gimbel, Ronald W; Cruess, David F; Schor, Kenneth; Hooper, Tomoko I; Barbour, Galen L

2008-10-01

To provide baseline data on evaluation of faculty performance in U.S. schools and programs of public health. The authors administered an anonymous Internet-based questionnaire using PHP Surveyor. The invited sample consisted of individuals listed in the Council on Education for Public Health (CEPH) Directory of Accredited Schools and Programs of Public Health. The authors explored performance measures in teaching, research, and service, and assessed how faculty performance measures are used. A total of 64 individuals (60.4%) responded to the survey, with 26 (40.6%) reporting accreditation/reaccreditation by CEPH within the preceding 24 months. Although all schools and programs employ faculty performance evaluations, a significant difference exists between schools and programs in the use of results for merit pay increases and mentoring purposes. Thirty-one (48.4%) of the organizations published minimum performance expectations. Fifty-nine (92.2%) of the respondents counted number of publications, but only 22 (34.4%) formally evaluated their quality. Sixty-two (96.9%) evaluated teaching through student course evaluations, and only 29 (45.3%) engaged in peer assessment. Although aggregate results of teaching evaluation are available to faculty and administrators, this information is often unavailable to students and the public. Most schools and programs documented faculty service activities qualitatively but neither assessed it quantitatively nor evaluated its impact. This study provides insight into how schools and programs of public health evaluate faculty performance. Results suggest that although schools and programs do evaluate faculty performance on a basic level, many do not devote substantial attention to this process.

A Descriptive Study of Nursing Peer-Review Programs in US Magnet® Hospitals.

PubMed

Roberts, Holli; Cronin, Sherill Nones

2017-04-01

The goal of this study was to assess the types of nursing peer review (NPR) programs in US Magnet® organizations. The 2 most predominant models of NPR programs in the literature are performance evaluation and clinical peer review. The literature on clinical peer review is primarily descriptive, outlining structures and anecdotal outcomes. Participants from hospitals holding Magnet recognition were selected using a stratified random-sampling method. A survey developed by the researchers assessed the presence of NPR. If clinical NPR was in place, program design, evaluation measurements, and barriers were explored. Findings suggest wide variability in NPR models. More than one-third of the respondents conduct peer evaluation as the only mechanism of NPR. Most hospitals with a clinical peer-review program reported a case review structure and process measurements not supported by data. The variations noted in this study suggest more research is needed to measure the effectiveness of NPR models and associated outcomes.

Washington State Even Start 1993-1994: Final Evaluation. A Report to the Office of Adult Literacy.

ERIC Educational Resources Information Center

Iglitzin, Lynne; Wandschneider, Mary

All 18 Washington State Even Start sites participated in the program's evaluation. Site coordinators administered the assessment and evaluation measures to the adults served by the program and to teachers working with children at both entry and exit from the program. An indepth study was conducted of 134 families for whom there were complete sets…

A Program to Improve Social Studies Instruction in the Ottumwa Community Schools. Surveys, Grades 3-12, Measurement Instruments, Project #1009.

ERIC Educational Resources Information Center

Ahrens, Willis

As part of a social studies instruction improvement program, measurement instruments were developed to measure attitudes of and evaluate courses for teachers and students in grades 3-12. The measurement instruments presented are surveys used in the social studies program. The purposes of the improvement project are to use the multimedia approach…

77 FR 38071 - Council on Graduate Medical Education; Notice of Meeting

Federal Register 2010, 2011, 2012, 2013, 2014

2012-06-26

... graduate medical education, evaluation of teaching programs especially in terms of meeting community needs...' development of performance measures and methods of longitudinal evaluation specific to the training programs...

SUPERFUND INNOVATIVE TECHNOLOGY EVALUATION PROGRAM: INNOVATION MAKING A DIFFERENCE

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program encourages commercialization of innovative technologies for characterizing and remediating hazardous waste site contamination through four components: Demonstration, Emerging Technology, and Monitoring & Measurement Pr...

Training the Poor. A Benefit-Cost Analysis of Manpower Programs in the U.S. Antipoverty Program.

ERIC Educational Resources Information Center

Sewell, D. O.

This report critically reviews past benefit-cost studies of manpower programs and offers an alternative methodology for program evaluation. Because of selective admission criteria and a tendency to attribute all income changes to training, previous studies have had an upward bias in measuring the effectiveness of training. This study evaluates a…

Measuring Cooperative Biological Engagement Program (CBEP) Performance: Capacities, Capabilities, and Sustainability Enablers for Biorisk Management and Biosurveillance

DTIC Science & Technology

2014-01-01

valid OMB control number. 1. REPORT DATE 2014 2. REPORT TYPE 3. DATES COVERED 00-00-2014 to 00-00-2014 4. TITLE AND SUBTITLE Measuring...should approaches to monitoring program performance. Recognizing this, Congress requested that the Department of Defense improve metrics for measuring...Cooperative Biological Engagement Program Performance broader community of program evaluation practitioners, the work advances innovative approaches

Care management program evaluation: constituents, conflicts, and moves toward standardization.

PubMed

Long, D Adam; Perry, Theodore L; Pelletier, Kenneth R; Lehman, Gregg O

2006-06-01

Care management program evaluations bring together constituents from finance, medicine, and social sciences. The differing assumptions and scientific philosophies that these constituents bring to the task often lead to frustrations and even contentions. Given the forms and variations of care management programs, the difficulty associated with program outcomes measurement should not be surprising. It is no wonder then that methods for clinical and economic evaluations of program efficacy continue to be debated and have yet to be standardized. We describe these somewhat hidden processes, examine where the industry stands, and provide recommendations for steps to standardize evaluation methodology.

Usability evaluation of user interface of thesis title review system

NASA Astrophysics Data System (ADS)

Tri, Y.; Erna, A.; Gellysa, U.

2018-03-01

Presentation of programs with user interface that can be accessed online through the website of course greatly provide user benefits. User can easily access the program they need. There are usability values that serve as a benchmark for the success of a user accessible program, ie efficiency, effectiveness, and convenience. These usability values also determine the development of the program for the better use. Therefore, on the review title thesis program that will be implemented in STT Dumai was measured usability evaluation. It aims to see which sides are not yet perfect and need to be improved to improve the performance and utilization of the program. Usability evaluation was measured by using smartPLS software. Database used was the result of respondent questionnaires that include questions about the experience when they used program. The result of a review of thesis title program implemented in STT Dumai has an efficiency value of 22.615, the effectiveness of 20.612, and satisfaction of 33.177.

Effectiveness of the surviving the Teens® suicide prevention and depression awareness program: an impact evaluation utilizing a comparison group.

PubMed

Strunk, Catherine M; King, Keith A; Vidourek, Rebecca A; Sorter, Michael T

2014-12-01

Youth suicide is a serious public health issue in the United States. It is currently the third leading cause of death for youth aged 10 to 19. School-based prevention programs may be an effective method of educating youth and enhancing their help-seeking. Most school-based suicide prevention programs have not been rigorously evaluated for their effectiveness. This evaluation employs a comparison group to measure whether program group participants differed significantly from comparison group participants on pretest-posttest measures while assessing the immediate impact of the Surviving the Teens® Suicide Prevention and Depression Awareness Program. Findings indicate several positive outcomes in program group students' suicide and depression knowledge, attitudes, confidence, and behavioral intentions compared with the comparison group. Suicide prevention specialists and prevention planners may benefit from study findings. © 2014 Society for Public Health Education.

38 CFR 1.15 - Standards for program evaluation.

Code of Federal Regulations, 2011 CFR

2011-07-01

... the data. (f) Each program evaluation requires a systematic research design to collect the data necessary to measure the objectives. This research design should conform to the following: (1) Rationale. The research design for each evaluation should contain a specific rationale and should be structured...

Students Training for Academic Readiness (STAR): Year Three Evaluation Report

ERIC Educational Resources Information Center

Rainey, Katharine; Sheehan, Daniel; Maloney, Catherine

2010-01-01

This report presents findings from the Year 3 evaluation of Texas' state-level Gaining Early Awareness and Readiness for Undergraduate Programs, or GEAR UP, grant. GEAR UP grant requirements include an evaluation component designed to assess program effectiveness and to measure progress toward project goals. To this end, the evaluation considers…

Performance Evaluation Tests for Environmental Research (PETER): evaluation of 114 measures

NASA Technical Reports Server (NTRS)

Bittner, A. C. Jr; Carter, R. C.; Kennedy, R. S.; Harbeson, M. M.; Krause, M.

1986-01-01

The goal of the Performance Evaluation Tests for Environmental Research (PETER) Program was to identify a set of measures of human capabilities for use in the study of environmental and other time-course effects. 114 measures studied in the PETER Program were evaluated and categorized into four groups based upon task stability and task definition. The Recommended category contained 30 measures that clearly obtained total stabilization and had an acceptable level of reliability efficiency. The Acceptable-But-Redundant category contained 15 measures. The 37 measures in the Marginal category, which included an inordinate number of slope and other derived measures, usually had desirable features which were outweighed by faults. The 32 measures in the Unacceptable category had either differential instability or weak reliability efficiency. It is our opinion that the 30 measures in the Recommended category should be given first consideration for environmental research applications. Further, it is recommended that information pertaining to preexperimental practice requirements and stabilized reliabilities should be utilized in repeated-measures environmental studies.

Outcome Evaluation of the Hands-On Parent Empowerment (HOPE) Program

ERIC Educational Resources Information Center

Leung, Cynthia; Tsang, Sandra; Dean, Suzanne

2011-01-01

This study evaluated the effectiveness of the HOPE program. Participants included 120 Chinese new immigrant parents with preschool children in Hong Kong from 13 preschools which were randomized into intervention group (HOPE) and comparison group (6-session parent education program). Parent participants completed measures on child behavior,…

Multiple Year Extension Program Outcomes & Impacts through Evaluation

ERIC Educational Resources Information Center

Hachfeld, Gary A.; Bau, David B.; Holcomb, C. Robert; Craig, J. William

2013-01-01

Dwindling public funding as well as greater competition for grant dollars create a challenge for Extension. For Extension to remain a financially viable organization, educators have to be able to produce substantive, measurable program outcomes and impacts. Evaluative data can inform program development and delivery, and helps administrators…

Local Government Leadership Education: Measuring the Impact of Leadership Skill Development on Public Officials

ERIC Educational Resources Information Center

Davis, Gregory A.; Lucente, Joe

2012-01-01

Many Extension leadership development programs have been evaluated for effectiveness. Little literature exists focusing on the evaluation of leadership development programs involving elected and appointed local officials. This article describes an annual program involving elected and appointed local officials and shares quantitative and…

Industrial Education, General Programs. Courseware Evaluation for Vocational and Technical Education.

ERIC Educational Resources Information Center

Turner, Gordon; And Others

This courseware evaluation rates a set of 25 industrial education general software programs developed by the Iowa Department of Public Instruction. (These programs--not contained in this document--include measurement drill, fraction drill, loan and interest cost analysis, assault vehicle speeds, sun's rays, Ohm's law, concrete calculations, roof…

Wood Programs. Courseware Evaluation for Vocational and Technical Education.

ERIC Educational Resources Information Center

Kaylor, Robert; And Others

This courseware evaluation rates the Wood Programs software developed by the Iowa Department of Public Instruction. (These programs--not contained in this document--include understanding board feet, wood characteristics, wood safety drill, wood dimensions, wood moisture, operating the table saw, radial arm, measurement drill, fraction drill, and…

Evaluation of a cross-cultural training program for Pakistani educators: Lessons learned and implications for program planning.

PubMed

Mazur, Rebecca; Woodland, Rebecca H

2017-06-01

In this paper, we share the results of a summative evaluation of PEILI, a US-based adult professional development/training program for secondary school Pakistani teachers. The evaluation was guided by the theories of cultural competence (American Psychological Association, 2003; Bamberger, 1999; Wadsworth, 2001) and established frameworks for the evaluation of professional development/training and instructional design (Bennett, 1975; Guskey, 2002; King, 2014; Kirkpatrick, 1967). The explicit and implicit stakeholder assumptions about the connections between program resources, activities, outputs, and outcomes are described. Participant knowledge and skills were measured via scores on a pre/posttest of professional knowledge, and a standards-based performance assessment rubric. In addition to measuring short-term program outcomes, we also sought to incorporate theory-driven thinking into the evaluation design. Hence, we examined participant self-efficacy and access to social capital, two evidenced-based determinants or "levers" that theoretically explain the transformative space between an intervention and its outcomes (Chen, 2012). Data about program determinants were collected and analyzed through a pre/posttest of self-efficacy and social network analysis. Key evaluation findings include participant acquisition of new instructional skills, increased self-efficacy, and the formation of a nascent professional support network. Lessons learned and implications for the design and evaluation of cross-cultural teacher professional development programs are discussed. Copyright © 2017 Elsevier Ltd. All rights reserved.

External quality-assurance programs managed by the U.S. Geological Survey in support of the National Atmospheric Deposition Program/National Trends Network

USGS Publications Warehouse

Latysh, Natalie E.; Wetherbee, Gregory A.

2005-01-01

The U.S. Geological Survey, Branch of Quality Systems, operates the external quality-assurance programs for the National Atmospheric Deposition Program/National Trends Network (NADP/NTN). Beginning in 1978, six different programs have been implemented?the intersite-comparison program, the blind-audit program, the sample-handling evaluation program, the field-audit program, the interlaboratory-comparison program, and the collocated-sampler program. Each program was designed to measure error contributed by specific components in the data-collection process. The intersite-comparison program, which was discontinued in 2004, was designed to assess the accuracy and reliability of field pH and specific-conductance measurements made by site operators. The blind-audit and sample-handling evaluation programs, which also were discontinued in 2002 and 2004, respectively, assessed contamination that may result from sampling equipment and routine handling and processing of the wet-deposition samples. The field-audit program assesses the effects of sample handling, processing, and field exposure. The interlaboratory-comparison program evaluates bias and precision of analytical results produced by the contract laboratory for NADP, the Illinois State Water Survey, Central Analytical Laboratory, and compares its performance with the performance of international laboratories. The collocated-sampler program assesses the overall precision of wet-deposition data collected by NADP/NTN. This report documents historical operations and the operating procedures for each of these external quality-assurance programs. USGS quality-assurance information allows NADP/NTN data users to discern between actual environmental trends and inherent measurement variability.

Updated operational protocols for the U.S. Geological Survey Precipitation Chemistry Quality Assurance Project in support of the National Atmospheric Deposition Program

USGS Publications Warehouse

Wetherbee, Gregory A.; Martin, RoseAnn

2017-02-06

The U.S. Geological Survey Branch of Quality Systems operates the Precipitation Chemistry Quality Assurance Project (PCQA) for the National Atmospheric Deposition Program/National Trends Network (NADP/NTN) and National Atmospheric Deposition Program/Mercury Deposition Network (NADP/MDN). Since 1978, various programs have been implemented by the PCQA to estimate data variability and bias contributed by changing protocols, equipment, and sample submission schemes within NADP networks. These programs independently measure the field and laboratory components which contribute to the overall variability of NADP wet-deposition chemistry and precipitation depth measurements. The PCQA evaluates the quality of analyte-specific chemical analyses from the two, currently (2016) contracted NADP laboratories, Central Analytical Laboratory and Mercury Analytical Laboratory, by comparing laboratory performance among participating national and international laboratories. Sample contamination and stability are evaluated for NTN and MDN by using externally field-processed blank samples provided by the Branch of Quality Systems. A colocated sampler program evaluates the overall variability of NTN measurements and bias between dissimilar precipitation gages and sample collectors.This report documents historical PCQA operations and general procedures for each of the external quality-assurance programs from 2007 to 2016.

Benchmarking. Issues in the Design and Implementation of a Benchmarking System for Employment and Training Programs for Young People.

ERIC Educational Resources Information Center

Coughlin, David C.; Bielen, Rhonda P.

This paper has been prepared to assist the United States Department of Labor to explore new approaches to evaluating and measuring the performance of employment and training activities for youth. As one of several tools for evaluating success of local youth training programs, "benchmarking" provides a system for measuring the development…

MEASUREMENT ISSUES IN HOME-VISITING RESEARCH WITHIN TRIBAL COMMUNITIES: CHALLENGES AND STRATEGIES.

PubMed

Whitesell, Nancy Rumbaugh; Bolan, Marc; Chomos, Julianna C; Heath, Debra; Miles, Jon; Salvador, Melina; Whitmore, Corrie; Barlow, Allison

2018-05-04

In this article, Tribal Maternal, Infant, and Early Childhood Home Visiting (MIECHV) grantees share strategies they have developed and adopted to address the most common barriers to effective measurement (and thus to effective evaluation) encountered in the course of implementation and evaluation of their home-visiting programs. We identify key challenges in measuring outcomes in Tribal MIECHV Programs and provide practical examples of various strategies used to address these challenges within diverse American Indian and Alaska Native cultural and contextual settings. Notably, high-quality community engagement is a consistent thread throughout these strategies and fundamental to successful measurement in these communities. These strategies and practices reflect the experiences and innovative solutions of practitioners working on the ground to deliver and evaluate intervention programs to tribal communities. They may serve as models for getting high-quality data to inform intervention while working within the constraints and requirements of program funding. The utility of these practical solutions extends beyond the Tribal MIECHV grantees and offers the potential to inform a broad array of intervention evaluation efforts in tribal and other community contexts. © 2018 Michigan Association for Infant Mental Health.

From Knowledge to Action: Tips for Encouraging and Measuring Program-Related Behavior Change

ERIC Educational Resources Information Center

Chazdon, Scott; Horntvedt, Jody; Templin, Elizabeth

2016-01-01

It is challenging to document the behavior changes that result from Extension programming. This article describes an evaluation method we call the "action items method." Unlike other approaches for measuring behavior change, this method requires program participants to define their own action plans as part of a program and then asks them…

Forging a Strategic and Comprehensive Approach to Evaluation within Public and Nonprofit Organizations: Integrating Measurement and Analytics within Evaluation

ERIC Educational Resources Information Center

Newcomer, Kathryn; Brass, Clinton T.

2016-01-01

The "performance movement" has been a subject of enthusiasm and frustration for evaluators. Performance measurement, data analytics, and program evaluation have been treated as different tasks, and those addressing them speak their own languages in their own circles. We suggest that situating performance measurement and data analytics…

Getting Ready for College: An Implementation and Early Impacts Study of Eight Texas Developmental Summer Bridge Programs. NCPR Brief

ERIC Educational Resources Information Center

Wathington, Heather D.; Barnett, Elisabeth A.; Weissman, Evan; Teres, Jedediah; Pretlow, Joshua; Nakanishi, Aki

2011-01-01

In 2009, the National Center for Postsecondary Research (NCPR) launched an evaluation of eight developmental summer bridge programs in Texas to assess whether these programs reduce the need for developmental coursework and improve student outcomes in college. The evaluation uses an experimental design to measure the effects of these programs on…

Evaluation of the Nursing Program at Caldwell Community College and Technical Institute--Summer, 1983.

ERIC Educational Resources Information Center

Pipes, V. David

In summer 1983, an evaluation of the nursing program at Caldwell Community College and Technical Institute was conducted to determine whether program objectives were being met, to measure program success, and to identify areas needing improvement. Surveys were sent to 19 early (pre-1978) and 47 recent Licensed Practical Nurse (LPN) graduates; 17…

Indicators of Program Quality, Measures of Performance & Standards. Adult Basic Education and ESL Programs in NJ. Summary Report.

ERIC Educational Resources Information Center

Merkel-Keller, Claudia; Streeter-Scrupski, Sandra

In 1992, adult education staff and adult literacy volunteer organizations developed 8 indicators of program quality to be used for evaluating adult basic education and English as a Second Language (ESL) programs in New Jersey. Performance standards were developed to match the standards. An evaluation was conducted to determine how the indicators…

Evaluation of a Hospital-Based Pneumonia Nurse Navigator Program.

PubMed

Seldon, Lisa E; McDonough, Kelly; Turner, Barbara; Simmons, Leigh Ann

2016-12-01

The aim of this study is to evaluate the effectiveness of a hospital-based pneumonia nurse navigator program. This study used a retrospective, formative evaluation. Data of patients admitted from January 2012 through December 2014 to a large community hospital with a primary or secondary diagnosis of pneumonia, excluding aspiration pneumonia, were used. Data included patient demographics, diagnoses, insurance coverage, core measures, average length of stay (ALOS), disposition, readmission rate, financial outcomes, and patient barriers to care were collected. Descriptive statistics and parametric testing were used to analyze data. Core measure performance was sustained at the 90th percentile 2 years after the implementation of the navigator program. The ALOS did not decrease to established benchmarks; however, the SD for ALOS decreased by nearly half after implementation of the navigator program, suggesting the program decreased the number and length of extended stays. Charges per case decreased by 21% from 2012 to 2014. Variable costs decreased by 4% over a 2-year period, which increased net profit per case by 5%. Average readmission payments increased by 8% from 2012 to 2014, and the net revenue per case increased by 8.3%. The pneumonia nurse navigator program may improve core measures, reduce ALOS, and increase net revenue. Future evaluations are necessary to substantiate these findings and optimize the cost and quality performance of navigator programs.

Measuring Stakeholder Participation in Evaluation: An Empirical Validation of the Participatory Evaluation Measurement Instrument (PEMI)

ERIC Educational Resources Information Center

Daigneault, Pierre-Marc; Jacob, Steve; Tremblay, Joel

2012-01-01

Background: Stakeholder participation is an important trend in the field of program evaluation. Although a few measurement instruments have been proposed, they either have not been empirically validated or do not cover the full content of the concept. Objectives: This study consists of a first empirical validation of a measurement instrument that…

Evaluating the implementation and impacts of middle grades inquiry-based engineering design modules

NASA Astrophysics Data System (ADS)

Harlan, Jessica M.

This dissertation uses a format where I present and discuss three articles that were written in conjunction with my work on a research and evaluation team. The articles are based on a multi-year project researching and evaluating the design, development, and implementation of a middle grades integrated Science, Technology, Engineering, and Mathematics (STEM) program. Each of the articles demonstrates the complexity in researching and evaluating curricular development in multifaceted, rapidly changing environments. While the focus of each article differs, they all examine research and evaluation in the context of the design and development of middle grades engineering-design modules. The selected articles address challenges associated with assessing program objectives and evaluating program quality in complex education programs.The first article, presented in Chapter 2, provides an overview of the nature of the EYE program and examines the extent to which participation in the EYE program resulted in the achievement of program objectives. There is evidence that EYE Module participation has a positive impact on participating students as well as teachers. This study also revealed challenges associated with determining the impact of program participation simultaneously with program revisions and assessment development. The second article, presented in Chapter 3, examined the evaluation of fidelity of implementation of inquiry-oriented educational programs. This article was intended to examine one way of triangulating information to determine fidelity while considering variation in implementation consistent with program theories of learning. When applying this model to implementation of the EYE program, we found many teachers were implementing the modules with low to moderate fidelity, especially math teachers. The third article, presented in Chapter 4, examined the factor structure of an occupational values scale intended to measure student interest in STEM careers. When reviewing the assessment's evidence base, we found no published literature regarding the assessment's psychometric properties. We found that the AWE Work Values scale assesses two separate sets of occupational values: 1) using analytical and problem solving skills and 2) personal satisfaction. In Chapter 5, I discuss the implications of the research presented in these articles to the field of integrated STEM. This includes a need to further develop and test tools and methods for measuring program impact. While the articles address several issues related to research and evaluation within the context of K-12 integrated STEM education, the concepts and implications also apply to the general field of instructional design. The primary lesson is the need to systematically evaluate designed instruction, including measuring fidelity of implementation and designing evaluations in advance to ensure appropriate data are gathered. Recommendations for future research include identification of common outcomes for integrated STEM, psychometric testing of integrated STEM attitude measures, impacts of other factors such as interest in STEM, and development and testing of instructional design models for integrated STEM.

Evaluation of audit-based performance measures for dental care plans.

PubMed

Bader, J D; Shugars, D A; White, B A; Rindal, D B

1999-01-01

Although a set of clinical performance measures, i.e., a report card for dental plans, has been designed for use with administrative data, most plans do not have administrative data systems containing the data needed to calculate the measures. Therefore, we evaluated the use of a set of proxy clinical performance measures calculated from data obtained through chart audits. Chart audits were conducted in seven dental programs--three public health clinics, two dental health maintenance organizations (DHMO), and two preferred provider organizations (PPO). In all instances audits were completed by clinical staff who had been trained using telephone consultation and a self-instructional audit manual. The performance measures were calculated for the seven programs, audit reliability was assessed in four programs, and for one program the audit-based proxy measures were compared to the measures calculated using administrative data. The audit-based measures were sensitive to known differences in program performance. The chart audit procedures yielded reasonably reliable data. However, missing data in patient charts rendered the calculation of some measures problematic--namely, caries and periodontal disease assessment and experience. Agreement between administrative and audit-based measures was good for most, but not all, measures in one program. The audit-based proxy measures represent a complex but feasible approach to the calculation of performance measures for those programs lacking robust administrative data systems. However, until charts contain more complete diagnostic information (i.e., periodontal charting and diagnostic codes or reason-for-treatment codes), accurate determination of these aspects of clinical performance will be difficult.

The Impact of the Measures of Academic Progress (MAP) Program on Student Reading Achievement

ERIC Educational Resources Information Center

Cordray, David S.; Pion, Georgine M.; Brandt, Chris; Molefe, Ayrin

2013-01-01

One of the most widely used commercially available systems incorporating benchmark assessment and training in differentiated instruction is the Northwest Evaluation Association's (NWEA) Measures of Academic Progress (MAP) program. The MAP program involves two components: (1) computer-adaptive assessments administered to students three to four…

Evaluation of an occupational rehabilitation program.

PubMed

Goodman, Glenn; Browning, Margaret; Campbell, Sims; Hudak, Huison

2005-01-01

The purpose of this study was to report the findings of a program evaluation for an occupational rehabilitation program in the Midwest. An ex-post facto chart review was performed on 50 charts using demographic data, results from tests of performance and pain measures, and data from patient satisfaction questionnaires to find evidence of excellence in the program, and to identify areas for improvement. Over 97% of the participants actually completed the program and 76% returned to work within 3 months after program completion. Ninety-two percent indicated overall satisfaction with the program, and all aspects of the program were rated with a mean score of 4.25 or above on a 1 to 5 Likert scale. Evaluation of perceived pain scales of the participants indicate no discernable relationship between intensity of pain and successful return to work. The program showed a high completion rate, a high return to work rate, and high levels of patient satisfaction. Suggestions for improvement include an increase in use of real and simulated work activities, better documentation of pain measures, better programs to address psychosocial issues, lengthening the program, and increased communication with case managers and professionals outside of the work program.

Modular biowaste monitoring system

NASA Technical Reports Server (NTRS)

Fogal, G. L.

1975-01-01

The objective of the Modular Biowaste Monitoring System Program was to generate and evaluate hardware for supporting shuttle life science experimental and diagnostic programs. An initial conceptual design effort established requirements and defined an overall modular system for the collection, measurement, sampling and storage of urine and feces biowastes. This conceptual design effort was followed by the design, fabrication and performance evaluation of a flight prototype model urine collection, volume measurement and sampling capability. No operational or performance deficiencies were uncovered as a result of the performance evaluation tests.

Pretesting Effects in the Evaluation of Social Skills Training.

ERIC Educational Resources Information Center

Mungas, Dan M.; Walters, Herman A.

1979-01-01

Evaluated effects of pretesting in conjunction with a group social skills training program. A Solomon four-groups design was used to evaluate the effects of pretesting, the skills training program, and their interaction. Strong evidence of pretesting effects was found for measures associated with a behavioral forced interaction task. (Author)

A Participatory Action Research Approach To Evaluating Inclusive School Programs.

ERIC Educational Resources Information Center

Dymond, Stacy K.

2001-01-01

This article proposes a model for evaluating inclusive schools. Key elements of the model are inclusion of stakeholders in the evaluation process through a participatory action research approach, analysis of program processes and outcomes, use of multiple methods and measures, and obtaining perceptions from diverse stakeholder groups. (Contains…

Outreach and Program Evaluation: Some Measurement Issues.

ERIC Educational Resources Information Center

Richard, Alan J.; And Others

1996-01-01

The experience of 10 years evaluating HIV risk-reduction intervention for drug users leads to the argument that program outreach workers are part of the intervention continuum and have important effects on service delivery. Improving pre-enrollment data collection will address evaluation issues raised by pre-enrollment outreach contacts. (SLD)

A pavement management research program for Oregon highways : final report.

DOT National Transportation Integrated Search

1989-12-01

An extensive program was developed to measure pavement deflection skid resistance, and rideability throughout Oregon. The data from those "objective" measures were then evaluated for correlations with observed pavement distress and traffic factors. :...

A Census of Prison-Based Drug Treatment Programs: Implications for Programming, Policy, and Evaluation

ERIC Educational Resources Information Center

Welsh, Wayne N.; Zajac, Gary

2004-01-01

Despite a growing realization that unmeasured programmatic differences influence prison-based drug treatment effectiveness, few attempts to systematically measure such differences have been made. To improve program planning and evaluation in this area, we developed a census instrument to collect descriptive information about 118 prison-based drug…

PAL[R] Services Being Measured through Scientifically-Based Evaluation Process

ERIC Educational Resources Information Center

Perspectives in Peer Programs, 2007

2007-01-01

In January 2006, PAL[R] Peer Assistance and Leadership, a Promising Prevention Program of Workers Assistance Program, Inc. (WAP), received a $30,000 grant from the Center for Substance Abuse Prevention (CSAP) in order to be scientifically-evaluated on the outcomes and effectiveness of its programs and services. According to the grant, the…

Evaluation of a Dual Bilingual Program: English-Spanish and English-Southeast Asian Languages.

ERIC Educational Resources Information Center

Wheeler, George; And Others

An external evaluation of the Title VII Bilingual Education Program was designed to: (1) gather a subjective assessment from personnel working with the program; (2) observe classroom teaching and activities utilized in bilingual instruction; and (3) measure differences in levels of achievement. The instructional personnel appear to accept the…

Community Engaged Service-Learning: The Effect on University Students and Community Partners

ERIC Educational Resources Information Center

Steele, Susan C.

2012-01-01

The notion that an ethic of service can be fostered through participation in an academic community engaged service-learning program was the underlying premise for this program evaluation study. The purpose of this program evaluation study, using both quantitative and qualitative measures in a thoughtfully organized setting of university and…

Process Evaluation for Improving K12 Program Effectiveness: Case Study of a National Institutes of Health Building Interdisciplinary Research Careers in Women's Health Research Career Development Program.

PubMed

Raymond, Nancy C; Wyman, Jean F; Dighe, Satlaj; Harwood, Eileen M; Hang, Mikow

2018-06-01

Process evaluation is an important tool in quality improvement efforts. This article illustrates how a systematic and continuous evaluation process can be used to improve the quality of faculty career development programs by using the University of Minnesota's Building Interdisciplinary Research Careers in Women's Health (BIRCWH) K12 program as an exemplar. Data from a rigorous process evaluation incorporating quantitative and qualitative measurements were analyzed and reviewed by the BIRCWH program leadership on a regular basis. Examples are provided of how this evaluation model and processes were used to improve many aspects of the program, thereby improving scholar, mentor, and advisory committee members' satisfaction and scholar outcomes. A rigorous evaluation plan can increase the effectiveness and impact of a research career development plan.

Measurement of Civil Engineering Customer Satisfaction in Tactical Air Command: A Prototype Evaluation Program.

DTIC Science & Technology

1986-09-01

customers . The article states that in response to a White House Office of Consumer Affairs study and with the wide use of minicomputers: Companies are...D-A174 l16 MEASUREMENT OF CIVIL ENGINEERING CUSTOMER SRTISFACTIbN 1/ IN TACTICAL AIR CO (U) AIR FORCE INST OF TECH ...... RIGHT-PATTERSON AFB ON...BUREAU OF STANDARDS- 1963-A_ . -_- ’II I-F MEASUREMENT OF CIVIL ENGINEERING CUSTOMER SATISFACTION IN TACTICAL AIR COMMAND: A PROTOTYPE EVALUATION PROGRAM

Indicators for Evaluating Community- and Societal-Level Risk and Protective Factors for Violence Prevention: Findings From a Review of the Literature.

PubMed

Armstead, Theresa L; Wilkins, Natalie; Doreson, Amanda

Programs geared toward preventing violence before it occurs at the community and societal levels of the social ecology are particularly challenging to evaluate. These programs are often focused on impacting the antecedents (or risk and protective factors) to violence, making it difficult to determine program success when solely relying on measures of violence reduction. The goal of this literature review is to identify indicators to measure risk and protective factors for violence that are accessible and measured at the community level. Indicators of community- and societal-level risk and protective factors from 116 articles are identified. These indicators strengthen violence prevention researchers' and practitioners' ability to detect proximal effects of violence prevention programs, practices, and policies, and provide timely feedback on the impact of their work. Thus, opportunities exist for violence prevention researchers to further study the associations between various indicators and different violent outcomes and to inform practitioner, evaluator, and funder developed logic models that include indicators of relevant risk and protective factors for crosscutting violence prevention measures and outcomes.

A framework for improving the cost-effectiveness of DSM program evaluations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sonnenblick, R.; Eto, J.

The prudence of utility demand-side management (DSM) investments hinges on their performance, yet evaluating performance is complicated because the energy saved by DSM programs can never be observed directly but only inferred. This study frames and begins to answer the following questions: (1) how well do current evaluation methods perform in improving confidence in the measurement of energy savings produced by DSM programs; (2) in view of this performance, how can limited evaluation resources be best allocated to maximize the value of the information they provide? The authors review three major classes of methods for estimating annual energy savings: trackingmore » database (sometimes called engineering estimates), end-use metering, and billing analysis and examine them in light of the uncertainties in current estimates of DSM program measure lifetimes. The authors assess the accuracy and precision of each method and construct trade-off curves to examine the costs of increases in accuracy or precision. Several approaches for improving evaluations for the purpose of assessing program cost effectiveness are demonstrated. The methods can be easily generalized to other evaluation objectives, such as shared savings incentive payments.« less

Test Review for Preschool-Wide Evaluation Tool (PreSET) Manual: Assessing Universal Program-Wide Positive Behavior Support in Early Childhood

ERIC Educational Resources Information Center

Rodriguez, Billie Jo

2013-01-01

The Preschool-Wide Evaluation Tool (PreSET; Steed & Pomerleau, 2012) is published by Paul H. Brookes Publishing Company in Baltimore, MD. The PreSET purports to measure universal and program-wide features of early childhood programs' implementation fidelity of program-wide positive behavior intervention and support (PW-PBIS) and is,…

Evaluation of the German-American Partnership Program between Madison Public Schools and Neutorschule of Mainz, Federal Republic of Germany.

ERIC Educational Resources Information Center

Grittner, Frank M.

The program evaluated here involved a two-way exchange of students and teachers from a school in Mainz and two high schools in Madison, Wisconsin. Homestay and cultural touring were outstanding aspects of the program. To measure the impact of the program on American participants, tests and questionnaires were administered before and after the…

Numeric data distribution: The vital role of data exchange in today's world

NASA Technical Reports Server (NTRS)

Chase, Malcolm W.

1994-01-01

The major aim of the NIST standard Reference Data Program (SRD) is to provide critically evaluated numeric data to the scientific and technical community in a convenient and accessible form. A second aim of the program is to provide feedback into the experimental and theoretical programs to help raise the general standards of measurement. By communicating the experience gained in evaluating the world output of data in the physical sciences, NIST/SRD helps to advance the level of experimental techniques and improve the reliability of physical measurements.

Using student satisfaction data to evaluate a new online accelerated nursing education program.

PubMed

Gazza, Elizabeth A; Matthias, April

2016-10-01

As increasing numbers of students enroll in online education, institutions of higher education are responsible for delivering quality online courses and programs. Agencies that accredit institutions and programs require evidence of program quality, including student satisfaction. A large state university in the Southeastern United States transitioned an online nursing education degree completion, or Registered Nurse-to-Bachelor of Science in Nursing, program to an online accelerated format in order to meet the needs of working nurses and ultimately, increase the number of nurses prepared at the baccalaureate level. This article describes a descriptive, cross-sectional study that evaluated the effectiveness of the new online accelerated program using the quality indicator of student satisfaction. Ninety-one (32%) of the 284 students who were enrolled or had been enrolled in a course within the online accelerated degree completion program between fall 2013 session 1 and summer 2014 session participated in the study. The electronic Noel-Levitz Priorities Survey for Online Learners™ was used to measure student satisfaction with the program and associated services. Results provided insight into the students' satisfaction with the new program format and served as the basis for an interdepartmental program enhancement plan aimed at maintaining and enhancing student satisfaction and overall program quality. Findings indicated that measuring and evaluating student satisfaction can provide valuable information about the effectiveness of an online program. Recommendations for using the measurement tool in online program planning and studying student satisfaction in relation to retention and program completion were identified. Copyright © 2016 Elsevier Ltd. All rights reserved.

A systematic review of evaluated suicide prevention programs targeting indigenous youth.

PubMed

Harlow, Alyssa F; Bohanna, India; Clough, Alan

2014-01-01

Indigenous young people have significantly higher suicide rates than their non-indigenous counterparts. There is a need for culturally appropriate and effective suicide prevention programs for this demographic. This review assesses suicide prevention programs that have been evaluated for indigenous youth in Australia, Canada, New Zealand, and the United States. The databases MEDLINE and PsycINFO were searched for publications on suicide prevention programs targeting indigenous youth that include reports on evaluations and outcomes. Program content, indigenous involvement, evaluation design, program implementation, and outcomes were assessed for each article. The search yielded 229 articles; 90 abstracts were assessed, and 11 articles describing nine programs were reviewed. Two Australian programs and seven American programs were included. Programs were culturally tailored, flexible, and incorporated multiple-levels of prevention. No randomized controlled trials were found, and many programs employed ad hoc evaluations, poor program description, and no process evaluation. Despite culturally appropriate content, the results of the review indicate that more controlled study designs using planned evaluations and valid outcome measures are needed in research on indigenous youth suicide prevention. Such changes may positively influence the future of research on indigenous youth suicide prevention as the outcomes and efficacy will be more reliable.

The Development of NOAA Education Common Outcome Performance Measures (Invited)

NASA Astrophysics Data System (ADS)

Baek, J.

2013-12-01

The National Oceanic and Atmospheric Administration (NOAA) Education Council has embarked on an ambitious Monitoring and Evaluation (M&E) project that will allow it to assess education program outcomes and impacts across the agency, line offices, and programs. The purpose of this internal effort is to link outcome measures to program efforts and to evaluate the success of the agency's education programs in meeting the strategic goals. Using an outcome-based evaluation approach, the NOAA Education Council is developing two sets of common outcome performance measures, environmental stewardship and professional development. This presentation will examine the benefits and tradeoffs of common outcome performance measures that collect program results across a portfolio of education programs focused on common outcomes. Common outcome performance measures have a few benefits to our agency and to the climate education field at large. The primary benefit is shared understanding, which comes from our process for writing common outcome performance measures. Without a shared and agreed upon set of definitions for the measure of an outcome, the reported results may not be measuring the same things and would incorrectly indicate levels of performance. Therefore, our writing process relies on a commitment to developing a shared set of definitions based on consensus. We hope that by taking the time to debate and coming to agreement across a diverse set of programs, the strength of our common measures can indicate real progress towards outcomes we care about. An additional benefit is that these common measures can be adopted and adapted by other agencies and organizations that share similar theories of change. The measures are not without their drawbacks, and we do make tradeoffs as part of our process in order to continue making progress. We know that any measure is necessarily a narrow slice of performance. A slice that may not best represent the unique and remarkable contribution of an individual program, but does reflect a variety of contributions along a single dimension across a large portfolio of programs. The process has ended up pushing our working group to call for even more measures, to capture an increasing number of dimensions that reflect the nature of the portfolio of programs. This past year we have been working on developing two sets of common outcome performance measures for professional development (PD) and stewardship education programs. The outcome we chose for PD programs was the use of what was learned in the educator's practice. The outcome we chose for stewardship programs was the stewardship behaviors that participants learn and practice. The measurement of these outcomes will inform whether our strategies are having their intended impact. By knowing how and how much these outcomes are occurring as a result of our program, we can improve program performance over time. The common outcome performance measures help demonstrate how these programs engage audiences in supporting NOAA's mission. As AGU climate literacy community continues to grow, it is important to consider an approach to demonstrate the community's contribution to the Nation's climate literacy. Development of common outcome performance measures is one approach that could help focus the community in meeting its goals.

How Bigelow Laboratory Measured Broader Impacts: The Case Study of the Evaluation of the Keller BLOOM Program (Invited)

NASA Astrophysics Data System (ADS)

Fowler, R. A.; Repa, J.

2010-12-01

In this presentation we discuss the impetus for, the results of the short and long term effects, and the impacts of the Keller BLOOM Program, hosted by Bigelow Laboratory for Ocean Sciences of West Boothbay Harbor, Maine. Each May, for the last 21 years, 16 bright high school juniors, one from each county in Maine, have been invited to spend five days with the research scientists at the lab conducting and reporting research on the bottom layers of the ocean’s food chain: phytoplankton and zooplankton. Bigelow has chosen to evaluate BLOOM through a series of questionnaires delivered during the program, and long term tracking of participants after the program, in order to better understand the impact of the program on participants. The short term effect of the experience, measured at the end of the week, found that participants are able to: 1) develop testable research questions, 2) collect multiple water samples from a local estuary, 3) measure various characteristics of those samples with the sophisticated instruments in Bigelow’s labs assisted by their research scientists, 4) analyze and integrate the results from the various labs, and 5) present their findings to a non-scientific audience. To measure long term participation effects, a random sample of 40 of the 332 participants were interviewed resulting in the following findings: 100% attend college; 62% have STEM majors; 88% graduate from college; 57% pursue STEM careers; and 60% live and work in Maine. Bigelow scientists include a description of the BLOOM Program and the evaluation results in their NSF Broader Impacts statements to demonstrate that their research activities are being integrated into a successful STEM education program. Evaluation results are also used by Bigelow scientists and program administrators to refine program content and delivery, to promote the program to potential applicants, and to strengthen proposals to funding agencies when seeking financial support for BLOOM.

Community outreach: from measuring the difference to making a difference with health information*

PubMed Central

Ottoson, Judith M.; Green, Lawrence W.

2005-01-01

Background: Community-based outreach seeks to move libraries beyond their traditional institutional boundaries to improve both access to and effectiveness of health information. The evaluation of such outreach needs to involve the community in assessing the program's process and outcomes. Purpose: Evaluation of community-based library outreach programs benefits from a participatory approach. To explain this premise of the paper, three components of evaluation theory are paired with relevant participatory strategies. Concepts: The first component of evaluation theory is also a standard of program evaluation: use. Evaluation is intended to be useful for stakeholders to make decisions. A useful evaluation is credible, timely, and of adequate scope. Participatory approaches to increase use of evaluation findings include engaging end users early in planning the program itself and in deciding on the outcomes of the evaluation. A second component of evaluation theory seeks to understand what is being evaluated, such as specific aspects of outreach programs. A transparent understanding of the ways outreach achieves intended goals, its activities and linkages, and the context in which it operates precedes any attempt to measure it. Participatory approaches to evaluating outreach include having end users, such as health practitioners in other community-based organizations, identify what components of the outreach program are most important to their work. A third component of evaluation theory is concerned with the process by which value is placed on outreach. What will count as outreach success or failure? Who decides? Participatory approaches to valuing include assuring end-user representation in the formulation of evaluation questions and in the interpretation of evaluation results. Conclusions: The evaluation of community-based outreach is a complex process that is not made easier by a participatory approach. Nevertheless, a participatory approach is more likely to make the evaluation findings useful, ensure that program knowledge is shared, and make outreach valuing transparent. PMID:16239958

Tailoring an educational program on the AHRQ Patient Safety Indicators to meet stakeholder needs: lessons learned in the VA.

PubMed

Shin, Marlena H; Rivard, Peter E; Shwartz, Michael; Borzecki, Ann; Yaksic, Enzo; Stolzmann, Kelly; Zubkoff, Lisa; Rosen, Amy K

2018-02-14

Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders' needs. In this paper, we share what we learned from this program development process. Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders' educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders' needs/interests. A post-program survey evaluated program participants' perceptions about the PSI educational program. Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed-three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive. Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders' needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.

Driving social impact with common global indicators for healthy lifestyle programs: lessons learned.

PubMed

Robinson, Nicole R; Gin, Julia; Kamath-Jha, Shilpa; Infantes, Michel; Hernandez, Ricardo; Alberg-Seberich, Michael; Suri, Devika; Pérez-Escamilla, Rafael

2014-09-01

Partnerships between corporate entities and non-governmental organizations (NGOs) involved in delivering community focused health and well-being programs are becoming increasingly valuable especially in the context of promoting healthy lifestyles around the globe. The Mondelēz International Foundation (MIF) has funded healthy lifestyles community based programs targeting children and youth through partnership with seven global NGOs. To assess collective impact of these programs, it is crucial to identify best practices and common impact indicators that can be measured across programs. MIF therefore organized the Healthy Lifestyles Evaluation Workshop to explore these pertinent questions. Share best practices and identify common impact indicators to measure the success of current and future MIF funded healthy lifestyles programs. Analysis of the Program Impact Pathways (PIPs) and measured output of each of the seven programs. Individual and combined analysis of PIPs of the seven NGO programs led to identification of three critical impact indicators: nutrition knowledge, physical activity, and healthier eating, and also enabled NGOs to identify pathways to improve program delivery among the target population. This workshop enabled MIF and partner NGOs to came together to align on metrics and future engagement approaches for promoting and evaluating community based healthy lifestyles programs.

78 FR 37741 - Approval and Promulgation of Implementation Plans; California; South Coast; Contingency Measures...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-06-24

.... Id. \\7\\ Consistent with EPA's definition of ``design value'' in 40 CFR 58.1, we use the term ``design... evaluations below. Carl Moyer Memorial Air Quality Standards Attainment Program The Contingency Measures SIP identifies a portion of the Carl Moyer Memorial Air Quality Standards Attainment Program (Carl Moyer Program...

Evaluating Training Programs for Primary Care Providers in Child/Adolescent Mental Health in Canada: A Systematic Review.

PubMed

Gotovac, Sandra; Espinet, Stacey; Naqvi, Reza; Lingard, Lorelei; Steele, Margaret

2018-04-01

The need for child/adolescent mental health care in Canada is growing. Primary care can play a key role in filling this gap, yet most providers feel they do not have adequate training. This paper reviews the Canadian literature on capacity building programs in child and adolescent psychiatry for primary care providers, to examine how these programs are being implemented and evaluated to contribute to evidence-based initiatives. A systematic literature review of peer-reviewed published articles of capacity building initiatives in child/adolescent mental health care for primary care practitioners that have been implemented in Canada. Sixteen articles were identified that met inclusion criteria. Analysis revealed that capacity building initiatives in Canada are varied but rigorous evaluation methodology is lacking. Primary care providers welcome efforts to increase mental health care capacity and were satisfied with the implementation of most programs. Objective conclusions regarding the effectiveness of these programs to increase mental health care capacity is challenging given the evaluation methodology of these studies. Rigorous evaluation methods are needed to make evidence-based decisions on ways forward to be able to build child/adolescent mental health care capacity in primary care. Outcome measures need to move beyond self-report to more objective measures, and should expand the measurement of patient outcomes to ensure that these initiative are indeed leading to improved care for families.

A Multi-Stage Longitudinal Comparative Design Stage II Evaluation of the Changing Lives Program: The Life Course Interview (RDA-LCI)

ERIC Educational Resources Information Center

Arango, Lisa Lewis; Kurtines, William M.; Montgomery, Marilyn J.; Ritchie, Rachel

2008-01-01

The study reported in this article, a Multi-Stage Longitudinal Comparative Design Stage II evaluation conducted as a planned preliminary efficacy evaluation (psychometric evaluation of measures, short-term controlled outcome studies, etc.) of the Changing Lives Program (CLP), provided evidence for the reliability and validity of the qualitative…

Students Training for Academic Readiness (STAR): Year Three Evaluation Report. Executive Summary

ERIC Educational Resources Information Center

Texas Center for Educational Research, 2010

2010-01-01

This executive summary presents findings from the Year 3 evaluation of Texas' state-level Gaining Early Awareness and Readiness for Undergraduate Programs, or GEAR UP, grant. GEAR UP grant requirements include an evaluation component designed to assess program effectiveness and to measure progress toward project goals. To this end, the evaluation…

Silver Memories: implementation and evaluation of a unique radio program for older people.

PubMed

Travers, Catherine; Bartlett, Helen P

2011-03-01

A unique radio program, Silver Memories, specifically designed to address social isolation and loneliness in older people by broadcasting music (primarily), serials and other programs relevant to the period when older people grew up--the 1920-1950s--first aired in Brisbane, Australia, in April 2008. The impact of the program upon older listeners' mood, quality of life (QOL) and self-reported loneliness was independently evaluated. One hundred and thirteen community-dwelling persons and residents of residential care facilities, aged 60 years and older participated in a three month evaluation of Silver Memories. They were asked to listen to the program daily and baseline and follow-up measures of depression, QOL and loneliness were obtained. Participants were also asked for their opinions regarding the program's quality and appeal. The results showed a statistically significant improvement in measures of depression and QOL from baseline to follow-up but there was no change on the measure of loneliness. The results did not vary by living situation (community vs. residential care), whether the participant was lonely or not lonely, socially isolated or not isolated, or whether there had been any important changes in the participant's health or social circumstances throughout the evaluation. It was concluded that listening to Silver Memories appears to improve the QOL and mood of older people and is an inexpensive intervention that is flexible and readily implemented.

Measuring Program Quality, Part 2: Addressing Potential Cultural Bias in a Rater Reliability Exam

ERIC Educational Resources Information Center

Richer, Amanda; Charmaraman, Linda; Ceder, Ineke

2018-01-01

Like instruments used in afterschool programs to assess children's social and emotional growth or to evaluate staff members' performance, instruments used to evaluate program quality should be free from bias. Practitioners and researchers alike want to know that assessment instruments, whatever their type or intent, treat all people fairly and do…

Evaluating the Efficacy of Children Participating in Home Instruction for Parents of Preschool Youngsters and Head Start

ERIC Educational Resources Information Center

Brown, Amber L; Lee, Joohi

2017-01-01

The purpose of this study was to evaluate the efficacy of the Home Instruction for Parents of Preschool Youngsters program when implemented within Head Start programs by measuring children's language proficiency scores. Participants were kindergarteners concurrently enrolled in both a Head Start program and the Home Instruction for Parents of…

Evaluation of College Reading and Study Skills Programs; Problems and Approaches.

ERIC Educational Resources Information Center

Anderson, William W.

The purpose of this paper was to bring together some of the most outstanding and significant research in recent years relating to the problems of evaluating college reading and study skills programs and the corresponding prospects and implications for improvement. The sections of the paper include: "Criterion Measures of Program Success," which is…

Evaluative Thinking in Practice: The National Asthma Control Program.

PubMed

Fierro, Leslie A; Codd, Heather; Gill, Sarah; Pham, Phung K; Grandjean Targos, Piper T; Wilce, Maureen

2018-01-01

Although evaluative thinking lies at the heart of what we do as evaluators and what we hope to promote in others through our efforts to build evaluation capacity, researchers have given limited attention to measuring this concept. We undertook a research study to better understand how instances of evaluative thinking may present in practice-based settings-specifically within four state asthma control programs funded by the Centers for Disease Control and Prevention's National Asthma Control Program. Through content analyses of documents as well as interviews and a subsequent focus group with four state asthma control programs' evaluators and program managers we identified and defined twenty-two indicators of evaluative thinking. Findings provide insights about what practitioners may wish to look for when they intend to build evaluative thinking and the types of data sources that may be more or less helpful in such efforts.

The TMDL Program Results Analysis Project: Matching Results Measures with Program Expectations

EPA Pesticide Factsheets

The paper provides a detailed description of the aims, methods and outputs of the program evaluation project undertaken by EPA in order to generate the insights needed to make TMDL program improvements.

40 CFR 63.7191 - What records must I keep?

Code of Federal Regulations, 2010 CFR

2010-07-01

... malfunctions. (3) Records of performance tests and performance evaluations as required in § 63.10(b)(2)(viii... measurements, raw performance evaluation measurements). (3) All required CMS measurements (including monitoring... Section 63.7191 Protection of Environment ENVIRONMENTAL PROTECTION AGENCY (CONTINUED) AIR PROGRAMS...

Position Measurement Standard Evaluation

DOT National Transportation Integrated Search

1975-02-01

The objectives of the Position Measurement Standard Program were to collect navigation data from three DME receivers and a low-frequency GLOBAL Navigation system, and evaluate their relative performance against a reference radar. Flight test data dur...

Developing measures of community-relevant outcomes for violence prevention programs: a community-based participatory research approach to measurement.

PubMed

Hausman, Alice J; Baker, Courtney N; Komaroff, Eugene; Thomas, Nicole; Guerra, Terry; Hohl, Bernadette C; Leff, Stephen S

2013-12-01

Community-Based Participatory Research is a research paradigm that encourages community participation in designing and implementing evaluation research, though the actual outcome measures usually reflect the "external" academic researchers' view of program effect and the policy-makers' needs for decision-making. This paper describes a replicable process by which existing standardized psychometric scales commonly used in youth-related intervention programs were modified to measure indicators of program success defined by community partners. This study utilizes a secondary analysis of data gathered in the context of a community-based youth violence prevention program. Data were retooled into new measures developed using items from the Alabama Parenting Questionnaire, the Hare Area Specific Self-Esteem Scale, and the Youth Asset Survey. These measures evaluated two community-defined outcome indicators, "More Parental Involvement" and "Showing Kids Love." Results showed that existing scale items can be re-organized to create measures of community-defined outcomes that are psychometrically reliable and valid. Results also show that the community definitions of parent or parenting caregivers exemplified by the two indicators are similar to how these constructs have been defined in previous research, but they are not synonymous. There are nuanced differences that are important and worthy of better understanding, in part through better measurement.

Methods for evaluating a mature substance abuse prevention/early intervention program.

PubMed

Becker, L R; Hall, M; Fisher, D A; Miller, T R

2000-05-01

The authors describe methods for work in progress to evaluate four workplace prevention and/or early intervention programs designed to change occupational norms and reduce substance abuse at a major U.S. transportation company. The four programs are an employee assistance program, random drug testing, managed behavioral health care, and a peer-led intervention program. An elaborate mixed-methods evaluation combines data collection and analysis techniques from several traditions. A process-improvement evaluation focuses on the peer-led component to describe its evolution, document the implementation process for those interested in replicating it, and provide information for program improvement. An outcome-assessment evaluation examines impacts of the four programs on job performance measures (e.g., absenteeism, turnover, injury, and disability rates) and includes a cost-offset and employer cost-savings analysis. Issues related to using archival data, combining qualitative and quantitative designs, and working in a corporate environment are discussed.

Evaluation of a national quality use of medicines service in Australia: an evolving model.

PubMed

Beilby, Justin; Wutzke, Sonia E; Bowman, Jenny; Mackson, Judith M; Weekes, Lynn M

2006-04-01

To describe the first phase of a global evaluation framework for the National Prescribing Service (NPS), with a focus on services for health professionals, and in particular highlight the lessons learnt from evaluation around the establishment and implementation of this national program. The agreed evaluation framework used mixed methods focused around a series of evaluation questions, aimed at measuring the overall effect of this new organization as well as the individual programs within it. The evaluation questions were determined a priori and were based on the objectives established by the organization in its first year of operation. A detailed analyses has been completed of: the process, scope and reach of program delivery using both quantitative and qualitative measures; changes in attitudes and knowledge measured through key informant interviews and surveys of professional groups and consumers; and changes in prescribing behaviour and savings to the Pharmaceutical Benefits Scheme (PBS) through analysis of prescription data. The evaluation period for this report was mid-1998 to mid-2004. The NPS has successfully implemented a complex, multi-faceted program across Australia. From 1998 to 2004, in addition to print material provided to all general practitioners (GPs) and pharmacists, 90% of all GPs have been actively involved in one or more educational activity; 116 of 120 divisions of general practice have coordinated local NPS programs; and 9% of pharmacists have actively participated in at least one educational activity. Sixty per cent of GPs and pharmacists rated the printed educational materials as good or very good. In the last three years, the NPS activities have generated savings in the range of $121-163 million to the PBS, owing to changes in prescribing practices. The national evaluation framework has informed program delivery and ongoing design and development. Continued refinement of existing evaluation methods and further exploration of new techniques will remain a priority for the organization.

Thinking Inside the Box: The Health Cube Paradigm for Health and Wellness Program Evaluation and Design

PubMed Central

Harris, Sharon

2013-01-01

Abstract Appropriately constructed health promotions can improve population health. The authors developed a practical model for designing, evaluating, and improving initiatives to provide optimal value. Three independent model dimensions (impact, engagement, and sustainability) and the resultant three-dimensional paradigm were described using hypothetical case studies, including a walking challenge, a health risk assessment survey, and an individual condition management program. The 3-dimensional model is illustrated and the dimensions are defined. Calculation of a 3-dimensional score for program comparisons, refinements, and measurement is explained. Program 1, the walking challenge, had high engagement and impact, but limited sustainability. Program 2, the health risk assessment survey, had high engagement and sustainability but limited impact. Program 3, the on-site condition management program, had measurable impact and sustainability but limited engagement, because of a lack of program capacity. Each initiative, though successful in 2 dimensions, lacked sufficient evolution along the third axis for optimal value. Calculation of a 3-dimensional score is useful for health promotion program development comparison and refinements, and overall measurement of program success. (Population Health Management 2013;16:291–295) PMID:23869538

EMI Measurement and Mitigation Testing for the ARPA Hybrid Electric Vehicle Program

DTIC Science & Technology

1996-08-27

communication range is reduced, computers malfunction, or monitoring systems fail. Various electric vehicles ( EVs ) were measured to evaluate their...electric vehicles ( EVs ) were measured to evaluate their potential EMI emissions when used in today’s hostile commercial electromagnetic environment...monitoring systems fail. Various electric vehicles ( EVs ) were measured to evaluate their potential EMI emissions when used in today’s hostile commercial

National 4-H Common Measures: Initial Evaluation from California 4-H

ERIC Educational Resources Information Center

Lewis, Kendra M.; Horrillo, Shannon J.; Widaman, Keith; Worker, Steven M.; Trzesniewski, Kali

2015-01-01

Evaluation is a key component to learning about the effectiveness of a program. This article provides descriptive statistics of the newly developed National 4-H Common Measures (science, healthy living, citizenship, and youth development) based on data from 721 California 4-H youth. The measures were evaluated for their reliability and validity of…

76 FR 18615 - 30-Day Notice of Proposed Information Collection: Performance Measurement, Evaluation and Public...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-04-04

...: Performance Measurement, Evaluation and Public Diplomacy Program Surveys. OMB Control Number: 1405-0158. Type... and Cultural Affairs, Office of Policy and Evaluation, Evaluation Division (ECA/P/ V). Form Number: SV... State. [FR Doc. 2011-7931 Filed 4-1-11; 8:45 am] BILLING CODE 4710-05-P ...

Evaluating a county-based Healthy nail Salon Recognition Program

EPA Science Inventory

To determine whether nail solons that participate in the SF recognition program have reduced measured levels of toluene, methyl methacrylate (MMA), and total volatile organic compounds (TVOC)as compared to nail salons that do not participate. We also evaluated changes in worker ...

Enhancing the Possibility of Success by Measuring the Probability of Failure in an Educational Program.

ERIC Educational Resources Information Center

Brookhart, Susan M.; And Others

1997-01-01

Process Analysis is described as a method for identifying and measuring the probability of events that could cause the failure of a program, resulting in a cause-and-effect tree structure of events. The method is illustrated through the evaluation of a pilot instructional program at an elementary school. (SLD)

Statistical evaluation of blood alcohol measurements

DOT National Transportation Integrated Search

1981-10-01

The U.S. Department of Transportation, National Highway Traffic Safety Administration (NHTSA) has instituted a voluntary program to evaluate the proficiency of laboratories measuring the amount of alcohol in blood. In this report, data from that prog...

Measuring the Impact of Programs that Challenge the Public Stigma of Mental Illness

PubMed Central

Corrigan, Patrick W.; Shapiro, Jenessa R.

2010-01-01

Public stigma robs people with mental illnesses from rightful opportunities related to work and other important life goals. Advocates have developed anti-stigma programs meant to address the prejudice and discrimination associated with these conditions. Evidence is now needed to make sense of program impact; this paper looks at measurement issues related to stigma change. Community based participatory research is central to this research and includes the involvement of a diverse collection of stakeholders in all phases of evaluation. Investigators should be cautious about measures vis-à-vis social desirability effects and should directed by social validity of targeted audiences. Conceptual domains with some research support that correspond with assessments include behavior, penetration, psychological perspective, knowledge, and physiological/information processes. These issues are summarized as ten recommendations for evaluation of anti-stigma programs. PMID:20674114

Designing a Measurement Framework for Response to Intervention in Early Childhood Programs

ERIC Educational Resources Information Center

McConnell, Scott R.; Wackerle-Hollman, Alisha K.; Roloff, Tracy A.; Rodriguez, Michael

2014-01-01

The overall architecture and major components of a measurement system designed and evaluated to support Response to Intervention (RTI) in the areas of language and literacy in early childhood programs are described. Efficient and reliable measurement is essential for implementing any viable RTI system, and implementing such a system in early…

Using LISREL to Evaluate Measurement Models and Scale Reliability.

ERIC Educational Resources Information Center

Fleishman, John; Benson, Jeri

1987-01-01

LISREL program was used to examine measurement model assumptions and to assess reliability of Coopersmith Self-Esteem Inventory for Children, Form B. Data on 722 third-sixth graders from over 70 schools in large urban school district were used. LISREL program assessed (1) nature of basic measurement model for scale, (2) scale invariance across…

Biomedical and Human Factors Requirements for a Manned Earth Orbiting Station

NASA Technical Reports Server (NTRS)

Helvey, W.; Martell, C.; Peters, J.; Rosenthal, G.; Benjamin, F.; Albright, G.

1964-01-01

The primary objective of this study is to determine which biomedical and human factors measurements must be made aboard a space station to assure adequate evaluation of the astronaut's health and performance during prolonged space flights. The study has employed, where possible, a medical and engineering systems analysis to define the pertinent life sciences and space station design parameters and their influence on a measurement program. The major areas requiring evaluation in meeting the study objectives include a definition of the space environment, man's response to the environment, selection of measurement and data management techniques, experimental program, space station design requirements, and a trade-off analysis with final recommendations. The space environment factors that are believed to have a significant effect on man were evaluated. This includes those factors characteristic of the space environment (e. g. weightlessness, radiation) as well as those created within the space station (e. g. toxic contaminants, capsule atmosphere). After establishing the general features of the environment, an appraisal was made of the anticipated response of the astronaut to each of these factors. For thoroughness, the major organ systems and functions of the body were delineated, and a determination was made of their anticipated response to each of the environmental categories. A judgment was then made on the medical significance or importance of each response, which enabled a determination of which physiological and psychological effects should be monitored. Concurrently, an extensive list of measurement techniques and methods of data management was evaluated for applicability to the space station program. The various space station configurations and design parameters were defined in terms of the biomedical and human factors requirements to provide the measurements program. Research design of experimental programs for various station configurations, mission durations, and crew sizes were prepared, and, finally, a trade-off analysis of the critical variables in the station planning was completed with recommendations to enhance the confidence in the measurement program.

Addressing Social Injustices, Displacement, and Minority Rights through Cases of Culturally Responsive Evaluation

ERIC Educational Resources Information Center

Stokes, Helga; Chaplin, Shane S.; Dessouky, Shimaa; Aklilu, Liya; Hopson, Rodney K.

2011-01-01

Evaluation of programs that address the lingering effects of human rights abuses during times of conflict is necessary to improve program sustainability and create a knowledge bank about the effectiveness of strategies. Outcomes, however, are hard to measure. Evaluators have to gain understanding of the roots of a conflict, surrounding events,…

Evaluation of Arabic Language Learning Program for Non-Native Speakers in Saudi Electronic University According to Total Quality Standards

ERIC Educational Resources Information Center

Alowaydhi, Wafa Hafez

2016-01-01

The current study aimed at standardizing the program of learning Arabic for non-native speakers in Saudi Electronic University according to certain standards of total quality. To achieve its purpose, the study adopted the descriptive analytical method. The author prepared a measurement tool for evaluating the electronic learning programs in light…

An Approach to Evaluation of Rural Development Act Title V Programs. Northeast Regional Center for Rural Development Publication 11, February 1976.

ERIC Educational Resources Information Center

Miller, Robert W.

Presenting suggestions for conceptualizing Rural Development Act Title V program evaluations, this paper focuses upon a five-level program outcomes model which encompasses a causal sequence of anticipated shortrun, intermediate, and longrun outcomes. Examples of possible Title V measurement foci are presented for each of five outcome levels as…

A Model-Based Evaluation of a Cultural Mediator Outreach Program for HIV+ Ethiopian Immigrants in Israel.

ERIC Educational Resources Information Center

Kaplan, Edward H.; Soskolne, Varda; Adler, Bella; Leventhal, Alex; Shtarkshall, Ronny A.

2002-01-01

Conducted a model-based evaluation of a program designed to reduce HIV transmission from HIV-infected Ethiopian immigrants in Israel. Focused on pregnancy rate reduction as a measure of sexual exposure. Results for 145 female and 176 male clients in the intervention suggest reduction in unprotected sexual exposures among program participants. (SLD)

Online evaluation programs: benefits and limitations.

PubMed

Burhansstipanov, Linda; Clark, Richard E; Watanabe-Galloway, Shinobu; Petereit, Daniel G; Eschiti, Valerie; Krebs, Linda U; Pingatore, Noel L

2012-04-01

Patient navigation programs are increasing throughout the USA, yet some evaluation measures are too vague to determine what and how navigation functions. Through collaborative efforts an online evaluation program was developed. The goal of this evaluation program is to make data entry accurate, simple, and efficient. This comprehensive program includes major components on staff, mentoring, committees, partnerships, grants/studies, products, dissemination, patient navigation, and reports. Pull down menus, radio buttons, and check boxes are incorporated whenever possible. Although the program has limitations, the benefits of having access to current, up-to-date program data 24/7 are worth overcoming the challenges. Of major benefit is the ability of the staff to tailor summary reports to provide anonymous feedback in a timely manner to community partners and participants. The tailored data are useful for the partners to generate summaries for inclusion in new grant applications.

Measures of Student Non-Cognitive Skills and Political Tolerance after Two Years of the Louisiana Scholarship Program. Louisiana Scholarship Program Evaluation Report #2. Technical Report

ERIC Educational Resources Information Center

Mills, Jonathan N.; Cheng, Albert; Hitt, Collin E.; Wolf, Patrick J.; Greene, Jay P.

2016-01-01

This report examines the short-term effects of the Louisiana Scholarship Program (LSP) on students' non-cognitive skills and civic values. While a growing number of studies have evaluated K-12 school voucher programs along academic dimensions, few have focused on the development of non-cognitive skills and civic values. This study aims to address…

Evaluation of the Environmental Scoring System in Multiple Child Asthma Intervention Programs in Boston, Massachusetts.

PubMed

Dong, Zhao; Nath, Anjali; Guo, Jing; Bhaumik, Urmi; Chin, May Y; Dong, Sherry; Marshall, Erica; Murphy, Johnna S; Sandel, Megan T; Sommer, Susan J; Ursprung, W W Sanouri; Woods, Elizabeth R; Reid, Margaret; Adamkiewicz, Gary

2018-01-01

To test the applicability of the Environmental Scoring System, a quick and simple approach for quantitatively measuring environmental triggers collected during home visits, and to evaluate its contribution to improving asthma outcomes among various child asthma programs. We pooled and analyzed data from multiple child asthma programs in the Greater Boston Area, Massachusetts, collected in 2011 to 2016, to examine the association of environmental scores (ES) with measures of asthma outcomes and compare the results across programs. Our analysis showed that demographics were important contributors to variability in asthma outcomes and total ES, and largely explained the differences among programs at baseline. Among all programs in general, we found that asthma outcomes were significantly improved and total ES significantly reduced over visits, with the total Asthma Control Test score negatively associated with total ES. Our study demonstrated that the Environmental Scoring System is a useful tool for measuring home asthma triggers and can be applied regardless of program and survey designs, and that demographics of the target population may influence the improvement in asthma outcomes.

Partnership, knowledge translation, and substance abuse prevention with a First Nations community.

PubMed

Baydala, Lola; Fletcher, Fay; Worrell, Stephanie; Kajner, Tania; Letendre, Sherry; Letendre, Liz; Rasmussen, Carmen

2014-01-01

Having identified substance abuse as an issue of concern in their community, the Alexis Nakota Sioux Nation invited University of Alberta researchers to partner on the cultural adaptation, delivery, and evaluation of a school-based drug and alcohol abuse prevention program. Researchers conducted a literature review of available drug and alcohol prevention programs for children and youth, identifying the Life Skills Training (LST) program as a viable model for cultural adaptation. Four program objectives were developed: (1) Review and cultural adaptation of the elementary and junior high LST programs, (2) delivery of the adapted programs, (3) measurement of changes in students' knowledge of the negative effects of drug and alcohol use, attitudes toward drugs and alcohol, drug and alcohol refusal and life skills, and changes in self-esteem/self-concept, and (4) documentation of the community's experience of the project. Using the principles of community-based participatory research (CBPR), we employed both qualitative and quantitative methods to evaluate the impact of the project. Qualitative evaluation of the program adaptation and implementation were both positive. Qualitative measures of program impact on students revealed a positive effect, whereas results of the quantitative measures were mixed. Culturally adapted, evidence-based programs can have a positive effect on Aboriginal youth and their communities. Strategies to expand knowledge translation (KT) when working with Aboriginal communities include working to create an "ethical space" that draws on the strengths of both Western and Indigenous worldviews.

In situ ozone data for evaluation of the laser absorption spectrometer ozone remote sensor: 1979 southeastern Virginia urban plume study summer field program

NASA Technical Reports Server (NTRS)

Gregory, G. L.; Mcdougal, D. S.; Mathis, J. J., Jr.

1980-01-01

Ozone data from the 1979 Southeastern Virginia Urban Study (SEV-UPS) field program are presented. The SEV-UPS was conducted for evaluation of an ozone remote sensor, the Laser Absorption Spectrometer. During the measurement program, remote-sensor evaluation was in two areas; (1) determination of the remote sensor's accuracy, repeatability, and operational characteristics, and (2) demonstration of the application of remotely sensed ozone data in air-quality studies. Data from six experiments designed to provide in situ ozone data for evaluation of the sensor in area 1, above, are presented. Experiments consisted of overflights of a test area with the remote sensor aircraft while in situ measurements with a second aircraft and selected surface stations provided correlative ozone data within the viewing area of the remote sensor.

Measuring Outcomes in Children's Services.

ERIC Educational Resources Information Center

Christner, Anne Marshall, Ed.

Outcomes evaluation can provide program managers and clinical directors in child welfare, juvenile justice, child mental health, and child protective services the necessary tools for program quality assurance and accountability. This guide describes the outcomes evaluation process and provides a summary of articles and reports detailing current…

Crossroads Cafe Implementation Florida Evaluation.

ERIC Educational Resources Information Center

McLean, Teri

The evaluation reviews the implementation of the "Crossroads Cafe" English language instruction program in Florida, focusing on the program's management, training, and overall effectiveness as measured by its impact on adult English-as-a-Second-Language (ESL) teachers and learners. "Crossroads Cafe" is a series of videotape…

Canadian Pacific Railway mechanical services' 5-Alive safety program shows promise in reducing injuries.

DOT National Transportation Integrated Search

2006-09-01

The Federal Railroad Administration (FRA) Human Factors Research and Development (R&D) Program is sponsoring an Alternative Safety Measures Program to explore alternative methods for evaluating whether safety programs improve safety outcomes and the ...

Employer evaluations of nurse graduates: a critical program assessment element.

PubMed

Ryan, M E; Hodson, K E

1992-05-01

Accountability in higher education dictates implementation of a comprehensive evaluation plan. Employer evaluation of graduates is an important component of program evaluation and contributes a different view that is rarely reported in the literature. The purpose of this study was to establish a database by surveying employers of baccalaureate-prepared nurses, postgraduation, over a five-year period. Employer surveys measured perceptions of graduates' functioning. Findings indicated that graduates function above expected levels for leadership skills, nursing skills, communication skills, and professionalism. Systematic program evaluation by employers is recommended at one and five years after graduation. A tool for employer evaluation of baccalaureate graduates is discussed.

A training program for anthropometric measurements by a dedicated nutrition support team improves nutritional status assessment of the critically ill child.

PubMed

Valla, Frederic V; Ford-Chessel, Carole; Meyer, Rosan; Berthiller, Julien; Dupenloup, Christine; Follin-Arbelet, Nathalie; Hubert, Anna; Javouhey, Etienne; Peretti, Noel

2015-03-01

The cornerstone of an optimal nutrition approach in PICUs is to evaluate the nutritional status of any patient. Anthropometric measurements and nutritional indices calculation allow for nutritional status assessment, which is not often part of routine management, as it is considered difficult to perform in this setting. We designed a study to evaluate the impact of a training program by the PICU nutritional support team on the implementation of routine anthropometric measurements on our PICU. A prospective study was performed over a 2-year period, which included: a baseline evaluation of nutritional assessment, knowledge, anthropometric measurements (weight, height, and head and mid upper arm circumferences), and nutritional indices calculation in patient files. This was followed by a training program to implement the newly developed nutrition assessment guidelines, which included anthropometrical measurements and also the interpretation of these. The impact of this nutritional assessment program was reviewed annually for 2 years after the implementation. PICU--Lyon, France. PICU nursing and medical staff, and patients admitted in February 2011, 2012, and 2013. Training program. Ninety-nine percent of staff (n = 145) attended the individual teaching. We found significant progress in nutritional awareness and confidence about nutritional assessment following the teaching program. In addition, an improvement in staff knowledge about undernutrition and its consequences were found. We enrolled 41, 55, and 91 patients in 2011, 2012, and 2013, respectively. There was a significant increase in anthropometric measurements during this time: 32%, 65% (p = 0.002), and 96% in 2013 (p < 0.001). Nutritional indices were calculated in 20%, 74% (p < 0.001), and 96% (p < 0.001) of cases. This is the first study, showing that a targeted nutritional assessment teaching program that highlights both the importance and techniques of anthropometrical measurements has successfully been implemented in a PICU. It managed to improve staff knowledge and nutritional practice.

New Brunswick Laboratory: Progress report, October 1987--September 1988

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

NBL has been tasked by the DOE Office of Safeguards and Security, Defense Programs (OSS/DP) to assure the application of accurate and reliable measurement technology for the safeguarding of special nuclear materials. NBL is fulfilling its mission responsibilities by identifying and addressing the measurement and measurement-related needs of the nuclear material safeguards community. These responsibilities are being addressed by activities in the following program areas: (1) reference and calibration materials, (2) measurement development, (3) measurement services, (4) measurement evaluation, (5) safeguards assessment, and (6) site-specific assistance. Highlights of each of these programs areas are provided in this summary.

Evaluating Training Programs for Primary Care Providers in Child/Adolescent Mental Health in Canada: A Systematic Review

PubMed Central

Espinet, Stacey; Naqvi, Reza; Lingard, Lorelei; Steele, Margaret

2018-01-01

Introduction The need for child/adolescent mental health care in Canada is growing. Primary care can play a key role in filling this gap, yet most providers feel they do not have adequate training. This paper reviews the Canadian literature on capacity building programs in child and adolescent psychiatry for primary care providers, to examine how these programs are being implemented and evaluated to contribute to evidence-based initiatives. Methods A systematic literature review of peer-reviewed published articles of capacity building initiatives in child/adolescent mental health care for primary care practitioners that have been implemented in Canada. Results Sixteen articles were identified that met inclusion criteria. Analysis revealed that capacity building initiatives in Canada are varied but rigorous evaluation methodology is lacking. Primary care providers welcome efforts to increase mental health care capacity and were satisfied with the implementation of most programs. Discussion Objective conclusions regarding the effectiveness of these programs to increase mental health care capacity is challenging given the evaluation methodology of these studies. Conclusion Rigorous evaluation methods are needed to make evidence-based decisions on ways forward to be able to build child/adolescent mental health care capacity in primary care. Outcome measures need to move beyond self-report to more objective measures, and should expand the measurement of patient outcomes to ensure that these initiative are indeed leading to improved care for families. PMID:29662521

Can We Afford Not to Evaluate Services for Elderly Persons with Dementia?

PubMed Central

Worrall, Graham; Chambers, Larry W.

1989-01-01

With the increasing expenditure on health-care programs for seniors, there is an urgent need to evaluate such programs. The Measurement Iterative Loop is a tool that can provide both health administrators and health researchers with a method of evaluation of existing programs and identification of gaps in knowledge, and forms a rational basis for health-care policy decisions. In this article, the Loop is applied to one common problem of the elderly: dementia. PMID:21248993

Using archived data to measure operational benefits of ITS investments, volume 2 : region 1 incident response program.

DOT National Transportation Integrated Search

2004-06-01

The objective of this project is to use the existing data, surveillance and communications infrastructure (to the extent possible) to : develop two case study evaluations for Oregon, including an evaluation of the COMET incident management program an...

Evaluating Training by Four Economic Indices.

ERIC Educational Resources Information Center

Rosentreter, Gary

1979-01-01

This study was intended to demonstrate the effectiveness of four selected economic indicators as criteria for evaluation of a manufacturing firm's management training program in communication skills. Though the results indicated that the program effected change only in employee turnover, the study supports the measurement of training by economic…

Using evaluation methods to guide the development of a tobacco-use prevention curriculum for youth: a case study.

PubMed

Bridge, P D; Gallagher, R E; Berry-Bobovski, L C

2000-01-01

Fundamental to the development of educational programs and curricula is the evaluation of processes and outcomes. Unfortunately, many otherwise well-designed programs do not incorporate stringent evaluation methods and are limited in measuring program development and effectiveness. Using an advertising lesson in a school-based tobacco-use prevention curriculum as a case study, the authors examine the role of evaluation in the development, implementation, and enhancement of the curricular lesson. A four-phase formative and summative evaluation design was developed to divide the program-evaluation continuum into a structured process that would aid in the management of the evaluation, as well as assess curricular components. Formative and summative evaluation can provide important guidance in the development, implementation, and enhancement of educational curricula. Evaluation strategies identified unexpected barriers and allowed the project team to make necessary "time-relevant" curricular adjustments during each stage of the process.

Responding to the Crisis of Accountability: A Review of Program Assessment Methods.

ERIC Educational Resources Information Center

Haley, Eric; Jackson, DeForrest

The advertising program at the University of Tennessee, Knoxville (UTK) has at least 12 measures of program assessment, which serve as a basis for discussion rather than as a prescription for an effective assessment program. The program assessment methods are accrediting, internal program review, teaching evaluations, a university survey of…

Satellite Ocean Color Validation Using Merchant Ships. Chapter 10

NASA Technical Reports Server (NTRS)

Frouin, Robert; Cutchin, David L.; Deschamps, Pierre-Yves

2001-01-01

A collaborative measurement program for evaluating satellite-derived ocean color has been developed based on ships of opportunity (merchant, oceanographic) and specific instrumentation, the SIMBAD radiometer. The purpose of the measurement program is to complement, in a cost-effective way, dedicated evaluation experiments at sea, which are expensive, cannot be carried out over the full range of expected oceanic and atmospheric conditions, and generally provide a few match-ups. Ships participate in the program on a volunteer basis or at a very small cost, and measurement procedures do not interfere with other ship activities. The SIMBAD radiometer is a portable, easy-to-operate instrument that measures the basic ocean color variables, namely aerosol optical thickness and water-leaving radiance, in typical spectral bands of ocean-color sensors, i.e., 443, 490, 560, 670, and 870 nm. Measuring these variables at the time of satellite overpass is usually sufficient to verify satellite-derived ocean color and to evaluate atmospheric correction algorithms. Any ordinary crew can learn quickly how to make measurements. Importantly, the ship is not required to stop, making it possible to collect data along regular routes traveled by merchant ships in the world's oceans.

Early Intervention Preservice Preparation: Program Evaluation and Reflections. Master's Personnel Prep Program. Final Report.

ERIC Educational Resources Information Center

Bricker, Diane; Squires, Jane

This final report discusses the activities and outcomes of the early intervention interdisciplinary preservice program at the University of Oregon. This master's degree program used both "measurement of" and "reflection about" preservice efforts to address important questions regarding program effectiveness and identify…

Program Effectiveness in AISD 1992-93.

ERIC Educational Resources Information Center

Wilkinson, David; Mangino, Evangelina

The Office of Research and Evaluation (ORE) of the Austin Independent School District (AISD) (Texas) reviews the effectiveness of many of the school district's special programs. In 1992-93, ORE reviewed 60 programs or program components. Cost effectiveness was calculated for 31 programs, using an achievement effect measure for 10, a dropout…

Evaluating Dermatology Residency Program Websites.

PubMed

Ashack, Kurt A; Burton, Kyle A; Soh, Jonathan M; Lanoue, Julien; Boyd, Anne H; Milford, Emily E; Dunnick, Cory; Dellavalle, Robert P

2016-03-16

Internet resources play an important role in how medical students access information related to residency programs.Evaluating program websites is necessary in order to provide accurate information for applicants and provide information regarding areas of website improvement for programs. To date, dermatology residency websites (D  WS) have not been evaluated.This paper evaluates dermatology residency websites based on availability of predefined measures. Using the FREIDA (Fellowship and Residency Electronic Interactive Database) Online database, authors searched forall accredited dermatology program websites. Eligible programs were identified through the FREIDA Online database and had a functioning website. Two authors independently extracted data with consensus or third researcher resolution of differences. This data was accessed and archived from July 15th to July 17th, 2015.Primary outcomes measured were presence of content on education, resident and faculty information, program environment, applicant recruitment, schedule, salary, and website quality evaluated using an online tool (WooRank.com). Out of 117 accredited dermatology residencies, 115 had functioning webpages. Of these, 76.5% (75) had direct links found on the FRIEDA Online database. Most programs contained information on education, faculty, program environment, and applicant recruitment. However, website quality and marketing effectiveness were highly variable; most programs were deemed to need improvements in the functioning of their webpages. Also, additional information on current residents and about potential away rotations were lacking from most websites with only 52.2% (60) and 41.7% (48) of programs providing this content, respectively. A majority of dermatology residency websites contained adequate information on many of the factors we evaluated. However, many were lacking in areas that matter to applicants. We hope this report will encourage dermatology residencyprograms to improve their websites and provide adequate content to attract the top residents for their respective programs.

Activities of the DOE Nuclear Criticality Safety Program (NCSP) at the Oak Ridge Electron Linear Accelerator (ORELA)

NASA Astrophysics Data System (ADS)

Valentine, Timothy E.; Leal, Luiz C.; Guber, Klaus H.

2002-12-01

The Department of Energy established the Nuclear Criticality Safety Program (NCSP) in response to the Recommendation 97-2 by the Defense Nuclear Facilities Safety Board. The NCSP consists of seven elements of which nuclear data measurements and evaluations is a key component. The intent of the nuclear data activities is to provide high resolution nuclear data measurements that are evaluated, validated, and formatted for use by the nuclear criticality safety community to provide improved and reliable calculations for nuclear criticality safety evaluations. High resolution capture, fission, and transmission measurements are performed at the Oak Ridge Electron Linear Accelerator (ORELA) to address the needs of the criticality safety community and to address known deficiencies in nuclear data evaluations. The activities at ORELA include measurements on both light and heavy nuclei and have been used to identify improvements in measurement techniques that greatly improve the measurement of small capture cross sections. The measurement activities at ORELA provide precise and reliable high-resolution nuclear data for the nuclear criticality safety community.

7 CFR 1484.72 - How is program effectiveness measured?

Code of Federal Regulations, 2010 CFR

2010-01-01

... 7 Agriculture 10 2010-01-01 2010-01-01 false How is program effectiveness measured? 1484.72... effectiveness measured? (a) The Government Performance and Results Act (GPRA) of 1993 (5 U.S.C. 306; 31 U.S.C..., providing the basis for the strategic plan. The evaluation results guide the development and scope of a...

Key Design Considerations When Calculating Cost Savings for Population Health Management Programs in an Observational Setting.

PubMed

Murphy, Shannon M E; Hough, Douglas E; Sylvia, Martha L; Dunbar, Linda J; Frick, Kevin D

2018-02-08

To illustrate the impact of key quasi-experimental design elements on cost savings measurement for population health management (PHM) programs. Population health management program records and Medicaid claims and enrollment data from December 2011 through March 2016. The study uses a difference-in-difference design to compare changes in cost and utilization outcomes between program participants and propensity score-matched nonparticipants. Comparisons of measured savings are made based on (1) stable versus dynamic population enrollment and (2) all eligible versus enrolled-only participant definitions. Options for the operationalization of time are also discussed. Individual-level Medicaid administrative and claims data and PHM program records are used to match study groups on baseline risk factors and assess changes in costs and utilization. Savings estimates are statistically similar but smaller in magnitude when eliminating variability based on duration of population enrollment and when evaluating program impact on the entire target population. Measurement in calendar time, when possible, simplifies interpretability. Program evaluation design elements, including population stability and participant definitions, can influence the estimated magnitude of program savings for the payer and should be considered carefully. Time specifications can also affect interpretability and usefulness. © Health Research and Educational Trust.

Incarcerated women develop a nutrition and fitness program: participatory research.

PubMed

Elwood Martin, Ruth; Adamson, Sue; Korchinski, Mo; Granger-Brown, Alison; R Ramsden, Vivian; A Buxton, Jane; Espinoza-Magana, Nancy; L Pollock, Sue; J F Smith, Megan; C Macaulay, Ann; Lisa Condello, Lara; Gregory Hislop, T

2013-01-01

Women in prison throughout the world experience higher rates of mental and physical illness compared with the general population and compared with men in prison. The paper finds no published studies that report on men or women in prison engaging in participatory health research to address their concerns about nutrition and fitness. The purpose of this paper is to describe a pilot nutrition and fitness program, which resulted from a unique prison participatory health research project. Women in prison designed, led, and evaluated a six-week pilot fitness program in a minimum/medium security women's prison. Pre- and post-program assessments included a self-administered questionnaire and body measures. Open-ended questionnaire responses illuminated the quantitative findings. Sixteen women in prison completed the program evaluation. Weight, body mass index, waist-to-hip ratio, and chest measurements decreased, and energy, sleep, and stress levels improved by the end of the program. As a component of a participatory research project, incarcerated women designed and led a nutrition and fitness program, which resulted in improved body measures and self-reported health benefits. ORIGINALITY VALUE: Incarceration provides opportunities to engage women in designing their own health programs with consequent potential long-term "healing" benefits.

A randomized, single-blind cross-over design evaluating the effectiveness of an individually defined, targeted physical therapy approach in treatment of children with cerebral palsy.

PubMed

Franki, Inge; Van den Broeck, Christine; De Cat, Josse; Tijhuis, Wieke; Molenaers, Guy; Vanderstraeten, Guy; Desloovere, Kaat

2014-10-01

A pilot study to compare the effectiveness of an individual therapy program with the effects of a general physical therapy program. A randomized, single-blind cross-over design. Ten ambulant children with bilateral spastic cerebral palsy, age four to nine years. Participants were randomly assigned into a ten-week individually defined, targeted or a general program, followed by a cross-over. Evaluation was performed using the Gross Motor Function Measure-88 and three-dimensional gait analysis. General outcome parameters were Gross Motor Function Measure-88 scores, time and distance parameters, gait profile score and movement analysis profiles. Individual goal achievement was evaluated using z-scores for gait parameters and Goal Attainment Scale for gross motor function. No significant changes were observed regarding gross motor function. Only after individualized therapy, step- and stride-length increased significantly (p = 0.022; p = 0.017). Change in step-length was higher after the individualized program (p = 0.045). Within-group effects were found for the pelvis in transversal plane after the individualized program (p = 0.047) and in coronal plane after the general program (p = 0.047). Between-program differences were found for changes in the knee in sagittal plane, in the advantage of the individual program (p = 0.047). A median difference in z-score of 0.279 and 0.419 was measured after the general and individualized program, respectively. Functional goal attainment was higher after the individual therapy program compared with the general program (48 to 43.5). The results indicate slightly favorable effects towards the individualized program. To detect clinically significant changes, future studies require a minimal sample size of 72 to 90 participants. © The Author(s) 2014.

Making quality improvement programs more effective.

PubMed

Shaw-Taylor, Yoku

2014-01-01

In the past 25 years, and as recent as 2011, all external evaluations of the Quality Improvement Organization (QIO) Program have found its impact to be small or difficult to discern. The QIO program costs about $200 million on average to administer each year to improve quality of healthcare for people of 65 years or older. The program was created to address questionable quality of care. QIOs review how care is provided based on performance measures. The paper aims to discuss these issues. In 2012, the author supported the production of quarterly reports and reviewed internal monitoring and evaluation protocols of the program. The task also required reviewing all previous program evaluations. The task involved many conversations about the complexities of the program, why impact is difficult to discern and possible ways for eventual improvement. Process flow charts were created to simulate the data life cycle and discrete event models were created based on the sequence of data collection and reporting to identify gaps in data flow. The internal evaluation uncovered data gaps within the program. The need for a system of specification rules for data conceptualization, collection, distribution, discovery, analysis and repurposing is clear. There were data inconsistencies and difficulty of integrating data from one instance of measurement to the next. The lack of good and reliable data makes it difficult to discern true impact. The prescription is for a formal data policy or data governance structure to integrate and document all aspects of the data life cycle. The specification rules for governance are exemplified by the Data Documentation Initiative and the requirements published by the Data Governance Institute. The elements are all in place for a solid foundation of the data governance structure. These recommendations will increase the value of program data. The model specifies which agency units must be included in the governance authority and the data team. The model prescribes in detail a data governance model to address gaps in the life cycle. These prescriptive measures will allow the program to integrate all of its data. Without this formal data governance structure, the QIO program will be undetermined by the persistent lack of good data for monitoring and evaluation.

Evaluation of programs to improve complementary feeding in infants and young children.

PubMed

Frongillo, Edward A

2017-10-01

Evaluation of complementary feeding programs is needed to enhance knowledge on what works, to document responsible use of resources, and for advocacy. Evaluation is done during program conceptualization and design, implementation, and determination of effectiveness. This paper explains the role of evaluation in the advancement of complementary feeding programs, presenting concepts and methods and illustrating them through examples. Planning and investments for evaluations should occur from the beginning of the project life cycle. Essential to evaluation is articulation of a program theory on how change would occur and what program actions are required for change. Analysis of program impact pathways makes explicit the dynamic connections in the program theory and accounts for contextual factors that could influence program effectiveness. Evaluating implementation functioning is done through addressing questions about needs, coverage, provision, and utilization using information obtained from process evaluation, operations research, and monitoring. Evaluating effectiveness is done through assessing impact, efficiency, coverage, process, and causality. Plausibility designs ask whether the program seemed to have an effect above and beyond external influences, often using a nonrandomized control group and baseline and end line measures. Probability designs ask whether there was an effect using a randomized control group. Evaluations may not be able to use randomization, particularly for programs implemented at a large scale. Plausibility designs, innovative designs, or innovative combinations of designs sometimes are best able to provide useful information. Further work is needed to develop practical designs for evaluation of large-scale country programs on complementary feeding. © 2017 John Wiley & Sons Ltd.

The Effectiveness of a Bicycle Safety Program for Improving Safety-Related Knowledge and Behavior in Young Elementary Students

PubMed Central

Glang, Ann

2010-01-01

Objective The purpose of this study was to evaluate the “Bike Smart” program, an eHealth software program that teaches bicycle safety behaviors to young children. Methods Participants were 206 elementary students in grades kindergarten to 3. A random control design was employed to evaluate the program, with students assigned to either the treatment condition (Bike Smart) or the control condition (a video on childhood safety). Outcome measures included computer-based knowledge items (safety rules, helmet placement, hazard discrimination) and a behavioral measure of helmet placement. Results Results demonstrated that regardless of gender, cohort, and grade the participants in the treatment group showed greater gains than control participants in both the computer-presented knowledge items (p > .01) and the observational helmet measure (p > .05). Conclusions Findings suggest that the Bike Smart program can be a low cost, effective component of safety training packages that include both skills-based and experiential training. PMID:19755497

Primary prevention research: a preliminary review of program outcome studies.

PubMed

Schaps, E; Churgin, S; Palley, C S; Takata, B; Cohen, A Y

1980-07-01

This article reviews 35 drug abuse prevention program evaluations employing drug-specific outcome measures. Many of these evaluations assessed the effects of "new generation" prevention strategies: affective, peer-oriented, and multidimensional approaches. Only 14 studies evaluated purely informational programs. Evaluations were analyzed to ascertain (1) characteristics of the programs under study, (2) characteristics of the research designs, and (3) patterns among findings. This review provides some evidence that the newer prevention strategies may produce more positive and fewer negative outcomes than did older drug information approaches. Over 70% of the programs using the newer strategies produced some positive effects; only 29% showed negative effects. In contrast, 46% of informational programs showed positive effects; 46% showed negative effects. These findings must be approached with great caution, since the research was frequently scientifically inadequate, and since rigor of research was negatively correlated with intensity and duration of program services.

Methods for the evaluation of the Jamie Oliver Ministry of Food program, Australia.

PubMed

Flego, Anna; Herbert, Jessica; Gibbs, Lisa; Swinburn, Boyd; Keating, Catherine; Waters, Elizabeth; Moodie, Marj

2013-04-30

Community-based programs aimed at improving cooking skills, cooking confidence and individual eating behaviours have grown in number over the past two decades. Whilst some evidence exists to support their effectiveness, only small behavioural changes have been reported and limitations in study design may have impacted on results.This paper describes the first evaluation of the Jamie Oliver Ministry of Food Program (JMoF) Australia, in Ipswich, Queensland. JMoF Australia is a community-based cooking skills program open to the general public consisting of 1.5 hour classes weekly over a 10 week period, based on the program of the same name originating in the United Kingdom. A mixed methods study design is proposed. Given the programmatic implementation of JMoF in Ipswich, the quantitative study is a non-randomised, pre-post design comparing participants undergoing the program with a wait-list control group. There will be two primary outcome measures: (i) change in cooking confidence (self-efficacy) and (ii) change in self-reported mean vegetable intake (serves per day). Secondary outcome measures will include change in individual cooking and eating behaviours and psycho-social measures such as social connectedness and self-esteem. Repeated measures will be collected at baseline, program completion (10 weeks) and 6 months follow up from program completion. A sample of 250 participants per group will be recruited for the evaluation to detect a mean change of 0.5 serves a day of vegetables at 80% power (0.5% significance level). Data analysis will assess the magnitude of change of these variables both within and between groups and use sub group analysis to explore the relationships between socio-demographic characteristics and outcomes.The qualitative study will be a longitudinal design consisting of semi-structured interviews with approximately 10-15 participants conducted at successive time points. An inductive thematic analysis will be conducted to explore social, attitudinal and behavioural changes experienced by program participants. This evaluation will contribute to the evidence of whether cooking programs work in terms of improving health and wellbeing and the underlying mechanisms which may lead to positive behaviour change. Australian and New Zealand Trial registration number: ACTRN12611001209987.

A long-term plan for evaluating the FHWA's Office of Safety programs : final draft

DOT National Transportation Integrated Search

2006-01-01

The purpose of this report is to develop a generalized plan for evaluating and measuring the effectiveness of the major safety programs of the the FHWA Office of Safety. This report is organized into three sections in order to coincide with the state...

The Vale rangeland rehabilitation program: an evaluation.

Treesearch

Harold F. Heady

1988-01-01

This manuscript discusses the initiation, execution, and outcome of an 11-year (1962-1972) rangeland rehabilitation program in southeastern Oregon. Res. Bull. PNW-RB-070 (1977) is updated with 1986 measurements and evaluations of vegetational conditions, wildlife, recreational use, livestock grazing, and management of public rangelands. The mix of multiple uses has...

A Multicultural Assessment of Adolescent Connectedness: Testing Measurement Invariance across Gender and Ethnicity

ERIC Educational Resources Information Center

Karcher, Michael J.; Sass, Daniel

2010-01-01

Counselors, psychologists, and evaluators of intervention programs for youth increasingly view the promotion of connectedness as an important intervention outcome. When evaluating these programs, researchers frequently test whether the treatment effects differ across gender and ethnic or racial groups. Doing so necessitates the availability of…

Evaluating Cooperative Education Programs.

ERIC Educational Resources Information Center

Alvir, Howard P.

This document defines cooperative education as any form of occupational or professional activity that required the cooperation of both school and the labor market. In some cases, this might be the school and industry or business. In this process, evaluation is defined as the improvement of learner success through measurement of program components.…

The Use and Evaluation of Videodiscs in the Chemistry Laboratory.

ERIC Educational Resources Information Center

Russell, Arlene A.; And Others

1985-01-01

Describes a quantitative evaluation of an interactive videodisc program in which students measure the temperature dependence of the solubility product of lead chloride by titration of chloride with silver nitrate using a Mohr titration. Student reaction (based on responses made using the program, quiz answers, and laboratory performance) was…

Evaluation of Nighttime Seat Belt Enforcement Demonstration Program and Identification of Characteristics of Unbelted High-Risk Drivers

DOT National Transportation Integrated Search

2018-04-01

The present study evaluated the effectiveness of a high-visibility nighttime seat belt enforcement program conducted in Maryland by measuring changes in day and night self-reported and observed seat belt use and crash outcomes. To better understand t...

Direct Measures in Environmental Education Evaluation: Behavioral Intentions versus Observable Actions

ERIC Educational Resources Information Center

Camargo, Camilo; Shavelson, Richard

2009-01-01

The objective of many environmental education programs is to promote pro-environmental attitudes and behaviors in students. However, evaluation of these programs has focused on asking participants what they think (attitudes) and what they do (behaviors) regarding the environment problems through self-report questionnaires and interviews. These…

Space transportation system shuttle turnabout analysis report

NASA Technical Reports Server (NTRS)

Reedy, R. E.

1979-01-01

The progress made and the problems encountered by the various program elements of the shuttle program in achieving the 160 hour ground turnaround goal are presented and evaluated. Task assessment time is measured against the program allocation time.

Multidimensional outcome considerations in assessing the efficacy of medical educational programs.

PubMed

Blumberg, Phyllis

2003-01-01

To be accredited, schools must evaluate the effectiveness of their programs. Educators are looking for specific indicators beyond the traditional measures. Data from multiple methods, including indicators of student performance and of the institutional environment, can be used to determine if educational program goals are met. This article outlines specific ways to consider three types of efficacy outcomes that are consistent with accreditation guidelines: educational, clinical career, and environmental outcomes. Specific measurable outcomes for each of these categories are derived from higher education literature: for example, learning approaches and information acquisition for education; professional behaviors and interpersonal dimensions for clinical career, and scholarship of teaching and teaching itself as a shared and valued activity for environmental outcomes. These outcomes address student assessment and program evaluation. Data from problem-based learning programs illustrate these outcomes. Educators can determine if educational program innovations have met their goals by collecting efficacy outcome data.

Evaluative Thinking in Practice: The National Asthma Control Program

PubMed Central

Fierro, Leslie A.; Codd, Heather; Gill, Sarah; Pham, Phung K.; Grandjean Targos, Piper T.; Wilce, Maureen

2018-01-01

Although evaluative thinking lies at the heart of what we do as evaluators and what we hope to promote in others through our efforts to build evaluation capacity, researchers have given limited attention to measuring this concept. We undertook a research study to better understand how instances of evaluative thinking may present in practice-based settings—specifically within four state asthma control programs funded by the Centers for Disease Control and Prevention’s National Asthma Control Program. Through content analyses of documents as well as interviews and a subsequent focus group with four state asthma control programs’ evaluators and program managers we identified and defined twenty-two indicators of evaluative thinking. Findings provide insights about what practitioners may wish to look for when they intend to build evaluative thinking and the types of data sources that may be more or less helpful in such efforts. PMID:29950803

A retrospective evaluation of the Perfecting Patient Care University training program for health care organizations.

PubMed

Morganti, Kristy Gonzalez; Lovejoy, Susan; Beckjord, Ellen Burke; Haviland, Amelia M; Haas, Ann C; Farley, Donna O

2014-01-01

This study evaluated how the Perfecting Patient Care (PPC) University, a quality improvement (QI) training program for health care leaders and clinicians, affected the ability of organizations to improve the health care they provide. This training program teaches improvement methods based on Lean concepts and principles of the Toyota Production System and is offered in several formats. A retrospective evaluation was performed that gathered data on training, other process factors, and outcomes after staff completed the PPC training. A majority of respondents reported gaining QI competencies and cultural achievements from the training. Organizations had high average scores for the success measures of "outcomes improved" and "sustainable monitoring" but lower scores for diffusion of QI efforts. Total training dosage was significantly associated with the measures of QI success. This evaluation provides evidence that organizations gained the PPC competencies and cultural achievements and that training dosage is a driver of QI success.

Suggested Courseware for the Non-Calculus Physics Student: Measurement, Vectors, and One-Dimensional Motion.

ERIC Educational Resources Information Center

Mahoney, Joyce; And Others

1988-01-01

Evaluates 16 commercially available courseware packages covering topics for introductory physics. Discusses the price, sub-topics, program type, interaction, time, calculus required, graphics, and comments of each program. Recommends two packages in measurement and vectors, and one-dimensional motion respectively. (YP)

Evaluation of the Florida coordinated school health program pilot schools project.

PubMed

Weiler, Robert M; Pigg, R Morgan; McDermott, Robert J

2003-01-01

The Florida Department of Education, with CDC funding, designed the Florida Coordinated School Health Program Pilot Schools Project (PSP) to encourage innovative approaches to promote coordinated school health programs (CSHP) in Florida schools. Each of eight pilot schools received $15,000 in project funding, three years of technical assistance including on-site and off-site assistance, a project office resource center, mailings of resource materials, needs assessment and evaluation assistance, and three PSP Summer Institutes. Project evaluators created a context evaluation, approaching each school independently as a "case study" to measure the school's progress in meeting goals established at baseline. Data were collected using the How Healthy is Your School? needs assessment instrument, a School Health Portfolio constructed by each school team, a Pilot Schools Project Team Member Survey instrument, midcourse team interviews, final team interviews, and performance indicator data obtained from pilot and control schools. The PSP posed two fundamental questions: "Can financial resources, professional training, and technical assistance enable individual schools to create and sustain a coordinated school health program?" and "What outcomes reasonably can one expect from a coordinated school health program, assuming programs receive adequate support over time?" First, activities at the eight schools confirmed that a coordinated school health programs can be established and sustained. Program strength and sustainability depend on long-term resources, qualified personnel, and administrative support. Second, though coordinated school health programs may improve school performance indicators, the PSP yielded insufficient evidence to support that belief. Future projects should include robust measurement and evaluation designs, thereby producing conclusive evidence about the influence of a coordinated school health program on such outcomes.

Evaluating the impacts of the clean cities program.

PubMed

Qiu, Shiyong; Kaza, Nikhil

2017-02-01

The Department of Energy's Clean Cities program was created in 1993 to reduce petroleum usage in the transportation sector. The program promotes alternative fuels such as biofuels and fuel-saving strategies such as idle reduction and fleet management through coalitions of local government, non-profit, and private actors. Few studies have evaluated the impact of the program because of its complexity that include interrelated strategies of grants, education and training and diversity of participants. This paper uses a Difference-in-Differences (DiD) approach to evaluate the effectiveness of the program between 1990 and 2010. We quantify the effectiveness of the Clean Cities program by focusing on performance measures such as air quality, number of alternative fueling stations, private vehicle occupancy and transit ridership. We find that counties that participate in the program perform better on all these measures compared to counties that did not participate. Compared to the control group, counties in the Clean Cities program experienced a reduction in days with bad air quality (3.7%), a decrease in automobile commuters (2.9%), an overall increase in transit commuters (2.1%) and had greater numbers of new alternative fueling stations (12.9). The results suggest that the program is a qualified success. Copyright © 2016 Elsevier B.V. All rights reserved.

MOE vs. M&E: considering the difference between measuring strategic effectiveness and monitoring tactical evaluation.

PubMed

Diehl, Glen; Major, Solomon

2015-01-01

Measuring the effectiveness of military Global Health Engagements (GHEs) has become an area of increasing interest to the military medical field. As a result, there have been efforts to more logically and rigorously evaluate GHE projects and programs; many of these have been based on the Logic and Results Frameworks. However, while these Frameworks are apt and appropriate planning tools, they are not ideally suited to measuring programs' effectiveness. This article introduces military medicine professionals to the Measures of Effectiveness for Defense Engagement and Learning (MODEL) program, which implements a new method of assessment, one that seeks to rigorously use Measures of Effectiveness (vs. Measures of Performance) to gauge programs' and projects' success and fidelity to Theater Campaign goals. While the MODEL method draws on the Logic and Results Frameworks where appropriate, it goes beyond their planning focus by using the latest social scientific and econometric evaluation methodologies to link on-the-ground GHE "lines of effort" to the realization of national and strategic goals and end-states. It is hoped these methods will find use beyond the MODEL project itself, and will catalyze a new body of rigorous, empirically based work, which measures the effectiveness of a broad spectrum of GHE and security cooperation activities. We based our strategies on the principle that it is much more cost-effective to prevent conflicts than it is to stop one once it's started. I cannot overstate the importance of our theater security cooperation programs as the centerpiece to securing our Homeland from the irregular and catastrophic threats of the 21st Century.-GEN James L. Jones, USMC (Ret.). Reprint & Copyright © 2015 Association of Military Surgeons of the U.S.

Measuring the quality of child health care at first-level facilities.

PubMed

Gouws, Eleanor; Bryce, Jennifer; Pariyo, George; Armstrong Schellenberg, Joanna; Amaral, João; Habicht, Jean-Pierre

2005-08-01

Sound policy and program decisions require timely information based on valid and relevant measures. Recent findings suggest that despite the availability of effective and affordable guidelines for the management of sick children in first-level health facilities in developing countries, the quality and coverage of these services remains low. We report on the development and evaluation of a set of summary indices reflecting the quality of care received by sick children in first-level facilities. The indices were first developed through a consultative process to achieve face validity by involving technical experts and policymakers. The definition of evaluation measures for many public health programs stops at this point. We added a second phase in which standard statistical techniques were used to evaluate the content and construct validity of the indices and their reliability, drawing on data sets from the multi-country evaluation of integrated management of childhood illness (MCE) in Brazil, Tanzania and Uganda. The statistical evaluation identified important conceptual errors in the indices arising from the theory-driven expert review. The experts had combined items into inappropriate indicators resulting in summary indices that were difficult to interpret and had limited validity for program decision making. We propose a revised set of summary indices for the measurement of child health care in developing countries that is supported by both expert and statistical reviews and that led to similar programmatic insights across the three countries. We advocate increased cross-disciplinary research within public health to improve measurement approaches. Child survival policymakers, program planners and implementers can use these tools to improve their monitoring and so increase the health impact of investments in health facility care.

A Comparison of Evaluation Practices Based on E-Learning and Mobile Learning Delivery Rates

ERIC Educational Resources Information Center

Marshall, James

2018-01-01

Learning and performance professionals are increasingly pressed to measure the results of their learning program design efforts, and ultimately prove their worth. However, evaluation efforts are often limited to measuring participant reaction. This study sought to quantify evaluation practices in organizations and investigate how the use of…

Alternative Methods in the Evaluation of School District Cash Management Programs.

ERIC Educational Resources Information Center

Dembowski, Frederick L.

1980-01-01

Empirically evaluates three measures of effectiveness of school district cash management: the rate of return method in common use and two new measures--efficiency rating and Net Present Value (NPV). The NPV approach allows examination of efficiency and provides a framework for evaluating other areas of educational policy. (Author/IRT)

An evaluation of Washington's Medicaid disease-management program.

PubMed

Lind, Alice; Kaplan, Louise

2007-10-01

In 2002, Washington State Medicaid implemented a disease-management program for clients with diagnoses of asthma, chronic obstructive pulmonary disease, heart failure, and diabetes. The program represented a unique attempt to manage disabled clients in a fee-for-services environment, and at its onset, was one of the first statewide programs in the United States. This article reviews the effectiveness of the program based on the results from two independent evaluations. Results of cost-savings measurements and health outcomes are presented for each of the conditions. These results were used to make program changes, which began in 2007.

A novel clinical evaluation method using maximum angular velocity during knee extension to assess lower extremity muscle function of older adults.

PubMed

Arai, Takeshi; Obuchi, Shuichi; Shiba, Yoshitaka

2017-11-01

The purpose of this study is to examine the utilities of maximum angular velocity (AV) assessment during knee extension (KE) using a gyroscope for clinical evaluation of exercise program for older adults. Two hundred and 4 community-dwelling older adults underwent a 3-month exercise intervention program. Outcome measures included AV during KE and other physical functions (isometric strength (IS), walking abilities, and balance functions). A correlation coefficient was used to evaluate the relationships between AV and other physical functions at baseline. The differences of physical functions before and after intervention were evaluated and the effect size of each measurement was calculated after the program. The AV measurement was significantly correlated with IS during KE (r=0.303, P<0.01) and other physical functions. Most correlation coefficients of angular velocity were greater than that of IS. All of physical assessments were significantly improved. Also, effect size of AV was greater than that of IS (d=0.45 vs. 0.42). AV of the lower extremities is useful to evaluate the effects of exercise intervention in the elderly. Copyright © 2017 Elsevier B.V. All rights reserved.

Individualized Education Program (IEP) Planning Guide for Special Education Students Entering High School Vocational Education Programs.

ERIC Educational Resources Information Center

Burrell, Lewis P.

This guide is designed to assist school program planners who are working with mainstreamed special education students in vocational education programs. The guide, covering grades 11 and 12, contains vocational program goals, objectives, and evaluation measures for 30 secondary vocational education programs in 15 curriculum areas, as well as for…

New criteria for measuring range management activities.

Treesearch

T.M. Quigley; D.S. Dillard; J.B. [and others] Reese

1989-01-01

The USDA Forest Service national range program is currently evaluating its information needs at the national level A Range Measurement Task Group of Agency personnel was assembled in January 1988 to evaluate the information needs and recommend appropriate measures and reports. This document is the final recommendation of the task group. The recommendation includes...

PubMed Central

Worrall, Graham; Chambers, Larry W.

1990-01-01

With the increasing expenditure on health care programs for seniors, there is an urgent need to evaluate such programs. The Measurement Iterative Loop is a tool that can provide both health administrators and health researchers with a method of evaluation of existing programs and identification of gaps in knowledge, and forms a rational basis for health-care policy decisions. In this article, the Loop is applied to one common problem of the elderly: dementia. PMID:21233998

Defining and evaluating quality for ambulatory care educational programs.

PubMed

Bowen, J L; Stearns, J A; Dohner, C; Blackman, J; Simpson, D

1997-06-01

As the training of medical students and residents increasingly moves to ambulatory care settings, clerkship and program directors must find a way to use their limited resources to guide the development and evaluation of the quality of these ambulatory-based learning experiences. To evaluate quality, directors must first define, in operational and measurable terms, what is meant by the term "quality" as it is applied to ambulatory-based education. Using educational theories and the definition of quality used by health care systems, the authors propose an operational definition of quality for guiding the planning, implementation, and evaluation of ambulatory care educational programs. They assert that quality is achieved through the interaction of an optimal learning environment, defined educational goals and positive outcomes, participant satisfaction, and cost-effectiveness. By describing the components of quality along with examples of measurable indicators, the authors provide a foundation for the evaluation and improvement of instructional innovations in ambulatory care education for the benefit of teachers, learners, and patients.

Correlates of quality educational programs.

PubMed

Chester, Deborah R; Tracy, Jessamyn A; Earp, Emily; Chauhan, Reetu

2002-06-01

Preliminary evaluation findings are presented that explore relationships between educational program quality and program characteristics such as program type, security level, aftercare, teacher certification, facility size, and private versus public provider. Several program characteristics are found to be related to measurements of educational program quality. Among the major quality characteristics are proportion of program teachers that are professionally certified, smaller sized facilities versus larger facilities, level of aftercare services, and provider sources, with private for-profit providers being the lowest performing and public providers being the highest performing. The article closes with description of the Juvenile Justice Educational Enhancement Program's continuing evaluation of correlates to educational program quality through the continued development of a comprehensive database.

Rigorous Measures of Implementation: A Methodological Framework for Evaluating Innovative STEM Programs

ERIC Educational Resources Information Center

Cassata-Widera, Amy; Century, Jeanne; Kim, Dae Y.

2011-01-01

The practical need for multidimensional measures of fidelity of implementation (FOI) of reform-based science, technology, engineering, and mathematics (STEM) instructional materials, combined with a theoretical need in the field for a shared conceptual framework that could support accumulating knowledge on specific enacted program elements across…

Measuring the effectiveness of conservation programs for shrubland birds

Treesearch

Scott Schlossberg; David I. King

2015-01-01

Disturbance-dependent habitats such as grasslands and shrublands are declining in many regions. To mitigate these declines, government agencies are using anthropogenic disturbances like logging and mowing to mimic natural ones. Because these programs can be costly or controversial, measuring their effectiveness is important. Here, we evaluate the conservation...

Phoenix Rising: use of a participatory approach to evaluate a federally funded HIV, hepatitis and substance abuse prevention program.

PubMed

Dryden, Eileen; Hyde, Justeen; Livny, Ayala; Tula, Monique

2010-11-01

This paper highlights the value of utilizing a participatory evaluation approach when working with community agencies receiving federal funding for prevention and intervention services. Drawing from our experience as evaluators of a SAMHSA-funded substance abuse, HIV and Hepatitis prevention program targeting homeless young adults, we describe the importance of and strategies for creating a participatory evaluation partnership with program implementers. By participatory evaluation we mean the active involvement of program implementers in defining the evaluation, developing instruments, collecting data, discussing findings, and disseminating results. There are a number of challenges faced when using this approach with federally funded programs that require the use of standardized measurement tools and data collection procedures. Strategies we used to strike a balance between federal requirements and local needs are presented. By increasing the understanding of and participation in the evaluation process, program implementers have greater support for data collection requirements and are appreciably more interested in learning from the evaluation data. This approach has helped to build the capacity of a program and stimulated new possibilities for learning, growing, and ultimately improving the services offered to those the program strives to reach. Copyright (c) 2010 Elsevier Ltd. All rights reserved.

Usability study of the EduMod eLearning Program for contouring nodal stations of the head and neck.

PubMed

Deraniyagala, Rohan; Amdur, Robert J; Boyer, Arthur L; Kaylor, Scott

2015-01-01

A major strategy for improving radiation oncology education and competence evaluation is to develop eLearning programs that reproduce the real work environment. A valuable measure of the quality of an eLearning program is "usability," which is a multidimensional endpoint defined from the end user's perspective. The gold standard for measuring usability is the Software Usability Measurement Inventory (SUMI). The purpose of this study is to use the SUMI to measure usability of an eLearning course that uses innovative software to teach and test contouring of nodal stations of the head and neck. This is a prospective institutional review board-approved study in which all participants gave written informed consent. The study population was radiation oncology residents from 8 different programs across the United States. The subjects had to pass all sections of the same 2 eLearning modules and then complete the SUMI usability evaluation instrument. We reached the accrual goal of 25 participants. Usability results for the EduMod eLearning course, "Nodal Stations of the Head and Neck," were compared with a large database of scores of other major software programs. Results were evaluated in 5 domains: Affect, Helpfulness, Control, Learnability, and Global Usability. In all 5 domains, usability scores for the study modules were higher than the database mean and statistically superior in 4 domains. This is the first study to evaluate usability of an eLearning program related to radiation oncology. Usability of 2 representative modules related to contouring nodal stations of the head and neck was highly favorable, with scores that were superior to the industry standard in multiple domains. These results support the continued development of this type of eLearning program for teaching and testing radiation oncology technical skills. Copyright © 2015 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.

Using the Fennema-Sherman Mathematics Attitude Scales with lower-primary teachers

NASA Astrophysics Data System (ADS)

Ren, Lixin; Green, Jennifer L.; Smith, Wendy M.

2016-06-01

The Fennema-Sherman Mathematics Attitude Scales (FSMAS) are among the most popular instruments used in studies of attitudes toward mathematics. However, the FSMAS has been mainly used among student populations and rarely used with teachers. In the present study, three scales from the FSMAS— Confidence, Effectance Motivation, and Anxiety—were revised and used with lower-primary (kindergarten to third grade) teachers. This study includes three parts: (1) a pilot study to ensure the modifications made to the FSMAS were appropriate to use with teachers, (2) confirmatory factor analyses to assess the factor structure of the revised FSMAS with 225 lower-primary teachers, and (3) measurement invariance analyses using data from a similar sample of 171 lower-primary teachers to examine whether the revised FSMAS measures each construct in the same way as in the previous sample. The final three-factor model, after removing three problematic items, achieves acceptable model fit, with each construct meeting all conditions for strict measurement invariance. Additionally, repeated measures analyses were performed on data collected from 39 in-service lower-primary teachers who participated in an elementary mathematics specialist program to examine the use of the revised FSMAS in program evaluation. Overall results suggest that researchers and program evaluators may use the revised FSMAS to reliably measure lower-primary teachers' mathematical attitudes, and it can be a valuable tool for evaluating the effectiveness of professional development programs.

Graduate nurses' evaluation of mentorship: Development of a new tool.

PubMed

Tiew, Lay Hwa; Koh, Catherine S L; Creedy, Debra K; Tam, W S W

2017-07-01

Develop and test an instrument to measure graduate-nurses' perceptions of a structured mentorship program. New graduate nurses may experience difficulties in the transition from student to practitioner. Mentoring is commonly used to support graduates. However, there is a lack of published tools measuring graduate nurses' perceptions of mentorship. As mentoring is resource intensive, development and testing of a validated tool are important to assist in determining program effectiveness. A pretest-posttest interventional design was used. Following a critical review of literature and content experts' input, the 10-item National University Hospital Mentorship Evaluation (NUH ME) instrument was tested with a convenience sample of 83 graduate nurses. Psychometric tests included internal reliability, stability, content validity, and factor analysis. Changed scores were evaluated using paired samples t-test. Seventy-three graduates (88%) out of a possible 83 completed the pre-and post-program survey. Internal reliability was excellent with a Cronbach's alpha of 0.92. Test-retest reliability was stable over time (ICC=0.81). Exploratory factor analysis supported a 1-factor solution explaining 58.2% of variance. Paired samples t-test showed statistical significance between the pre- and post-program scores (p<0.001). The NUH-ME measure was found to be valid and reliable. Confirmatory Factor Analysis of the tool with different groups of nursing graduates is required. Mentorship programs can be an effective recruitment and retention strategy, but are also resource intensive. Measuring new graduates' perceptions of mentoring contributes to program relevance in addressing their personal, professional and clinical skill development needs. As mentoring engages a diverse range of mentors, feedback through measurement may also positively alter organizational learning culture. Copyright © 2017. Published by Elsevier Ltd.

Self Evaluation of Organizations.

ERIC Educational Resources Information Center

Pooley, Richard C.

Evaluation within human service organizations is defined in terms of accepted evaluation criteria, with reasonable expectations shown and structured into a model of systematic evaluation practice. The evaluation criteria of program effort, performance, adequacy, efficiency and process mechanisms are discussed, along with measurement information…

Clinical peer review program self-evaluation for US hospitals.

PubMed

Edwards, Marc T

2010-01-01

Prior research has shown wide variation in clinical peer review program structure, process, governance, and perceived effectiveness. This study sought to validate the utility of a Peer Review Program Self-Evaluation Tool as a potential guide to physician and hospital leaders seeking greater program value. Data from 330 hospitals show that the total score from the self-evaluation tool is strongly associated with perceived quality impact. Organizational culture also plays a significant role. When controlling for these factors, there was no evidence of benefit from a multispecialty review process. Physicians do not generally use reliable methods to measure clinical performance. A high rate of change since 2007 has not produced much improvement. The Peer Review Program Self-Evaluation Tool reliably differentiates hospitals along a continuum of perceived program performance. The full potential of peer review as a process to improve the quality and safety of care has yet to be realized.

Measuring the Impact of a New Faculty Program Using Institutional Data

ERIC Educational Resources Information Center

Meizlish, Deborah S.; Wright, Mary C.; Howard, Joseph; Kaplan, Matthew L.

2018-01-01

This paper presents a quasi-experimental evaluation of a required, teaching-focused, new faculty program at a large research university. The study makes use of institutional data, including student evaluations of teaching and faculty participation in educational development activities, which are available on many campuses yet rarely used in…

Social Validity of a Positive Behavior Interventions and Support Model

ERIC Educational Resources Information Center

Miramontes, Nancy Y.; Marchant, Michelle; Heath, Melissa Allen; Fischer, Lane

2011-01-01

As more schools turn to positive behavior interventions and support (PBIS) to address students' academic and behavioral problems, there is an increased need to adequately evaluate these programs for social relevance. The present study used social validation measures to evaluate a statewide PBIS initiative. Active consumers of the program were…

Detailed Test Plan Redundant Sensor Strapdown IMU Evaluation Program

NASA Technical Reports Server (NTRS)

Hartwell, T.; Miyatake, Y.; Wedekind, D. E.

1971-01-01

The test plan for a redundant sensor strapdown inertial measuring unit evaluation program is presented. The subjects discussed are: (1) test philosophy and limitations, (2) test sequence, (3) equipment specifications, (4) general operating procedures, (5) calibration procedures, (6) alignment test phase, and (7) navigation test phase. The data and analysis requirements are analyzed.

Applied Workplace Literacy for the Facilities Maintenance Industry. Performance Report and Third Party Evaluation Report.

ERIC Educational Resources Information Center

Texas State Technical Coll., Waco.

A project was conducted to determine if interactive video programs could produce positive results in literacy programs. During the project, staff from a technical college developed a task analysis, curriculum, and evaluation measures for the training of facilities maintenance workers in mathematical concepts. From this activity, an instructional…

Family Intervention Services Program Evaluation: A Brief Report on Initial Outcomes for Families.

ERIC Educational Resources Information Center

Cann, Warren; Rogers, Helen; Matthews, Jan

2003-01-01

Reports on a preliminary evaluation of the Metropolitan Family Intervention Service at the Victorian Parenting Centre, Melbourne, Australia. It presents an analysis of pre-post data collected from 589 mothers who commenced and completed parenting programs between 1999 and early 2003. Significant improvements were noted in measures of parental…

Removing a Nail from the Boot Camp Coffin: An Outcome Evaluation of Minnesota's Challenge Incarceration Program

ERIC Educational Resources Information Center

Duwe, Grant; Kerschner, Deborah

2008-01-01

Using a retrospective, quasiexperimental design, this study evaluates Minnesota's Challenge Incarceration Program (CIP), examining whether it has lowered recidivism and saved money. In addition to utilizing a lengthy follow-up period and multiple measures of recidivism and participation, a multistage sampling design was employed to create a…

Process Evaluation of a Parenting Program for Low-Income Families in South Africa

ERIC Educational Resources Information Center

Lachman, Jamie M.; Kelly, Jane; Cluver, Lucie; Ward, Catherine L.; Hutchings, Judy; Gardner, Frances

2018-01-01

Objective: This mixed-methods process evaluation examined the feasibility of a parenting program delivered by community facilitators to reduce the risk of child maltreatment in low-income families with children aged 3-8 years in Cape Town, South Africa (N = 68). Method: Quantitative measures included attendance registers, fidelity checklists,…

Evaluation of a Community-Based Parenting Program with the Parents of Young Children.

ERIC Educational Resources Information Center

Brenner, Viktor; Nicholson, Bonnie C.; Fox, Robert A.

1999-01-01

Evaluated effectiveness of a cognitive-behavioral parenting program for parents of children ages 1 to 5 years offered through community-based family resource centers. Found that participants showed significant decreases in use of verbal and corporal punishment, and increases in nurturing behaviors as measured by the Parent Behavior Checklist.…

An Evaluation of Sisters of Nia: A Cultural Program for African American Girls

ERIC Educational Resources Information Center

Belgrave, Faye Z.; Reed, Melba C.; Plybon, Laura E.; Butler, Deborah S.; Allison, Kevin W.; Davis, Trina

2004-01-01

In this study, the authors evaluated the effectiveness of a cultural intervention for increasing cultural values and beliefs. Fifty-nine African American girls in early adolescence participated in a 15-session cultural program or in an activity comparison group. Measures of ethnic identity, gender roles, and relational aggression were administered…

34 CFR 403.204 - What are the State's responsibilities for program evaluation and improvement?

Code of Federal Regulations, 2010 CFR

2010-07-01

... APPLIED TECHNOLOGY EDUCATION PROGRAM What Are the Administrative Responsibilities of a State Under the State Vocational and Applied Technology Education Program? § 403.204 What are the State's... description of vocational education strategies designed to improve the performance of the program as measured...

Measurement in Comparative Effectiveness Research

PubMed Central

Chubak, Jessica; Rutter, Carolyn M.; Kamineni, Aruna; Johnson, Eric A.; Stout, Natasha K.; Weiss, Noel S.; Doria-Rose, V. Paul; Doubeni, Chyke A.; Buist, Diana S.M.

2013-01-01

Comparative effectiveness research (CER) on preventive services can shape policy and help patients, their providers, and public health practitioners select regimens and programs for disease prevention. Patients and providers need information about the relative effectiveness of various regimens they may choose. Decision makers need information about the relative effectiveness of various programs to offer or recommend. The goal of this paper is to define and differentiate measures of relative effectiveness of regimens and programs for disease prevention. Cancer screening is used to demonstrate how these measures differ in an example of two hypothetic screening regimens and programs. Conceptually and algebraically defined measures of relative regimen and program effectiveness are also presented. The measures evaluate preventive services that range from individual tests through organized, population-wide prevention programs. Examples illustrate how effective screening regimens may not result in effective screening programs and how measures can vary across subgroups and settings. Both regimen and program relative effectiveness measures assess benefits of prevention services in real-world settings, but each addresses different scientific and policy questions. As the body of CER grows, a common lexicon for various measures of relative effectiveness becomes increasingly important to facilitate communication and shared understanding among researchers, healthcare providers, patients, and policymakers. PMID:23597816

Assessment and Evaluation of Primary Prevention in Spinal Cord Injury

PubMed Central

2013-01-01

Although the incidence of spinal cord injury (SCI) is low, the consequences of this disabling condition are extremely significant for the individual, family, and the community. Sequelae occur in the physical, psychosocial, sexual, and financial arenas, making global prevention of SCI crucial. Understanding how to assess and evaluate primary prevention programs is an important competency for SCI professionals. Assessing a program’s success requires measuring processes, outcomes, and impact. Effective evaluation can lead future efforts for program design while ensuring accountability for the program itself. The intended impact of primary prevention programs for SCI is to decrease the number of individuals who sustain traumatic injury; many programs have process and outcome goals as well. An understanding of the basic types of evaluation, evaluation design, and the overall process of program evaluation is essential for ensuring that these programs are efficacious. All health care professionals have the opportunity to put prevention at the forefront of their practice. With the current paucity of available data, it is important that clinicians share their program design, their successes, and their failures so that all can benefit and future injury can be prevented. PMID:23678281

Canadian Pacific Railway Investigation of Safety-Related Occurrences Protocol considered helpful by both labor and management.

DOT National Transportation Integrated Search

2006-09-01

The Federal Railroad Administration (FRA) Human Factors Research and Development (R&D) Program sponsored an Alternative Safety Measures Program designed to explore alternative methods for evaluating whether safety programs improve safety outcomes and...

Measurement of health-related quality of life in the national emphysema treatment trial.

PubMed

Kaplan, Robert M; Ries, Andrew L; Reilly, John; Mohsenifar, Zab

2004-09-01

To evaluate two generic and two disease-specific measures of health-related quality of life (QOL) using prerandomization data from the National Emphysema Treatment Trial (NETT). The analyses used data collected from the 1,218 subjects who were randomized in the NETT. Patients completed evaluations before and after completion of the prerandomization phase of the NETT pulmonary rehabilitation program. Using data obtained prior to participation in the rehabilitation program, QOL measures were evaluated against physiologic and functional criteria using correlational analysis. The physiologic criteria included estimates of emphysema severity based on FEV(1) and measures of Pao(2) obtained with the subject at rest and breathing room air. Functional measures included the 6-min walk distance (6MWD), maximum work, and hospitalizations in the prior 3 months. Correlation coefficients between QOL measures ranged from -0.31 to 0.70. In comparison to normative samples, scores on general QOL measures were low, suggesting that the NETT participants were quite ill. All QOL measures were modestly but significantly correlated with FEV(1), maximum work, and 6MWD. Patients who had stayed overnight in a hospital in the prior 3 months reported lower QOL on average than those who had not been hospitalized. There were significant improvements for all QOL measures following the rehabilitation program, and improvements in QOL were correlated with improvements in 6MWD. The disease-specific and general QOL measures used in the NETT were correlated. Analyses suggested that these measures improved significantly following the rehabilitation phase of the NETT.

Evaluation Report of Cooperative Urban Teacher Education Program. Fall 1969. Working Paper, Vol. 3 No. 1.

ERIC Educational Resources Information Center

Chan, James; Kingsley, Elizabeth

This report on the operation of the Cooperative Urban Teacher Education Program (CUTE) for the fall semester 1969 is divided into two parts. Part 1 examines the objective data in terms of the hypotheses: 1) On each of the measures taken during the semester, students participating in the program do not change. 2) On each of the measures taken,…

Theory and models for planning and evaluating institutional influenza prevention and control programs.

PubMed

Russell, Margaret L; Thurston, Wilfreda E; Henderson, Elizabeth A

2003-10-01

Low rates of staff influenza vaccine coverage occur in many health care facilities. Many programs do not offer vaccination to physicians or to volunteers, and some programs do not measure coverage or do so only for a subset of staff. The use of theory in planning and evaluation may prevent these problems and lead to more effective programs. We discuss the use of theory in the planning and evaluation of health programs and demonstrate how it can be used for the evaluation and planning of a hospital or nursing home influenza control program. The application of theory required explicit statement of the goals of the program and examination of the assumptions underlying potential program activities. This indicated that staff should probably be considered as employees, volunteers, physicians, and contractors of the facility. It also directed attention to evidence-based strategies for increasing vaccination rates. The application of a program planning model to a problem of institutional influenza prevention may prevent planners from excluding important target populations and failing to monitor the important indicators of program success.

Measurement and Design Issues in the Study of Adolescent Sexual Behavior and the Evaluation of Adolescent Sexual Health Behavior Interventions

ERIC Educational Resources Information Center

Young, Michael; Palacios, Rebecca; Penhollow, Tina M.

2012-01-01

To improve the quality of research and commentary concerning adolescent sexuality and evaluation of both comprehensive sexuality education and abstinence education programs, this article aims to help readers (1) select appropriate measures to study adolescent sexual behavior, (2) develop appropriate study designs to evaluate adolescent sexual…

Study protocol: evaluation of 'JenMe', a commercially-delivered weight management program for adolescents: a randomised controlled trial.

PubMed

Dordevic, Aimee L; Bonham, Maxine P; Ware, Robert S; Brennan, Leah; Truby, Helen

2015-06-19

Early lifestyle intervention with overweight and obese adolescents could help to avoid serious health events in early adulthood, ultimately alleviating some of the strain on the public health system due to obesity-related morbidity. Commercial weight loss programs have wide reach into the community setting, and have demonstrated success in long term weight management in adults, beyond that of current public health care. Commercial weight-management programs have not been evaluated as a method of delivery for overweight and obese adolescents. This study aims to evaluate the efficacy of a new adolescent weight management program in a commercial environment. One hundred and forty adolescents, 13 to 17 years old, will be randomised to either a weight management program intervention or a wait-listed group for 12 weeks. The commercial program will consist of a combined dietary and lifestyle approach targeting improved health behaviours for weight-loss or weight-stability. Participants will be overweight or obese (above the 85(th) percentile for BMI) and without existing co-morbidities. Outcome measures will be assessed at baseline and after 12 weeks. Primary outcome measures will be changes in BMI Z-score and waist-height ratio. Secondary outcome measures will include changes in behaviour, physical activity and psychosocial wellbeing. Intervention participants will be followed up at 6 months following completion of the initial program. Ethics approval has been granted from the Monash University Human Research Ethics Committee (CF11/3687-2011001940). This independent evaluation of a weight management program for adolescents, delivered in a commercial setting, will provide initial evidence for the effectiveness of such programs; which may offer adolescents an avenue of weight-management with ongoing support prior to the development of obesity related co-morbidities. The protocol for this study is registered with the International Clinical Trials Registry ISRCTN13602313.

Evaluation of thermal network correction program using test temperature data

NASA Technical Reports Server (NTRS)

Ishimoto, T.; Fink, L. C.

1972-01-01

An evaluation process to determine the accuracy of a computer program for thermal network correction is discussed. The evaluation is required since factors such as inaccuracies of temperatures, insufficient number of temperature points over a specified time period, lack of one-to-one correlation between temperature sensor and nodal locations, and incomplete temperature measurements are not present in the computer-generated information. The mathematical models used in the evaluation are those that describe a physical system composed of both a conventional and a heat pipe platform. A description of the models used, the results of the evaluation of the thermal network correction, and input instructions for the thermal network correction program are presented.

Connected Vehicle Pilot Deployment Program phase 1 : performance measurement and evaluation support plan : New York City : final report.

DOT National Transportation Integrated Search

2016-07-12

This document describes the Performance Measurement and Evaluation Support Plan for the New York City Department of Transportation New York City (NYC) Connected Vehicle Pilot Deployment (CVPD) Project. The report documents the performance metrics tha...

Proceedings of the 66th National Conference on Weights and Measures, 1981

NASA Astrophysics Data System (ADS)

Wollin, H. F.; Barbrow, L. E.; Heffernan, A. P.

1981-12-01

Major issues discussed included measurement science education, enforcement uniformly, national type approval, inch pound and metric labeling provisions, new design and performance requirements for weighing and measuring technology, metric conversion of retail gasoline dispensers, weights and measures program evaluation studies of model State laws and regulations and their adoption by citation or other means by State and local jurisdictions, and report of States conducting grain moisture meter testing programs.

Treatment effects model for assessing disease management: measuring outcomes and strengthening program management.

PubMed

Wendel, Jeanne; Dumitras, Diana

2005-06-01

This paper describes an analytical methodology for obtaining statistically unbiased outcomes estimates for programs in which participation decisions may be correlated with variables that impact outcomes. This methodology is particularly useful for intraorganizational program evaluations conducted for business purposes. In this situation, data is likely to be available for a population of managed care members who are eligible to participate in a disease management (DM) program, with some electing to participate while others eschew the opportunity. The most pragmatic analytical strategy for in-house evaluation of such programs is likely to be the pre-intervention/post-intervention design in which the control group consists of people who were invited to participate in the DM program, but declined the invitation. Regression estimates of program impacts may be statistically biased if factors that impact participation decisions are correlated with outcomes measures. This paper describes an econometric procedure, the Treatment Effects model, developed to produce statistically unbiased estimates of program impacts in this type of situation. Two equations are estimated to (a) estimate the impacts of patient characteristics on decisions to participate in the program, and then (b) use this information to produce a statistically unbiased estimate of the impact of program participation on outcomes. This methodology is well-established in economics and econometrics, but has not been widely applied in the DM outcomes measurement literature; hence, this paper focuses on one illustrative application.

A Survey to Evaluate Facilitators and Barriers to Quality Measurement and Improvement: Adapting Tools for Implementation Research in Palliative Care Programs.

PubMed

Dy, Sydney M; Al Hamayel, Nebras Abu; Hannum, Susan M; Sharma, Ritu; Isenberg, Sarina R; Kuchinad, Kamini; Zhu, Junya; Smith, Katherine; Lorenz, Karl A; Kamal, Arif H; Walling, Anne M; Weaver, Sallie J

2017-12-01

Although critical for improving patient outcomes, palliative care quality indicators are not yet widely used. Better understanding of facilitators and barriers to palliative care quality measurement and improvement might improve their use and program quality. Development of a survey tool to assess palliative care team perspectives on facilitators and barriers to quality measurement and improvement in palliative care programs. We used the adapted Consolidated Framework for Implementation Research to define domains and constructs to select instruments. We assembled a draft survey and assessed content validity through pilot testing and cognitive interviews with experts and frontline practitioners for key items. We analyzed responses using a constant comparative process to assess survey item issues and potential solutions. We developed a final survey using these results. The survey includes five published instruments and two additional item sets. Domains include organizational characteristics, individual and team characteristics, intervention characteristics, and process of implementation. Survey modules include Quality Improvement in Palliative Care, Implementing Quality Improvement in the Palliative Care Program, Teamwork and Communication, Measuring the Quality of Palliative Care, and Palliative Care Quality in Your Program. Key refinements from cognitive interviews included item wording on palliative care team members, programs, and quality issues. This novel, adaptable instrument assesses palliative care team perspectives on barriers and facilitators for quality measurement and improvement in palliative care programs. Next steps include evaluation of the survey's construct validity and how survey results correlate with findings from program quality initiatives. Copyright © 2017 American Academy of Hospice and Palliative Medicine. All rights reserved.

Evaluating an interdisciplinary undergraduate training program in health promotion research.

PubMed

Misra, Shalini; Harvey, Richard H; Stokols, Daniel; Pine, Kathleen H; Fuqua, Juliana; Shokair, Said M; Whiteley, John M

2009-04-01

The University of California at Irvine Interdisciplinary Summer Undergraduate Research Experience (ID-SURE) program had three objectives: (1) designing an interdisciplinary health promotion training curriculum for undergraduate research fellows; (2) developing measures for evaluating and assessing program-related educational processes and products; and (3) comparing these educational process and product measures between groups of students who did or did not receive the training. A total of 101 students participated in the ID-SURE program during 2005, 2006, and 2007. A longitudinal research design was employed whereby students' interdisciplinary attitudes and behaviors were assessed at the beginning and end of the training program. The interdisciplinary and intellectual qualities of students' academic and research products were assessed at the conclusion of the training activities. In addition, ID-SURE participants' interdisciplinary attitudes, behaviors, and research products were compared to those of 70 participants in another fellowship program that did not have an interdisciplinary training component. Exposing undergraduate research fellows to the interdisciplinary curriculum led to increased participation in, and positive attitudes about, interdisciplinary classroom and laboratory activities. Products, such as the integrative and interdisciplinary quality of student research projects, showed no differences when compared to those of undergraduates who were not exposed to the interdisciplinary curriculum. However, undergraduates exposed to the training engaged in more interdisciplinary behaviors at the end of the program than students who were not trained in interdisciplinary research techniques. The findings from this study offer evidence for the efficacy of the ID-SURE program for training undergraduate students in transdisciplinary concepts, methods, and skills that are needed for effective scientific collaboration. Additionally, this study makes two important contributions to the development and evaluation of interdisciplinary health research training programs: (1) It presents and evaluates a novel curriculum for training undergraduate students in interdisciplinary theories, concepts, and methods of health promotion that can be replicated in other settings and contexts; (2) It provides and tests the reliability of new measures for evaluating interdisciplinary collaborative processes and develops objective criteria for rating the integrative and intellectual quality of students' research products.

Conceptual framework for development of comprehensive e-health evaluation tool.

PubMed

Khoja, Shariq; Durrani, Hammad; Scott, Richard E; Sajwani, Afroz; Piryani, Usha

2013-01-01

The main objective of this study was to develop an e-health evaluation tool based on a conceptual framework including relevant theories for evaluating use of technology in health programs. This article presents the development of an evaluation framework for e-health programs. The study was divided into three stages: Stage 1 involved a detailed literature search of different theories and concepts on evaluation of e-health, Stage 2 plotted e-health theories to identify relevant themes, and Stage 3 developed a matrix of evaluation themes and stages of e-health programs. The framework identifies and defines different stages of e-health programs and then applies evaluation theories to each of these stages for development of the evaluation tool. This framework builds on existing theories of health and technology evaluation and presents a conceptual framework for developing an e-health evaluation tool to examine and measure different factors that play a definite role in the success of e-health programs. The framework on the horizontal axis divides e-health into different stages of program implementation, while the vertical axis identifies different themes and areas of consideration for e-health evaluation. The framework helps understand various aspects of e-health programs and their impact that require evaluation at different stages of the life cycle. The study led to the development of a new and comprehensive e-health evaluation tool, named the Khoja-Durrani-Scott Framework for e-Health Evaluation.

[Impact of interventions in the pressure ulcer rate].

PubMed

Araya Farías, I; Febré, N

To evaluate the impact of a risk management program for prevention of pressure ulcers (PUs) in an adult Intensive Care Unit (ICU). A quantitative, prospective study performed with a «before and after» evaluation, and designed in three stages: 1) PU incidence study; 2) Intervention by implementing a risk management program, and 3) Assessment of the impact. Adherence to the preventive measures showed a significant increase (11.7%) between the first month of the program and the final month (58.5%) of the assessment. Initial PU rate was 20.9, with a decrease in the rate to 14.0 per 1000 bed occupancy days (P<.05) after the risk management program. The data show that the risk management program, using prevention measures, was effective in reducing the rate of PU in the period under study by more than 33%. Copyright © 2017 SECA. Publicado por Elsevier España, S.L.U. All rights reserved.

The South Australia Health Chronic Disease Self-Management Internet Trial

ERIC Educational Resources Information Center

Lorig, Kate; Ritter, Philip L.; Plant, Kathryn; Laurent, Diana D.; Kelly, Pauline; Rowe, Sally

2013-01-01

Objectives: To evaluate the effectiveness of an online chronic disease self-management program for South Australia residents. Method: Data were collected online at baseline, 6 months, and 12 months. The intervention was an asynchronous 6-week chronic disease self-management program offered online. The authors measured eight health status measures,…

Background, College Experiences, and the ACT-COMP Exam: Using Construct Validity to Evaluate Assessment Instruments.

ERIC Educational Resources Information Center

Pike, Gary R.

1989-01-01

A study investigated the appropriateness of the American College Testing Program's College Outcome Measures Program, conducted at the University of Tennessee, Knoxville, by applying the criterion of construct validity. Results indicated that while the test primarily measures individual differences, it is also sensitive to the effects of higher…

Questioning the Use of Outcome Measures to Evaluate Principal Preparation Programs

ERIC Educational Resources Information Center

Fuller, Ed; Hollingworth, Liz

2018-01-01

Policymakers are proposing the use of outcome measures as indicators of effective principal preparation programs. The three most common metrics recommended are: (1) graduates' effectiveness in improving student achievement test scores, (2) graduate job placement rates, and (3) principal job retention once employed. This article explores the use of…

Fuels Management-How to Measure Success: Conference Proceedings

Treesearch

Patricia L. Andrews; Bret W. Butler

2006-01-01

Fuels management programs are designed to reduce risks to communities and to improve and maintain ecosystem health. The International Association of Wildland Fire initiated the 1st Fire Behavior and Fuels Conference to address development, implementation, and evaluation of these programs. The focus was on how to measure success. Over 500 participants from several...

The use of objective measurements in the evaluation of therapy programs

NASA Technical Reports Server (NTRS)

Chaplin, H.

1974-01-01

The importance of objective measurements is discussed as a means of assessing the efficacy of physical and occupational therapy programs applied to patients recovering from neurological diseases. Considered are three primary categories of neurologically injured patients: patients with hemiplegia, patients with spinal cord injuries, and the heterogeneous group of cerebral palsy patients.

Evaluating Performance Measurement Systems in Nonprofit Agencies: The Program Accountability Quality Scale (PAQS).

ERIC Educational Resources Information Center

Poole, Dennis L.; Nelson, Joan; Carnahan, Sharon; Chepenik, Nancy G.; Tubiak, Christine

2000-01-01

Developed and field tested the Performance Accountability Quality Scale (PAQS) on 191 program performance measurement systems developed by nonprofit agencies in central Florida. Preliminary findings indicate that the PAQS provides a structure for obtaining expert opinions based on a theory-driven model about the quality of proposed measurement…

Measuring the Quality of Early Childhood Programs--Guidelines for Effective Evaluation Tools.

ERIC Educational Resources Information Center

Epstein, Ann S.

2000-01-01

Summarizes what High/Scope discovered to be the critical characteristics of a comprehensive and valid measure of early childhood program quality. Provides suggestions for how the tool can be used, and highlights with examples. Asserts that the guidelines effectively assess efforts of child development, staff development, and soundness of…

Chapter 21: Estimating Net Savings - Common Practices. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W; Violette, Daniel M.; Rathbun, Pamela

This chapter focuses on the methods used to estimate net energy savings in evaluation, measurement, and verification (EM and V) studies for energy efficiency (EE) programs. The chapter provides a definition of net savings, which remains an unsettled topic both within the EE evaluation community and across the broader public policy evaluation community, particularly in the context of attribution of savings to a program. The chapter differs from the measure-specific Uniform Methods Project (UMP) chapters in both its approach and work product. Unlike other UMP resources that provide recommended protocols for determining gross energy savings, this chapter describes and comparesmore » the current industry practices for determining net energy savings but does not prescribe methods.« less

A Cost-Effectiveness Analysis Model for Evaluating and Planning Secondary Vocational Programs

ERIC Educational Resources Information Center

Kim, Jin Eun

1977-01-01

This paper conceptualizes a cost-effectiveness analysis and describes a cost-effectiveness analysis model for secondary vocational programs. It generates three kinds of cost-effectiveness measures: program effectiveness, cost efficiency, and cost-effectiveness and/or performance ratio. (Author)

The Faculty Learning Outcome Assessment Framework

ERIC Educational Resources Information Center

Hurney, Carol A.; Brantmeier, Edward J.; Good, Megan R.; Harrison, Douglas; Meixner, Cara

2016-01-01

Assessment is a cyclical process within which educators construct outcomes, implement programs, assess constructs such as learning, evaluate results, and utilize results to craft stronger programs and services. Within educational and faculty development, assessment measures program impact on faculty, students, and/or institutional culture.…

Impact Evaluation of an Addiction Intervention Program in a Quebec Prison

PubMed Central

Arseneault, Catherine; Alain, Marc; Plourde, Chantal; Ferland, Francine; Blanchette-Martin, Nadine; Rousseau, Michel

2015-01-01

OBJECTIVES This study evaluates the effects of a prison-based addiction intervention program. The evaluation is based on a multidimensional data collection that draws a portrait of the respondents’ substance use, and of their psychological/emotional, social, and judicial spheres. It measures the changes, or lack thereof, in substance use; the psychological/emotional, social, and judicial spheres; as well as the post-treatment services used. METHOD A quasi-experimental repeated measures design (0, 6 weeks, and 6 months) was used. Effects of the program were identified by comparing the results obtained by a group of inmates who had participated in the program (n = 80; experimental group) with those of another group who had received no intervention (n = 70; control group). RESULTS The preliminary results suggested a certain treatment effect related to impulsivity and psychological distress. CONCLUSION Although the preliminary results were promising, the experimental and control groups did not differ significantly when more robust analyses were used. PMID:26056466

Measuring Success in Obesity Prevention: A Synthesis of Health Promotion Switzerland's Long-Term Monitoring and Evaluation Strategy

PubMed Central

Ackermann, Günter; Kirschner, Michael; Guggenbühl, Lisa; Abel, Bettina; Klohn, Axel; Mattig, Thomas

2015-01-01

Aims Since 2007, Health Promotion Switzerland has implemented a national priority program for a healthy body weight. This article provides insight into the methodological challenges and results of the program evaluation. Methods Evaluation of the long-term program required targeted monitoring and evaluation projects addressing different outcome levels. The evaluation was carried out according to the Swiss Model for Outcome Classification (SMOC), a model designed to classify the effects of health promotion and prevention efforts. Results The results presented in this article emphasize both content and methods. The national program successfully achieved outcomes on many different levels within complex societal structures. The evaluation system built around the SMOC enabled assessment of program progress and the development of key indicators. However, it is not possible to determine definitively to what extent the national program helped stabilize the prevalence of obesity in Switzerland. Conclusion The model has shown its utility in providing a basis for evaluation and monitoring of the national program. Continuous analysis of data from evaluation and monitoring has made it possible to check the plausibility of suspected causal relationships as well as to establish an overall perspective and assessment of effectiveness supported by a growing body of evidence. PMID:25765161

Excursions in technology policy

NASA Technical Reports Server (NTRS)

Archibald, Robert B.

1995-01-01

This technical report presents a summary of three distinct projects: (1) Measuring economic benefits; (2) Evaluating the SBIR program; and (3) A model for evaluating changes in support for science and technology. the first project deals with the Technology Applications Group (TAG) at NASA Langley Research Center. The mission of TAG is to assist firms interested in commercializing technologies. TAG is a relatively new group as is the emphasis on technology commercialization for NASA. One problem faced by TAG and similar groups at other centers is measuring their effectiveness. The first project this summer, a paper entitled, 'Measuring the Economic Benefits of Technology Transfer from a National Laboratory: A Primer,' focused on this measurement problem. We found that the existing studies of the impact of technology transfer on the economy were conceptually flawed. The 'primer' outlines the appropriate theoretical framework for measuring the economic benefits of technology transfer. The second project discusses, one of the programs of TAG, the Small Business Innovation Research (SBIR) program. This program has led to over 400 contracts with Small Business since its inception in 1985. The program has never been evaluated. Crucial questions such as those about the extent of commercial successes from the contracts need to be answered. This summer we designed and implemented a performance evaluation survey instrument. The analysis of the data will take place in the fall. The discussion of the third project focuses on a model for evaluating changes in support for science and technology. At present several powerful forces are combining to change the environment for science and technology policy. The end of the cold war eliminated the rationale for federal support for many projects. The new- found Congressional conviction to balance the budget without tax increases combined with demographic changes which automatically increase spending for some politically popular programs will make it difficult to find funding for science and technology. Also, the two political parties have very different conceptions of the appropriate future for research and development spending. All these changes create the potential for serious, perhaps unintended, consequences for the economic future of the country. In a paper entitled, 'A Conceptual Framework for Evaluating the Impact of Changes in Federal Support for Science and Technology,' we introduce a model to evaluate the effects of changes in federal spending for science and technology. This paper both provides a way of organizing informed discussions and points out important research topics for science and technology policy.

Educational Program Audit Handbook. Revised.

ERIC Educational Resources Information Center

Cohen, Ruth I., Ed.; And Others

Our publics could be better served if verification of a program evaluation were made by an educational program auditor (EPA). The role of the EPA would be to enter the educational cycle after the objectives have been determined and before the programs and criterion measures have been adopted. The audit process would begin with the hiring of the…

An Evaluation of the Psychometric Properties of an Advising Survey for Medical and Professional Program Students

ERIC Educational Resources Information Center

Royal, Kenneth D.; Gonzalez, Liara M.

2016-01-01

The purpose of this study was to evaluate the psychometric properties of a newly developed instrument intended to measure faculty competence as it pertains to their role as advisors, particularly in medical and professional programs. A total of 166 students completed the Faculty Advisor's Skills and Behaviors Inventory (FASBI). The psychometric…

An Evaluation of Project iRead: A Program Created to Improve Sight Word Recognition

ERIC Educational Resources Information Center

Marshall, Theresa Meade

2014-01-01

This program evaluation was undertaken to examine the relationship between participation in Project iRead and student gains in word recognition, fluency, and comprehension as measured by the Phonological Awareness Literacy Screening (PALS) Test. Linear regressions compared the 2012-13 PALS results from 5,140 first and second grade students at…

Application of Difference-in-Difference Techniques to the Evaluation of Drought-Tainted Water Conservation Programs.

ERIC Educational Resources Information Center

Bamezai, Anil

1995-01-01

Some of the threats to internal validity that arise when evaluating the impact of water conservation programs during a drought are illustrated. These include differential response to the drought, self-selection bias, and measurement error. How to deal with these problems when high-quality disaggregate data are available is discussed. (SLD)

Teaching Grocery Store Purchasing Skills to Students with Intellectual Disabilities Using a Computer-Based Instruction Program

ERIC Educational Resources Information Center

Hansen, David L.; Morgan, Robert L.

2008-01-01

This research evaluated effects of a multi-media computer-based instruction (CBI) program designed to teach grocery store purchasing skills to three high-school students with intellectual disabilities. A multiple baseline design across participants used measures of computer performance mastery and grocery store probes to evaluate the CBI. All…

Employment Assistance Planning Programs for Welfare Recipients: An Evaluation and the Development of a Screening Measure.

ERIC Educational Resources Information Center

Peterson, Gary W.; And Others

A project was conducted to develop an assessment instrument for use in identifying welfare recipients who are potentially employable and to evaluate the effectiveness of Florida's three existing pilot employment assistance planning programs (termed PAPAs after the Public Assistance Productivity Act). Data on the employment status of 98 individuals…

A Meta-Analytic Review of the Penn Resiliency Program's Effect on Depressive Symptoms

ERIC Educational Resources Information Center

Brunwasser, Steven M.; Gillham, Jane E.; Kim, Eric S.

2009-01-01

The purpose of this review was to evaluate whether the Penn Resiliency Program (PRP), a group cognitive-behavioral intervention, is effective in targeting depressive symptoms in youths. We identified 17 controlled evaluations of PRP (N = 2,498) in which depressive symptoms had been measured via an online search of PsycInfo, Medline, ERIC, and…

APPLICATION OF THE SPERM CHROMATIN STRUCTURE ASSAY TO THE TEPLICE PROGRAM SEMEN STUDIES: A NEW METHOD FOR EVALUATING SPERM NUCLEAR CHROMATIN DAMAGE

EPA Science Inventory

ABSTRACT
A measure of sperm chromatin integrity was added to the routine semen end points evaluated in the Teplice Program male reproductive health studies. To address the hypothesis that exposure to periods of elevated air pollution may be associated with abnormalities in sp...

Use of Interactive Electronic Audience Response Tools (Clickers) to Evaluate Knowledge Gained in Extension Programming

ERIC Educational Resources Information Center

Gunn, Patrick; Loy, Dan

2015-01-01

Effectively measuring short-term impact, particularly a change in knowledge resulting from Extension programming, can prove to be challenging. Clicker-based technology, when used properly, is one alternative that may allow educators to better evaluate this aspect of the logic model. While the potential interface between clicker technology and…

Measuring the Impacts of a Volunteer-Based Community Development Program in Developing Volunteers' Leadership Skills

ERIC Educational Resources Information Center

Meier, Amy; Singletary, Loretta; Hill, George

2012-01-01

This article summarizes the results of an evaluation of the impacts of a community development program to develop leadership skills in its adult volunteers. The evaluation featured 20 questions about leadership skills learned as a result of volunteer experiences. Data analysis strategies beyond a simple means ranking resulted in evidence…

Making Room for Foster Children: A Criterion-Referenced Approach to ESEA Title I Migrant Program Evaluation.

ERIC Educational Resources Information Center

Oxford, Rebecca L.; And Others

The Washington state Title I Migrant Program Evaluation project is a feasibility study designed to assess the suitability of existing normed criterion referenced tests to measuring mathematics achievement at grades four, five, and six. Objectives include judging the technical qualities and content of several normed criterion referenced tests;…

Vouchers for family planning and sexual and reproductive health services: a review of voucher programs involving Marie Stopes International among 11 Asian and African countries.

PubMed

Eva, Gillian; Quinn, Andrew; Ngo, Thoai D

2015-08-01

To evaluate provision of vouchers for family planning and sexual and reproductive health (SRH) services. A review was conducted to assess the effects of 24 voucher programs in Marie Stopes International programs across 11 countries in Asia and Africa between 2005 and the present. The outcome measures were uptake of services; service use among specific subgroups; user satisfaction with service quality; and efficiency of service delivery. Twelve of the 24 programs covered family planning only, whereas the other 12 programs covered family planning and/or SRH. Service uptake increased following implementation, although voucher redemption rates varied by program (44.1%-92.4%). Most programs were successful in reaching subgroups, such as the poor and young (under 25years), although this outcome depended on the targeting approach. Most programs recorded high user satisfaction; however, the evidence regarding efficiency was mixed. Vouchers increased uptake of services and, in some cases, improved service quality and reach to specific groups. Nevertheless, robust evaluation designs are required to measure efficiency. Copyright © 2015. Published by Elsevier Ireland Ltd.

Aluminum Data Measurements and Evaluation for Criticality Safety Applications

NASA Astrophysics Data System (ADS)

Leal, L. C.; Guber, K. H.; Spencer, R. R.; Derrien, H.; Wright, R. Q.

2002-12-01

The Defense Nuclear Facility Safety Board (DNFSB) Recommendation 93-2 motivated the US Department of Energy (DOE) to develop a comprehensive criticality safety program to maintain and to predict the criticality of systems throughout the DOE complex. To implement the response to the DNFSB Recommendation 93-2, a Nuclear Criticality Safety Program (NCSP) was created including the following tasks: Critical Experiments, Criticality Benchmarks, Training, Analytical Methods, and Nuclear Data. The Nuclear Data portion of the NCSP consists of a variety of differential measurements performed at the Oak Ridge Electron Linear Accelerator (ORELA) at the Oak Ridge National Laboratory (ORNL), data analysis and evaluation using the generalized least-squares fitting code SAMMY in the resolved, unresolved, and high energy ranges, and the development and benchmark testing of complete evaluations for a nuclide for inclusion into the Evaluated Nuclear Data File (ENDF/B). This paper outlines the work performed at ORNL to measure, evaluate, and test the nuclear data for aluminum for applications in criticality safety problems.

Measuring test productivity - The elusive dream

NASA Astrophysics Data System (ADS)

Ward, D. T.; Cross, E. J., Jr.

1983-11-01

The paper summarizes definitions and terminology relating to measurement of Test and Evaluation productivity before settling on the appropriate criteria for such a measurement model. A productivity measurement scheme suited for use by Test and Evaluation organizations is suggested. This mathematical model is a simplified version of one proposed by the American Productivity Center and applied to an aircraft maintenance facility by Fletcher. It includes only four primary variables: safety, schedule, cost, and deficiencies reported with varying degrees of objectivity and subjectivity involved in quantifying them. A hypothetical example of a fighter aircraft flight test program is used to illustrate the application of the productivity measurement model. The proposed model is intended to serve as a first iteration procedure and should be tested against real test programs to verify and refine it.

Overview of the Arizona Quiet Pavement Program

NASA Astrophysics Data System (ADS)

Donavan, Paul; Scofield, Larry

2005-09-01

The Arizona Quiet Pavement Pilot Program (QP3) was initially implemented to reduce highway related traffic noise by overlaying most of the Phoenix metropolitan area Portland cement concrete pavement with a one inch thick asphalt rubber friction coarse. With FHWA support, this program represents the first time that pavement surface type has been allowed as a noise mitigation strategy on federally funded projects. As a condition of using pavement type as a noise mitigation strategy, ADOT developed a ten-year, $3.8 million research program to evaluate the noise reduction performance over time. Historically, pavement surface type was not considered a permanent solution. As a result, the research program was designed to specifically address this issue. Noise performance is being evaluated through three means: (1) conventional roadside testing within the roadway corridor (e.g., far field measurements within the right-of-way) (2) the use of near field measurements, both close proximity (CPX) and sound intensity (SI); and (3) far field measurements obtained beyond the noise barriers within the surrounding neighborhoods. This paper provides an overview of the program development, presents the research conducted to support the decision to overlay the urban freeway, and the status of current research.

Methods for the evaluation of the Jamie Oliver Ministry of Food program, Australia

PubMed Central

2013-01-01

Background Community-based programs aimed at improving cooking skills, cooking confidence and individual eating behaviours have grown in number over the past two decades. Whilst some evidence exists to support their effectiveness, only small behavioural changes have been reported and limitations in study design may have impacted on results. This paper describes the first evaluation of the Jamie Oliver Ministry of Food Program (JMoF) Australia, in Ipswich, Queensland. JMoF Australia is a community-based cooking skills program open to the general public consisting of 1.5 hour classes weekly over a 10 week period, based on the program of the same name originating in the United Kingdom. Methods/Design A mixed methods study design is proposed. Given the programmatic implementation of JMoF in Ipswich, the quantitative study is a non-randomised, pre-post design comparing participants undergoing the program with a wait-list control group. There will be two primary outcome measures: (i) change in cooking confidence (self-efficacy) and (ii) change in self-reported mean vegetable intake (serves per day). Secondary outcome measures will include change in individual cooking and eating behaviours and psycho-social measures such as social connectedness and self-esteem. Repeated measures will be collected at baseline, program completion (10 weeks) and 6 months follow up from program completion. A sample of 250 participants per group will be recruited for the evaluation to detect a mean change of 0.5 serves a day of vegetables at 80% power (0.5% significance level). Data analysis will assess the magnitude of change of these variables both within and between groups and use sub group analysis to explore the relationships between socio-demographic characteristics and outcomes. The qualitative study will be a longitudinal design consisting of semi-structured interviews with approximately 10-15 participants conducted at successive time points. An inductive thematic analysis will be conducted to explore social, attitudinal and behavioural changes experienced by program participants. Discussion This evaluation will contribute to the evidence of whether cooking programs work in terms of improving health and wellbeing and the underlying mechanisms which may lead to positive behaviour change. Trial registration Australian and New Zealand Trial registration number: ACTRN12611001209987. PMID:23631683

Curriculum-Based Measurement of Oral Reading: Multi-Study Evaluation of Schedule, Duration, and Dataset Quality on Progress Monitoring Outcomes

ERIC Educational Resources Information Center

Christ, Theodore J.; Zopluoglu, Cengiz; Monaghen, Barbara D.; Van Norman, Ethan R.

2013-01-01

Curriculum-Based Measurement of Oral Reading (CBM-R) is used to collect time series data, estimate the rate of student achievement, and evaluate program effectiveness. A series of 5 studies were carried out to evaluate the validity, reliability, precision, and diagnostic accuracy of progress monitoring across a variety of progress monitoring…

Management Data Base Development.

ERIC Educational Resources Information Center

Dan, Robert L.

1975-01-01

A management data base is seen as essential for a management information system, program budgeting, program costing, management by objectives, program evaluation, productivity measures, and accountability in institutions of higher education. The necessity of a management data base is addressed, along with the benefits and limitations it may have…

SITE TECHNOLOGY PROFILES, TENTH EDITION, VOLUME 3 - MEASUREMENT AND MONITORING PROGRAM

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its thirteenth year, is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine us...

Assessment Study of an Undergraduate Research Training Abroad Program

ERIC Educational Resources Information Center

Nieto-Fernandez, Fernando; Race, Kathryn; Quarless, Duncan A.

2013-01-01

The Old Westbury Neuroscience International Research Program (OWNIP) encourages undergraduate students from health disparities populations and underrepresented minorities to pursue careers in basic science, biomedical, clinical, and behavioral health research fields. To evaluate this program, several measures were used tracked through an online…

PREFACE: SPECIAL ISSUE OF ATMOSPHERIC ENVIRONMENT ON FINDINGS FROM EPA'S PARTICULATE MATTER SUPERSITES PROGRAM

EPA Science Inventory

The Supersites Program is a methods development and evaluation, measurements, modeling, and data analysis program designed to provide key stakeholders in the science, regulatory, and policy communities with information to support primarily implementation of National Ambient Air Q...

Computer program determines performance efficiency of remote measuring systems

NASA Technical Reports Server (NTRS)

Merewether, E. K.

1966-01-01

Computer programs control and evaluate instrumentation system performance for numerous rocket engine test facilities and prescribe calibration and maintenance techniques to maintain the systems within process specifications. Similar programs can be written for other test equipment in an industry such as the petrochemical industry.

Experimental evaluation of sensorimotor patterning used with mentally retarded children.

PubMed

Neman, R; Roos, P; McCann, R M; Menolascino, F J; Heal, L W

1975-01-01

In the present study, a sensorimotor "patterning" program used with 66 institutionalized, mentally retarded children and adolescents was evaluated. The subjects were randomly assigned to one of three groups: (a) Experimental 1 group, which received a program of mobility exercises including patterning, creeping, and crawling; visual-motor training; and sensory stimulation exercises; (b) Experimental 2 group, which received a program of physical activity, personal attention, and the same sensory stimulation program given to the first group; or (c) Passive Control group, which provided baseline measures but which received no additional programming as part of the study. Experimental 1 group subjects improved more than subjects in the other groups in visual perception, program-related measures of mobility, and language ability. Intellectual functioning did not appear to be enhanced by the procedures, at least during the active phase of the project. The results were discussed with reference to other researchers who have failed to support the patterning approach, and some reasons were suggested for the differences between the present and past investigations.

Urban Rail Supporting Technology Program Fiscal Year 1974 Year End Summary

DOT National Transportation Integrated Search

1975-03-01

Major areas include program management, technical support and application engineering, facilities development, test and evaluation, and technology development. Specific technical discussion includes track measurement systems; UMTA facilities developm...

Evaluation of a Five-Year Demonstration Farm Program in Two Pennsylvania Counties.

ERIC Educational Resources Information Center

Hartman, Joel A.; Brown, Emory J.

This analysis of the impact of a result demonstration educational program was made in an effort to achieve the following goals: (1) identification of the objectives of the educational program; (2) description of the program; (3) assessment of awareness of the program by target clientele; (4) measurement of the amount of change that takes place;…

Detecting selection effects in community implementations of family-based substance abuse prevention programs.

PubMed

Hill, Laura G; Goates, Scott G; Rosenman, Robert

2010-04-01

To calculate valid estimates of the costs and benefits of substance abuse prevention programs, selection effects must be identified and corrected. A supplemental comparison sample is typically used for this purpose, but in community-based program implementations, such a sample is often not available. We present an evaluation design and analytic approach that can be used in program evaluations of real-world implementations to identify selection effects, which in turn can help inform recruitment strategies, pinpoint possible selection influences on measured program outcomes, and refine estimates of program costs and benefits. We illustrate our approach with data from a multisite implementation of a popular substance abuse prevention program. Our results indicate that the program's participants differed significantly from the population at large.

Conceptual Soundness, Metric Development, Benchmarking, and Targeting for PATH Subprogram Evaluation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mosey. G.; Doris, E.; Coggeshall, C.

The objective of this study is to evaluate the conceptual soundness of the U.S. Department of Housing and Urban Development (HUD) Partnership for Advancing Technology in Housing (PATH) program's revised goals and establish and apply a framework to identify and recommend metrics that are the most useful for measuring PATH's progress. This report provides an evaluative review of PATH's revised goals, outlines a structured method for identifying and selecting metrics, proposes metrics and benchmarks for a sampling of individual PATH programs, and discusses other metrics that potentially could be developed that may add value to the evaluation process. The frameworkmore » and individual program metrics can be used for ongoing management improvement efforts and to inform broader program-level metrics for government reporting requirements.« less

The development and validation of The Inquiry Science Observation Coding Sheet.

PubMed

Brandon, P R; Taum, A K H; Young, D B; Pottenger, F M

2008-08-01

Evaluation reports increasingly document the degree of program implementation, particularly the extent to which programs adhere to prescribed steps and procedures. Many reports are cursory, however, and few, if any, fully portray the long and winding path taken when developing evaluation instruments, particularly observation instruments. In this article, we describe the development of an observational method for evaluating the degree to which K-12 inquiry science programs are implemented, including the many steps and decisions that occurred during the development, and present evidence for the reliability and validity of the data that we collected with the instrument. The article introduces a method for measuring the adherence of inquiry science implementation and gives evaluators a full picture of what they might expect when developing observation instruments for assessing the degree of program implementation.

Cost-effectiveness of an exercise program during pregnancy to prevent gestational diabetes: results of an economic evaluation alongside a randomised controlled trial.

PubMed

Oostdam, Nicolette; Bosmans, Judith; Wouters, Maurice G A J; Eekhoff, Elisabeth M W; van Mechelen, Willem; van Poppel, Mireille N M

2012-07-04

The prevalence of gestational diabetes mellitus (GDM) is increasing worldwide. GDM and the risks associated with GDM lead to increased health care costs and losses in productivity. The objective of this study is to evaluate whether the FitFor2 exercise program during pregnancy is cost-effective from a societal perspective as compared to standard care. A randomised controlled trial (RCT) and simultaneous economic evaluation of the FitFor2 program were conducted. Pregnant women at risk for GDM were randomised to an exercise program to prevent high maternal blood glucose (n = 62) or to standard care (n = 59). The exercise program consisted of two sessions of aerobic and strengthening exercises per week. Clinical outcome measures were maternal fasting blood glucose levels, insulin sensitivity and infant birth weight. Quality of life was measured using the EuroQol 5-D and quality-adjusted life-years (QALYs) were calculated. Resource utilization and sick leave data were collected by questionnaires. Data were analysed according to the intention-to-treat principle. Missing data were imputed using multiple imputations. Bootstrapping techniques estimated the uncertainty surrounding the cost differences and incremental cost-effectiveness ratios. There were no statistically significant differences in any outcome measure. During pregnancy, total health care costs and costs of productivity losses were statistically non-significant (mean difference €1308; 95%CI €-229 - €3204). The cost-effectiveness analyses showed that the exercise program was not cost-effective in comparison to the control group for blood glucose levels, insulin sensitivity, infant birth weight or QALYs. The twice-weekly exercise program for pregnant women at risk for GDM evaluated in the present study was not cost-effective compared to standard care. Based on these results, implementation of this exercise program for the prevention of GDM cannot be recommended. NTR1139.

Instruments and Scoring Guide of the Experiential Education Evaluation Project.

ERIC Educational Resources Information Center

Conrad, Dan; Hedin, Diane

As a result of the Experiential Education Evaluation Project the publication identifies instruments used to measure and assess experiential learning programs. The following information is given for each instrument: rationale for its inclusion in the study; precise issues or outcomes designed to measure, validity and reliability data; and…

Evaluating Math Recovery: Measuring Fidelity of Implementation

ERIC Educational Resources Information Center

Munter, Charles; Garrison, Anne; Cobb, Paul; Cordray, David

2010-01-01

In this paper, the authors describe a case of measuring implementation fidelity within an evaluation study of Math Recovery (MR), a pullout tutoring program aimed at increasing the mathematics achievement of low-performing first graders, thereby closing the school-entry achievement gap by enabling them to achieve at the level of their…

A worksite prevention program for construction workers: design of a randomized controlled trial.

PubMed

Oude Hengel, Karen M; Joling, Catelijne I; Proper, Karin I; Blatter, Birgitte M; Bongers, Paulien M

2010-06-14

A worksite prevention program was developed to promote the work ability of construction workers and thereby prolong a healthy working life. The objective of this paper is to present the design of a randomized controlled trial evaluating the effectiveness of that intervention program compared with usual care for construction workers. The study is designed as a randomized controlled trial with a follow-up of one year. Employees eligible for this study are construction workers performing actual construction work. The worksite intervention will be compared with usual care. This intervention was developed by using the Intervention Mapping approach and consists of the following components: (1) two individual training sessions of a physical therapist to lower the physical workload, (2) a Rest-Break tool to improve the balance between work and recovery, and (3) two empowerment training sessions to increase the influence of the construction workers at the worksite. Outcome measures are assessed at baseline, 3, 6, and 12 months. The primary outcome measures of this study are work ability and health-related quality of life. Secondary outcome measures include need for recovery, musculoskeletal complaints, work engagement and self efficacy. Cost-effectiveness will be evaluated from the company perspective. Moreover, a process evaluation will be conducted. The feasibility of the intervention and the study has been enhanced by creating an intervention program that explicitly appeals to construction workers and will not interfere too much with the ongoing construction. The feasibility and effectiveness of this worksite prevention program will be investigated by means of an effect- and a process evaluation. If proven effective, this worksite prevention program can be implemented on a larger scale within the construction industry. NTR1278.

A worksite prevention program for construction workers: design of a randomized controlled trial

PubMed Central

2010-01-01

Background A worksite prevention program was developed to promote the work ability of construction workers and thereby prolong a healthy working life. The objective of this paper is to present the design of a randomized controlled trial evaluating the effectiveness of that intervention program compared with usual care for construction workers. Methods The study is designed as a randomized controlled trial with a follow-up of one year. Employees eligible for this study are construction workers performing actual construction work. The worksite intervention will be compared with usual care. This intervention was developed by using the Intervention Mapping approach and consists of the following components: (1) two individual training sessions of a physical therapist to lower the physical workload, (2) a Rest-Break tool to improve the balance between work and recovery, and (3) two empowerment training sessions to increase the influence of the construction workers at the worksite. Outcome measures are assessed at baseline, 3, 6, and 12 months. The primary outcome measures of this study are work ability and health-related quality of life. Secondary outcome measures include need for recovery, musculoskeletal complaints, work engagement and self efficacy. Cost-effectiveness will be evaluated from the company perspective. Moreover, a process evaluation will be conducted. Discussion The feasibility of the intervention and the study has been enhanced by creating an intervention program that explicitly appeals to construction workers and will not interfere too much with the ongoing construction. The feasibility and effectiveness of this worksite prevention program will be investigated by means of an effect- and a process evaluation. If proven effective, this worksite prevention program can be implemented on a larger scale within the construction industry. Trial Registration NTR1278 PMID:20546568

Psychometric Characteristics of Process Evaluation Measures for a Rural School-based Childhood Obesity Prevention Study: Louisiana Health

PubMed Central

Newton, R. L.; Thomson, J. L.; Rau, K.; Duhe’, S.; Sample, A.; Singleton, N.; Anton, S. D.; Webber, L. S.; Williamson, D. A.

2011-01-01

Purpose To evaluate the implementation of intervention components of the Louisiana Health study, which was a multi-component childhood obesity prevention program conducted in rural schools. Design Content analysis. Setting Process evaluation assessed implementation in the classrooms, gym classes, and cafeterias. Subjects Classroom teachers (n = 232), physical education teachers (n = 53), food service managers (n = 33), and trained observers (n = 9). Measures Five process evaluation measures were created: Physical Education Questionnaire (PEQ), Intervention Questionnaire (IQ), Food Service Manager Questionnaire (FSMQ), Classroom Observation (CO) and School Nutrition Environment Observation (SNEO). Analysis Inter-rater reliability and internal consistency were conducted on all measures. ANOVA and Chi-square were used to compare differences across study groups on questionnaires and observations. Results The PEQ and one sub-scale from the FSMQ were eliminated because their reliability coefficients fell below acceptable standards. The sub-scale internal consistencies for the IQ, FSMQ, CO, and SNEO (all Cronbach’s α > .60) were acceptable. Conclusions After the initial 4 months of intervention, there was evidence that the Louisiana Health intervention was being implemented as it was designed. In summary, four process evaluation measures were found to be sufficiently reliable and valid for assessing the delivery of various aspects of a school-based obesity prevention program. These process measures could be modified to evaluate the delivery of other similar school-based interventions. PMID:21721969

Evaluator and Program Manager Perceptions of Evaluation Capacity and Evaluation Practice

ERIC Educational Resources Information Center

Fierro, Leslie A.; Christie, Christina A.

2017-01-01

The evaluation community has demonstrated an increased emphasis and interest in evaluation capacity building in recent years. A need currently exists to better understand how to measure evaluation capacity and its potential outcomes. In this study, we distributed an online questionnaire to managers and evaluation points of contact working in…

Measurement in comparative effectiveness research.

PubMed

Chubak, Jessica; Rutter, Carolyn M; Kamineni, Aruna; Johnson, Eric A; Stout, Natasha K; Weiss, Noel S; Doria-Rose, V Paul; Doubeni, Chyke A; Buist, Diana S M

2013-05-01

Comparative effectiveness research (CER) on preventive services can shape policy and help patients, their providers, and public health practitioners select regimens and programs for disease prevention. Patients and providers need information about the relative effectiveness of various regimens they may choose. Decision makers need information about the relative effectiveness of various programs to offer or recommend. The goal of this paper is to define and differentiate measures of relative effectiveness of regimens and programs for disease prevention. Cancer screening is used to demonstrate how these measures differ in an example of two hypothetical screening regimens and programs. Conceptually and algebraically defined measures of relative regimen and program effectiveness also are presented. The measures evaluate preventive services that range from individual tests through organized, population-wide prevention programs. Examples illustrate how effective screening regimens may not result in effective screening programs and how measures can vary across subgroups and settings. Both regimen and program relative effectiveness measures assess benefits of prevention services in real-world settings, but each addresses different scientific and policy questions. As the body of CER grows, a common lexicon for various measures of relative effectiveness becomes increasingly important to facilitate communication and shared understanding among researchers, healthcare providers, patients, and policymakers. Copyright © 2013 American Journal of Preventive Medicine. All rights reserved.

Evaluating a nursing communication skills training course: The relationships between self-rated ability, satisfaction, and actual performance.

PubMed

Mullan, Barbara A; Kothe, Emily J

2010-11-01

Effective communication is a vital component of nursing care, however, nurses often lack the skills to communicate with patients, carers and other health care professionals. Communication skills training programs are frequently used to develop these skills. However, there is a paucity of data on how best to evaluate such courses. The aim of the current study was to evaluate the relationship between student self rating of their own ability and their satisfaction with a nurse training course as compared with an objective measure of communication skills. 209 first year nursing students completed a communication skills program. Both qualitative and quantitative data were collected and associations between measures were investigated. Paired samples t-tests showed significant improvement in self-rated ability over the course of the program. Students generally were very satisfied with the course which was reflected in both qualitative and quantitative measures. However, neither self-rated ability nor satisfaction was significantly correlated with the objective measure of performance, but self-rated ability and satisfaction were highly correlated with one another. The importance of these findings is discussed and implications for nurse education are proposed. Copyright © 2010 Elsevier Ltd. All rights reserved.

Framework for a National Testing and Evaluation Program ...

EPA Pesticide Factsheets

Abstract:The National STEPP Program seeks to improve water quality by accelerating the effective implementation and adoption of innovative stormwater management technologies. Itwill attempt to accomplish this by establishing practices through highly reliable, and cost-effective Stormwater control measures (SCM) testing, evaluation, and verification services. The program will aim to remove barriers to innovation, minimize duplicative performance evaluation needs, increase confidence that regulatory requirements are met by creating consistency among testing and evaluation protocols, and establishing equity between public domain and proprietary SCM evaluation approaches.The Environmental Technology Verification Program, established by the U.S. Environmental Protection Agency (EPA) 18 years ago, was the only national program of its kindin the stormwater sector, but is now defunct, leaving a national leadership void. The STEPP initiative was triggered in part by regulatory demands in the government and private sectors to fill this vacuum. A concerted focus and study of this matter led to the release of a Water Environment Federation (WEF) white paper entitled “Investigation into the Feasibility of a National Testing and Evaluation Program for Stormwater Products and Practices” in February 2014. During this second phase of the STEPP initiative, and with EPA support, five analogous technology evaluation programs related to both stormwater and non-stormwater were an

Evaluating an employee wellness program.

PubMed

Mukhopadhyay, Sankar; Wendel, Jeanne

2013-12-01

What criteria should be used to evaluate the impact of a new employee wellness program when the initial vendor contract expires? Published academic literature focuses on return-on-investment as the gold standard for wellness program evaluation, and a recent meta-analysis concludes that wellness programs can generate net savings after one or two years. In contrast, surveys indicate that fewer than half of these programs report net savings, and actuarial analysts argue that return-on-investment is an unrealistic metric for evaluating new programs. These analysts argue that evaluation of new programs should focus on contract management issues, such as the vendor's ability to: (i) recruit employees to participate and (ii) induce behavior change. We compute difference-in-difference propensity score matching estimates of the impact of a wellness program implemented by a mid-sized employer. The analysis includes one year of pre-implementation data and three years of post-implementation data. We find that the program successfully recruited a broad spectrum of employees to participate, and it successfully induced short-term behavior change, as manifested by increased preventive screening. However, the effects on health care expenditures are positive (but insignificant). If it is unrealistic to expect new programs to significantly reduce healthcare costs in a few years, then focusing on return-on-investment as the gold standard metric may lead to early termination of potentially useful wellness programs. Focusing short-term analysis of new programs on short-term measures may provide a more realistic evaluation strategy.

Evaluation of Learning Environments for Object-Oriented Programming: Measuring Cognitive Load with a Novel Measurement Technique

ERIC Educational Resources Information Center

Uysal, Murat Pasa

2016-01-01

Various methods and tools have been proposed to overcome the learning obstacles for Object-Oriented Programming (OOP). However, it remains difficult especially for novice learners. The problem may be not only adopting an instructional method, but also an Integrated Development Environment (IDE). Learners employ IDEs as a means to solve programming…

Multiple Measures of Outcome in Assessing a Prison-Based Drug Treatment Program

ERIC Educational Resources Information Center

Prendergast, Michael L.; Hall, Elizabeth A.; Wexler, Harry K.

2003-01-01

Evaluations of prison-based drug treatment programs typically focus on one or two dichotomous outcome variables related to recidivism. In contrast, this paper uses multiple measures of outcomes related to crime and drug use to examine the impact of prison treatment. Crime variables included self-report data of time to first illegal activity,…

AAFE RADSCAT data reduction programs user's guide

NASA Technical Reports Server (NTRS)

Claassen, J. P.

1976-01-01

Theory, design and operation of the computer programs which automate the reduction of joint radiometer and scatterometer observations are presented. The programs reduce scatterometer measurements to the normalized scattering coefficient; whereas the radiometer measurements are converted into antenna temperatures. The programs are both investigator and user oriented. Supplementary parameters are provided to aid in the interpretation of the observations. A hierarchy of diagnostics is available to evaluate the operation of the instrument, the conduct of the experiments and the quality of the records. General descriptions of the programs and their data products are also presented. This document therefore serves as a user's guide to the programs and is therefore intended to serve both the experimenter and the program operator.

Performance Evaluation of Community Health Workers: Case Study in the Amazon of Peru.

PubMed

Westgard, Christopher; Naraine, Renuka; Paucar Villacorta, Diego Mauricio

2018-03-26

A shortage in human resources for health is a growing crisis that has led to an inability to provide adequate health services to impoverished populations. By "task-shifting", health systems can delegate certain activities, such as health promotion and referral, to trained community members to help fill the human resource gap. An effective community health agent program can improve maternal and child health and overall effectiveness of rural health systems. Such a program is most effective when the community health agents receive supervision and evaluation of their performance. There is a shortage of literature that provides instruction and example on how to conduct a performance evaluation in the developing world to improve maternal and child health outcomes. The current study provides a case study of a performance evaluation in the Amazon region of Peru and how the findings can be used to make program adjustments. A set of instruments to measure the performance of CHWs was adapted from the literature and then implemented in the field. The instruments were used to measure the quality of home visits by the CHWs, their knowledge of the health topics, and structural activities. Three communities with an active CHW program in Loreto, Peru were chosen to receive the evaluation. All CHWs in the communities were evaluated. The scores from the evaluation were compared internally to identify strengths and weaknesses of the program and within the population of CHWs. The evaluation was completed on 52 home visits and 27 CHWs in three communities. The CHWs were found to be most effective at creating good relationships with caregivers and delivering health messages, and least effective at interacting with the child during the home visit and using material to deliver health messages. The evaluation instruments were well suited for the CHW program that utilizes home visits to teach about child health and development.

SITE TECHNOLOGY PROFILES - 11TH EDITION, MEASUREMENT AND MONITORING PROGRAM, VOLUME 3

EPA Science Inventory

The Superfund Innovative Technology Evaluation (SITE) Program, now in its eleventh year is an integral part of EPA's research into alternative cleanup methods for hazardous waste sites around the nation. The SITE Program was created to encourage the development and routine use o...

Evaluation of an employer-sponsored pedometer-based physical activity program.

PubMed

Behrens, Timothy K; Domina, Lorie; Fletcher, Gena M

2007-12-01

This study evaluated a competition-based employer-sponsored physical activity program using pedometers. City employees (N approximately 2,600) formed teams in groups of 10 persons (N = 640). The groups competed against each other over a 12-wk. period with the goal of attaining 10,000 steps per day, per person. Only teams with complete data were included in the formal evaluation. A repeated-measures analysis of variance indicated significant differences in team steps by week of the program, with post hoc comparisons indicating significant differences from baseline step counts evident during Weeks 6-8 but not at the end of the program. These data confirm that competition-based physical activity programs using pedometers may not be an effective means of increasing the long-term physical activity of employees.

Predicting Voice Disorder Status From Smoothed Measures of Cepstral Peak Prominence Using Praat and Analysis of Dysphonia in Speech and Voice (ADSV).

PubMed

Sauder, Cara; Bretl, Michelle; Eadie, Tanya

2017-09-01

The purposes of this study were to (1) determine and compare the diagnostic accuracy of a single acoustic measure, smoothed cepstral peak prominence (CPPS), to predict voice disorder status from connected speech samples using two software systems: Analysis of Dysphonia in Speech and Voice (ADSV) and Praat; and (2) to determine the relationship between measures of CPPS generated from these programs. This is a retrospective cross-sectional study. Measures of CPPS were obtained from connected speech recordings of 100 subjects with voice disorders and 70 nondysphonic subjects without vocal complaints using commercially available ADSV and freely downloadable Praat software programs. Logistic regression and receiver operating characteristic (ROC) analyses were used to evaluate and compare the diagnostic accuracy of CPPS measures. Relationships between CPPS measures from the programs were determined. Results showed acceptable overall accuracy rates (75% accuracy, ADSV; 82% accuracy, Praat) and area under the ROC curves (area under the curve [AUC] = 0.81, ADSV; AUC = 0.91, Praat) for predicting voice disorder status, with slight differences in sensitivity and specificity. CPPS measures derived from Praat were uniquely predictive of disorder status above and beyond CPPS measures from ADSV (χ 2 (1) = 40.71, P < 0.001). CPPS measures from both programs were significantly and highly correlated (r = 0.88, P < 0.001). A single acoustic measure of CPPS was highly predictive of voice disorder status using either program. Clinicians may consider using CPPS to complement clinical voice evaluation and screening protocols. Copyright © 2017 The Voice Foundation. Published by Elsevier Inc. All rights reserved.

A combined impact-process evaluation of a program promoting active transport to school: understanding the factors that shaped program effectiveness.

PubMed

Crawford, S; Garrard, J

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts.

A Combined Impact-Process Evaluation of a Program Promoting Active Transport to School: Understanding the Factors That Shaped Program Effectiveness

PubMed Central

Crawford, S.; Garrard, J.

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts. PMID:23606865

A Cost-Effectiveness/Benefit Analysis Model for Postsecondary Vocational Programs. Technical Report.

ERIC Educational Resources Information Center

Kim, Jin Eun

A cost-effectiveness/benefit analysis is defined as a technique for measuring the outputs of existing and new programs in relation to their specified program objectives, against the costs of those programs. In terms of its specific use, the technique is conceptualized as a systems analysis method, an evaluation method, and a planning tool for…

Evaluation of the Industrial Maintenance Program at Caldwell Community College and Technical Institute.

ERIC Educational Resources Information Center

Pipes, V. David

In 1982-83, a study was conducted at Caldwell Community College and Technical Institute to determine whether the objectives of the Industrial Maintenance Program were being met, to measure program success, and to identify aspects needing improvement. Surveys were sent to 30 students who graduated from the program before 1978, yielding a 57%…

A Process Evaluation of Project Developmental Continuity, Interim Report VII, Volume 1: Findings from the PDC Implementation Study.

ERIC Educational Resources Information Center

Smith, Allen G.; And Others

This third year interim report, one of a series of documents on the evaluation of Project Developmental Continuity (PDC), presents findings from three major analyses of program implementation; measurement of the extent each program has implemented the basic PDC Guidelines; a description of patterns of that implementation; and analysis of some…

West Virginia Safe and Supportive (S3) Schools Project: Year 2 Implementation Evaluation Report

ERIC Educational Resources Information Center

Whisman, Andy

2013-01-01

The 4-year federal Safe and Supportive Schools (S3) program supports targeted interventions to improve and measure conditions for learning at the high school level. For 2011-2012 (Year 2), two evaluation questions were investigated: (EQ1) To what extent do participating schools implement the program with fidelity relative to the WV Model for…

An Evaluation of the Model School Division (MSD) Preschool Program for the School Year 1973-74.

ERIC Educational Resources Information Center

District of Columbia Public Schools, Washington, DC. Dept. of Research and Evaluation.

This study was designed to assess the extent to which children served by the Model Schools Division Preschool Program developed socially, intellectually, physically and emotionally during the 1973-74 school year. This evaluation was also designed to measure the appropriateness of the learning environment and the amount of services provided by the…

NASA Human Research Program (HRP). International Space Station Medical Project (ISSMP)

NASA Technical Reports Server (NTRS)

Sams, Clarence F.

2009-01-01

This viewgraph presentation describes the various flight investigations performed on the International Space Station as part of the NASA Human Research Program (HRP). The evaluations include: 1) Stability; 2) Periodic Fitness Evaluation with Oxygen Uptake Measurement; 3) Nutrition; 4) CCISS; 5) Sleep; 6) Braslet; 7) Integrated Immune; 8) Epstein Barr; 9) Biophosphonates; 10) Integrated cardiovascular; and 11) VO2 max.

Far from Home: An Experimental Evaluation of the Mother-Child Home Program in Bermuda.

ERIC Educational Resources Information Center

Scarr, Sandra; McCartney, Kathleen

1988-01-01

Effects of the Mother-Child Home Program (MCHP) were evaluated with a broad range of measures on cognition, social behavior, and emotion. Findings indicated that children in Bermuda scored above U.S. norms on cognitive tests and were functioning well in the preschool period. The MCHP had few demonstrable effects on any segment of the sample. (RH)

The Measure of a Good School. A Guide to Evaluation of School Systems Adapted Particularly for Use in Kentucky School Districts.

ERIC Educational Resources Information Center

Kentucky Univ., Lexington. Coll. of Education.

Comments on school evaluation are divided into the following categories--(1) program goals and objectives of the schools, (2) staff--age, sex, experience, preparation, and problem of retaining good teachers, (3) administration and their application to district organization, (4) finance--an outline of how a foundation program operates, and comments…

Life Outcomes and Higher Education: The Need for Longitudinal Research Using a Broad Range of Quality of Life Indicators.

PubMed

Sheppard-Jones, Kathleen; Kleinert, Harold; Butler, Laura; Whaley, Barry

2018-02-01

Higher education is increasingly becoming an option for young adults with intellectual disability (ID). Although initial evaluations of postsecondary education for this population have been promising, a broader "quality of life" framework needs to be adopted in order to truly understand the impact of these programs. Moreover, researchers and program evaluators must collect longitudinal data that follows former students for multiple years and uses multiple measures. We conducted a pilot evaluation of the life outcomes of students who had attended at least two semesters in Kentucky's supported higher education program for students with ID, collecting data on life status and experiences using measures from the National Core Indicators-Adult Consumer Survey. The findings from this pilot study show better outcomes for young adults who participated in a postsecondary education program compared to young adults who did not, but these findings need to be considered in light of several limitations. In many respects, our data provided more new questions than answers. Recommendations for collecting and evaluating broad-based, longitudinal data to gain insight into the potential benefits of postsecondary education for people with intellectual disability are discussed.

The Vanderbilt Professional Nursing Practice Program: part 1: Growing and supporting professional nursing practice.

PubMed

Robinson, Karen; Eck, Carol; Keck, Becky; Wells, Nancy

2003-09-01

Professional practice programs are designed to attract, retain, and reward nurses. This three-part series will describe Vanderbilt's performance-based career advancement system, the Vanderbilt Professional Nursing Practice Program (VPNPP). Part 1 outlines the overall program's foundation, philosophical background, and basic structure. The VPNPP is built upon Benner's work, distinguishing among four levels of practice: novice, competent, proficient, and expert. Work by many in the organization identified the expected behaviors for nurses at each level, which were then used to develop clear process evaluation criteria. Part 2 will examine the performance measurement and evaluation system created to support the program. The process of advancing within the program will be described in part 3.

Lessons learned: Evaluating the program fidelity of UNWomen Partnership for Peace domestic violence diversion program in the Eastern Caribbean.

PubMed

Jeremiah, Rohan D; Quinn, Camille R; Alexis, Jicinta M

2018-08-01

To date, there have been a plethora of punitive and diversion programs to address domestic violence around the world. However, the evaluative scholarship of such programs overwhelmingly reflects studies in developed countries while barely showcasing the realities of addressing domestic violence in developing countries. This paper features a multi-year (2008-2011) evaluation study that measured the fidelity of the United Nations Partnership for Peace (PfP) domestic violence diversion program in the Eastern Caribbean country of Grenada. Our findings illuminate organic engagement strategies that were built within existing multi-sectoral partnerships that included magistrate court judges, law enforcement officials, and social service agencies. Furthermore, we documented how the locally-devised implementation strategies ensured the program's fidelity within a resource-limited context. This paper contributes to the global evaluative scholarship, highlighting the lessons learned about implementing culturally-adapted and theoretically-driven domestic violence diversion within a developing country. Copyright © 2018 Elsevier Ltd. All rights reserved.

Security Assistance: Evaluations Needed to Determine Effectiveness of U.S. Aid to Lebanon’s Security Forces

DTIC Science & Technology

2013-03-01

5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES...meet conditions on the ground, according to U.S. officials. For example , the Department of State (State) delayed committing Foreign Military...agencies measure program performance. For example , GAO found in 2011 that the IMET program evaluation efforts had few of the elements commonly

TECHNOLOGY EVALUATION REPORT, HYDROTECHNICS IN SITU FLOW SENSOR

EPA Science Inventory

The U.S. Environmental Protection Agency (EPA) Superfund Innovative Technology Evaluation (SITE) Program evaluated performance of HydroTechnics, Inc. flow sensors in measuring the three-dimensional flow pattern created by operation of the Wasatch Environmental, Inc. (WEI) ground...

Test Program for Evaluation of Variable Frequency Power Conditioners

DOT National Transportation Integrated Search

1973-08-01

A test program is outlined for variable frequency power conditioners for 3-phase induction motors in vehicle propulsion applications. The Power Conditioner Unit (PCU) performance characteristics are discussed in some detail. Measurement methods, reco...

Rainfall Product Evaluation for the TRMM Ground Validation Program

NASA Technical Reports Server (NTRS)

Amitai, E.; Wolff, D. B.; Robinson, M.; Silberstein, D. S.; Marks, D. A.; Kulie, M. S.; Fisher, B.; Einaudi, Franco (Technical Monitor)

2000-01-01

Evaluation of the Tropical Rainfall Measuring Mission (TRMM) satellite observations is conducted through a comprehensive Ground Validation (GV) Program. Standardized instantaneous and monthly rainfall products are routinely generated using quality-controlled ground based radar data from four primary GV sites. As part of the TRMM GV program, effort is being made to evaluate these GV products and to determine the uncertainties of the rainfall estimates. The evaluation effort is based on comparison to rain gauge data. The variance between the gauge measurement and the true averaged rain amount within the radar pixel is a limiting factor in the evaluation process. While monthly estimates are relatively simple to evaluate, the evaluation of the instantaneous products are much more of a challenge. Scattegrams of point comparisons between radar and rain gauges are extremely noisy for several reasons (e.g. sample volume discrepancies, timing and navigation mismatches, variability of Z(sub e)-R relationships), and therefore useless for evaluating the estimates. Several alternative methods, such as the analysis of the distribution of rain volume by rain rate as derived from gauge intensities and from reflectivities above the gauge network will be presented. Alternative procedures to increase the accuracy of the estimates and to reduce their uncertainties also will be discussed.

Improved self-exclusion program: preliminary results.

PubMed

Tremblay, Nicole; Boutin, Claude; Ladouceur, Robert

2008-12-01

The gambling industry has offered self-exclusion programs for quite a long time. Such measures are designed to limit access to gaming opportunities and provide problem gamblers with the help they need to cease or limit their gambling behaviour. However, few studies have empirically evaluated these programs. This study has three objectives: (1) to observe the participation in an improved self-exclusion program that includes an initial voluntary evaluation, phone support, and a mandatory meeting, (2) to evaluate satisfaction and usefulness of this service as perceived by self-excluders, (3) to measure the preliminary impact of this improved program. One hundred sixteen self-excluders completed a questionnaire about their satisfaction and their perception of the usefulness during the mandatory meeting. Among those participants, 39 attended an initial meeting. Comparisons between data collected at the initial meeting and data taken at the final meeting were made for those 39 participants. Data showed that gamblers chose the improved self-exclusion program 75% of the time; 25% preferred to sign a regular self-exclusion contract. Among those who chose the improved service, 40% wanted an initial voluntary evaluation and 37% of these individuals actually attended that meeting. Seventy percent of gamblers came to the mandatory meeting, which was a required condition to end their self-exclusion. The majority of participants were satisfied with the improved self-exclusion service and perceived it as useful. Major improvements were observed between the final and the initial evaluation on time and money spent, consequences of gambling, DSM-IV score, and psychological distress. The applicability of an improved self-exclusion program is discussed and, as shown in our study, the inclusion of a final mandatory meeting might not be so repulsive for self-excluders. Future research directives are also proposed.

75 FR 17940 - Draft Programmatic Environmental Assessment for the Evaluation of the Grant Programs Directorate...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-04-08

... and emergency preparedness grant programs that it administers. These programs provide grant funding to... effects of communication facilities construction or modification subject to review by the Federal... have or are likely to have a measurable negative effect on migratory bird populations. Avian mortality...

Outcome Measures of Health Programs: What and How?

ERIC Educational Resources Information Center

Chen, Martin K.

With the proliferation of new health programs, such as Health Maintenance Organizations (HMO's) and Professional Service Review Organizations (PSRO's), the task of evaluating the impact of such programs on the health delivery systems and on the health of the American people becomes more urgent. Thus far no experimental or quasi-experimental…

An Experimental Health Education Program: Effects on Urban Primary Students.

ERIC Educational Resources Information Center

Andrews, Richard L.; Hearne, Jill T.

1986-01-01

Presents findings of an evaluation of a health education program for primary school students of low SES. Related findings to two dependent measures--attitudes toward smoking and knowledge about smoking and health. Reports that program participants displayed more positive attitudes toward good health practices than did non-participating control…

78 FR 77204 - Proposed Information Collection (VA National Veterans Sports Programs and Special Event Surveys...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-12-20

... National Veterans Sports Programs and Special Event Surveys Data Collection) Activity: Comment Request... needed to evaluate the National Veterans Sports Programs and Special Event Surveys Data Collection is... treatments are being offered, and tangible, quantitative results are being measured and tracked for continual...

An Assessment of an Operational Educational Accountability System for Continuing Education in the Health Professions.

ERIC Educational Resources Information Center

Walsh, Patrick L.

1982-01-01

The Educational Quality Assessment and Assurance System for continuing education in health sciences includes three components to measure inputs, processes, and outcomes. They are (1) Program Coordinator Competency List, (2) Quality Assessment and Assurance Program, and (3) evaluation of educational programs. (SK)

Guidelines for Creating, Implementing, and Evaluating Mind-Body Programs in a Military Healthcare Setting.

PubMed

Smith, Katherine; Firth, Kimberly; Smeeding, Sandra; Wolever, Ruth; Kaufman, Joanna; Delgado, Roxana; Bellanti, Dawn; Xenakis, Lea

2016-01-01

Research suggests that the development of mind-body skills can improve individual and family resilience, particularly related to the stresses of illness, trauma, and caregiving. To operationalize the research evidence that mind-body skills help with health and recovery, Samueli Institute, in partnership with experts in mind-body programming, created a set of guidelines for developing and evaluating mind-body programs for service members, veterans, and their families. The Guidelines for Creating, Implementing, and Evaluating Mind-Body Programs in a Military Healthcare Setting outline key strategies and issues to consider when developing, implementing, and evaluating a mind-body focused family empowerment approach in a military healthcare setting. Although these guidelines were developed specifically for a military setting, most of the same principles can be applied to the development of programs in the civilian setting as well. The guidelines particularly address issues unique to mind-body programs, such as choosing evidence-based modalities, licensure and credentialing, safety and contraindications, and choosing evaluation measures that capture the holistic nature of these types of programs. The guidelines are practical, practice-based guidelines, developed by experts in the fields of program development and evaluation, mind-body therapies, patient- and family-centered care, as well as, experts in military and veteran's health systems. They provide a flexible framework to create mind-body family empowerment programs and describe important issues that program developers and evaluators are encouraged to address to ensure the development of the most impactful, successful, evidence-supported programs possible. Copyright © 2016 Elsevier Inc. All rights reserved.

The development of a culturally appropriate school based intervention for Australian Aboriginal children living in remote communities: A formative evaluation of the Alert Program® intervention.

PubMed

Wagner, Bree; Fitzpatrick, James; Symons, Martyn; Jirikowic, Tracy; Cross, Donna; Latimer, Jane

2017-06-01

Although previous research has demonstrated the benefits of targeting self-regulation in non-Aboriginal children, it is unclear whether such programs would be effective for Aboriginal children attending school in remote communities. Some of these children have been diagnosed with a fetal alcohol spectrum disorder (FASD) impairing their ability to self-regulate. The aim of this article is to describe a three phase formative process to develop and pilot a curriculum version of the Alert Program ® , a promising intervention for improving self-regulation that could be used in remote community schools. This modified version of the program will be subsequently tested in a cluster randomised controlled trial. A mixed methods approach was used. Modifications to the Alert Program ® , its delivery and evaluation were made after community and stakeholder consultation facilitated by a senior Aboriginal community researcher. Changes to lesson plans and program resources were made to reflect the remote community context, classroom environment and the challenging behaviours of children. Standardised study outcome measures were modified by removing several questions that had little relevance to the lives of children in remote communities. Program training for school staff was reduced in length to reduce staff burden. This study identified aspects of the Alert Program ® training, delivery and measures for evaluation that need modification before their use in assessing the efficacy of the Alert Program ® in remote Aboriginal community primary schools. © 2016 Occupational Therapy Australia.

[Evaluation of a program to promote network building between disciplinary agencies and informal community organizations: trial in a community comprehensive support center].

PubMed

Murayama, Hiroshi; Kojima, Tomoko; Tomaru, Meiko; Narabu, Harumi; Tachibana, Reiko; Yamaguchi, Takuhiro; Murashima, Sachiyo

2010-10-01

To examine the effectiveness of a program promoting network building between disciplinary agencies and informal community organizations (IGOs) comprising community residents, by implemention with staff of a community comprehensive support center (CJCSG). The program was implemented for nine staff of a GGSG in Setagaya Ward, Tokyo for a year. For process evaluation, items were assessed concerning the contents of the program such as satisfaction and understandability, participants' goal attainment level at each period of the program, and program satisfaction as a whole. Outcome evaluation included measurement of participants' self-efficacy regarding network building with ICOs before and after the program, using interviews of the members who completed the program. Eight out of the nine participants completed the program. All positively evaluated the contents of the program and their own goal attainment at each period of the program. After its completion, they felt highly satisfied. Moreover, there was an improvement in the cognition of the participants, including self-efficacy on network building with IGOs and the atmosphere in the GGSG with regard to network building. The efficacy of this program could be confirmed as demonstrated by the staff of the CCSC, although a more detailed assessment of validity and effectiveness will be necessary in the future.

Qualitative evaluation: A critical and interpretative complementary approach to improve health programs and services

PubMed Central

Tayabas, Luz María Tejada; León, Teresita Castillo; ESPINO, JOEL MONARREZ

2014-01-01

This short essay aims at commenting on the origin, development, rationale, and main characteristics of qualitative evaluation (QE), emphasizing the value of this methodological tool to evaluate health programs and services. During the past decades, different approaches have come to light proposing complementary alternatives to appraise the performance of public health programs, mainly focusing on the implementation process involved rather than on measuring the impact of such actions. QE is an alternative tool that can be used to illustrate and understand the process faced when executing health programs. It can also lead to useful suggestions to modify its implementation from the stakeholders’ perspectives, as it uses a qualitative approach that considers participants as reflective subjects, generators of meanings. This implies that beneficiaries become involved in an active manner in the evaluated phenomena with the aim of improving the health programs or services that they receive. With this work we want to encourage evaluators in the field of public health to consider the use of QE as a complementary tool for program evaluation to be able to identify areas of opportunity to improve programs’ implementation processes from the perspective of intended beneficiaries. PMID:25152220

EVALUATION OF A MEASUREMENT METHOD FOR FOREST VEGETATION IN A LARGE-SCALE ECOLOGICAL SURVEY

EPA Science Inventory

We evaluate a field method for determining species richness and canopy cover of vascular plants for the Forest Health Monitoring Program (FHM), an ecological survey of U.S. forests. Measurements are taken within 12 1-m2 quadrats on 1/15 ha plots in FHM. Species richness and cover...

GATEWAY Demonstrations: OLED Lighting in the Offices of DeJoy, Knauf & Blood, LLP

DOE Office of Scientific and Technical Information (OSTI.GOV)

Miller, Naomi J.

At the offices of the accounting firm of DeJoy, Knauf & Blood, LLP in Rochester, NY, the GATEWAY program evaluated a new lighting system that incorporates a number of different OLED luminaires. Evaluation of the OLED products included efficacy performance, field measurements of panel color, flicker measurements, and staff feedback.

Evaluating the effectiveness of a radiation safety training intervention for oncology nurses: a pretest-intervention-posttest study.

PubMed

Dauer, Lawrence T; Kelvin, Joanne F; Horan, Christopher L; St Germain, Jean

2006-06-08

Radiation, for either diagnosis or treatment, is used extensively in the field of oncology. An understanding of oncology radiation safety principles and how to apply them in practice is critical for nursing practice. Misconceptions about radiation are common, resulting in undue fears and concerns that may negatively impact patient care. Effectively educating nurses to help overcome these misconceptions is a challenge. Historically, radiation safety training programs for oncology nurses have been compliance-based and behavioral in philosophy. A new radiation safety training initiative was developed for Memorial Sloan-Kettering Cancer Center (MSKCC) adapting elements of current adult education theories to address common misconceptions and to enhance knowledge. A research design for evaluating the revised training program was also developed to assess whether the revised training program resulted in a measurable and/or statistically significant change in the knowledge or attitudes of nurses toward working with radiation. An evaluation research design based on a conceptual framework for measuring knowledge and attitude was developed and implemented using a pretest-intervention-posttest approach for 15% of the study population of 750 inpatient registered oncology nurses. As a result of the intervention program, there was a significant difference in nurse's cognitive knowledge as measured with the test instrument from pretest (58.9%) to posttest (71.6%). The evaluation also demonstrated that while positive nursing attitudes increased, the increase was significant for only 5 out of 9 of the areas evaluated. The training intervention was effective for increasing cognitive knowledge, but was less effective at improving overall attitudes. This evaluation provided insights into the effectiveness of training interventions on the radiation safety knowledge and attitude of oncology nurses.

Evaluating the effectiveness of a radiation safety training intervention for oncology nurses: a pretest – intervention – posttest study

PubMed Central

Dauer, Lawrence T; Kelvin, Joanne F; Horan, Christopher L; St Germain, Jean

2006-01-01

Background Radiation, for either diagnosis or treatment, is used extensively in the field of oncology. An understanding of oncology radiation safety principles and how to apply them in practice is critical for nursing practice. Misconceptions about radiation are common, resulting in undue fears and concerns that may negatively impact patient care. Effectively educating nurses to help overcome these misconceptions is a challenge. Historically, radiation safety training programs for oncology nurses have been compliance-based and behavioral in philosophy. Methods A new radiation safety training initiative was developed for Memorial Sloan-Kettering Cancer Center (MSKCC) adapting elements of current adult education theories to address common misconceptions and to enhance knowledge. A research design for evaluating the revised training program was also developed to assess whether the revised training program resulted in a measurable and/or statistically significant change in the knowledge or attitudes of nurses toward working with radiation. An evaluation research design based on a conceptual framework for measuring knowledge and attitude was developed and implemented using a pretest-intervention-posttest approach for 15% of the study population of 750 inpatient registered oncology nurses. Results As a result of the intervention program, there was a significant difference in nurse's cognitive knowledge as measured with the test instrument from pretest (58.9%) to posttest (71.6%). The evaluation also demonstrated that while positive nursing attitudes increased, the increase was significant for only 5 out of 9 of the areas evaluated. Conclusion The training intervention was effective for increasing cognitive knowledge, but was less effective at improving overall attitudes. This evaluation provided insights into the effectiveness of training interventions on the radiation safety knowledge and attitude of oncology nurses. PMID:16762060

Chapter 17: Residential Behavior Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W.; Stewart, James; Todd, Annika

Residential behavior-based (BB) programs use strategies grounded in the behavioral and social sciences to influence household energy use. These may include providing households with real-time or delayed feedback about their energy use; supplying energy efficiency education and tips; rewarding households for reducing their energy use; comparing households to their peers; and establishing games, tournaments, and competitions. BB programs often target multiple energy end uses and encourage energy savings, demand savings, or both. Savings from BB programs are usually a small percentage of energy use, typically less than 5 percent. Utilities will continue to implement residential BB programs as large-scale, randomizedmore » control trials (RCTs); however, some are now experimenting with alternative program designs that are smaller scale; involve new communication channels such as the web, social media, and text messaging; or that employ novel strategies for encouraging behavior change (for example, Facebook competitions). These programs will create new evaluation challenges and may require different evaluation methods than those currently employed to verify any savings they generate. Quasi-experimental methods, however, require stronger assumptions to yield valid savings estimates and may not measure savings with the same degree of validity and accuracy as randomized experiments.« less

Evaluation of the Capillary Blood Glucose Self-monitoring Program

PubMed Central

Augusto, Mariana Cristina; Nitsche, Maria José Trevizani; Parada, Cristina Maria Garcia de Lima; Zanetti, Maria Lúcia; Carvalhaes, Maria Antonieta de Barros Leite

2014-01-01

OBJECTIVE: to evaluate the structure, process and results of the Capillary Blood Glucose Self-monitoring Program in a Brazilian city. METHOD: epidemiological, cross-sectional study. The methodological framework of Donabedian was used to construct indicators of structure, process and outcome. A random sample (n = 288) of users enrolled and 96 health professionals who worked in the program was studied. Two questionnaires were used that were constructed for this study, one for professionals and one for users, both containing data for the evaluation of structure, process and outcome. Anthropometric measures and laboratory results were collected by consulting the patients' health records. The analysis involved descriptive statistics. RESULTS: most of the professionals were not qualified to work in the program and were not knowledgeable about the set of criteria for patient registration. None of the patients received complete and correct orientations about the program and the percentage with skills to perform conducts autonomously was 10%. As regards the result indicators, 86.4% of the patients and 81.3% of the professionals evaluated the program positively. CONCLUSION: the evaluation indicators designed revealed that one of the main objectives of the program, self-care skills, has not been achieved. PMID:25493676

Evaluating Youth Development Programs: Progress and Promise

PubMed Central

Brooks-Gunn, Jeanne

2016-01-01

Advances in theories of adolescent development and positive youth development have greatly increased our understanding of how programs and practices with adolescents can impede or enhance their development. In this paper the authors reflect on the progress in research on youth development programs in the last two decades, since possibly the first review of empirical evaluations by Roth, Brooks-Gunn, Murray, and Foster (1998). The authors use the terms Version 1.0, 2.0 and 3.0 to refer to changes in youth development research and programs over time. They argue that advances in theory and descriptive accounts of youth development programs (Version 2.0) need to be coupled with progress in definitions of youth development programs, measurement of inputs and outputs that incorporate an understanding of programs as contexts for development, and stronger design and evaluation of programs (Version 3.0). The authors also advocate for an integration of prevention and promotion research, and for use of the term youth development rather than positive youth development. PMID:28077922

40 CFR 195.2 - Definitions.

Code of Federal Regulations, 2011 CFR

2011-07-01

... program. Device/measurement device means a unit, component, or system designed to measure radon gas or... application for each location from which it provides radon measurement services. After the application is... exam means the exam which evaluates individuals who provide radon measurement services in a residential...

The Evaluation of Teachers.

ERIC Educational Resources Information Center

National Education Association, Washington, DC. Div. of Instruction and Professional Development.

The several components of this package on the evaluation of teachers and educational programs are designed to help affiliates deal constructively with the subject. The issue of evaluation continues to intensify as state legislatures increasingly mandate that evaluation systems be imposed throughout the state to measure the performance of teachers…

Defining success in diabetes disease management: digging deeper in the data.

PubMed

Bray, Karen; Turpin, Robin S; Jungkind, Kim; Heuser, George

2008-04-01

We evaluated the effectiveness of a diabetes life coach program designed to address the concerns of limited coordination and collaboration of care for chronically ill patients in the physician office. The program emphasized lipid, blood pressure, and glycemic control, using personal coaching, group classes, reminders, and customized feedback. The target population was all health plan members over age 18 with type 1 or 2 diabetes mellitus in 6 primary care practice sites in the Hampton Roads area of Virginia. Primary outcomes were 1 Health Plan Employer Data and Information Set measure (A1c poor control of >9% or no test), 3 American Diabetes Association (ADA) measures (A1c <7%, blood pressure of <130/80 mmHg, low-density lipoprotein cholesterol [LDL-C] of <100 mg/dL), 1 pharmacy measure (percentage of patients filling at least 1 insulin prescription), and 2 self-reported behavioral measures (percentage adherent to a meal plan and percentage adherent to an activity plan). We assessed overall program outcomes and differences between individual physician practices and evaluated outcomes separately for engaged compared with non-engaged program participants. Outcomes for 1117 participants were evaluated. Statistically significant improvement at P < 0.05 was noted in all 7 targeted measures compared with baseline. Participants who were engaged in the life coach program were 40% less likely to experience poor control of their A1c, 50% more likely to meet the ADA A1c goal of < 7%, 11% more likely to meet their blood pressure goal of <130/80 mmHg, and 7% more likely to meet their LDL-C goal of <100 mg/dL compared with those not engaged. Patients who became engaged in the program performed significantly better in the key diabetes indicators that ultimately lead to reductions in the complications of the disease over time. Our study contributes to the evidence that clinical multidisciplinary, collaborative models of care can influence and improve the management of diabetes.

Measuring Parental Treatment Adherence in a Multimodal Treatment Program for Children with ADHD: A Preliminary Investigation

ERIC Educational Resources Information Center

Springer, Craig; Reddy, Linda A.

2010-01-01

This study evaluated the clinical significance of measuring between session parental adherence on child and parent outcomes for 51 children (age 4 to 8.5 years) with attention deficit/hyperactivity disorder (ADHD) in a multimodal group training program. Three group treatment conditions: (a) child-only treatment (C1), (c) child and parent training…

Air quality monitor and acid rain networks

NASA Technical Reports Server (NTRS)

Rudolph, H.

1980-01-01

The air quality monitor program which consists of two permanent air monitor stations (PAMS's) and four mobile shuttle pollutant air monitor stations (SPAMS's) is evaluated. The PAMS measures SO sub X, NO sub X particulates, CO, O3, and nonmethane hydrocarbons. The SPAMS measures O3, SO2, HCl, and particulates. The collection and analysis of data in the rain monitor program are discussed.

Quantitative reactive modeling and verification.

PubMed

Henzinger, Thomas A

Formal verification aims to improve the quality of software by detecting errors before they do harm. At the basis of formal verification is the logical notion of correctness , which purports to capture whether or not a program behaves as desired. We suggest that the boolean partition of software into correct and incorrect programs falls short of the practical need to assess the behavior of software in a more nuanced fashion against multiple criteria. We therefore propose to introduce quantitative fitness measures for programs, specifically for measuring the function, performance, and robustness of reactive programs such as concurrent processes. This article describes the goals of the ERC Advanced Investigator Project QUAREM. The project aims to build and evaluate a theory of quantitative fitness measures for reactive models. Such a theory must strive to obtain quantitative generalizations of the paradigms that have been success stories in qualitative reactive modeling, such as compositionality, property-preserving abstraction and abstraction refinement, model checking, and synthesis. The theory will be evaluated not only in the context of software and hardware engineering, but also in the context of systems biology. In particular, we will use the quantitative reactive models and fitness measures developed in this project for testing hypotheses about the mechanisms behind data from biological experiments.

Impact of Appointment-Based Medication Synchronization on Proportion of Days Covered for Chronic Medications.

PubMed

Dao, Nancy; Lee, Sun; Hata, Micah; Sarino, Lord

2018-05-22

Appointment-based medication synchronization (ABMS) programs have been associated with increased adherence and persistence to chronic medications. Adherence to statin therapy, angiotensin-converting enzyme inhibitors (ACEIs), angiotensin receptor blockers (ARBs), and non-insulin antidiabetic medications (NIDM) are used to determine a health plan's Centers for Medicare and Medicaid Services (CMS) Star Rating under a pay-for-performance model. The objective of this study was to evaluate the impact of implementing an ABMS program on overall pharmacy adherence measures for statins, ACEI/ARBs, and NIDM, as presented through the Electronic Quality Improvement Platform for Plans and Pharmacies (EQuIPP©) platform. This retrospective, pre-post ABMS program study evaluated EQuIPP© generated adherence performance measures, represented as proportion of days covered (PDC), 6-months before and 6- and 12-months after the ABMS service for statin therapy, ACEIs/ARBs, and NIDM. All adherence measures showed statistically significant improvement in PDC percentage post ABMS implementation, except for NIDM percentage in 6-months post-ABMS service. This study shows that a comprehensive medication synchronization program can enhance adherence measures that are important to health plans to increase CMS Star Rating under a pay-for-performance model.

Use of a structured template to facilitate practice-based learning and improvement projects.

PubMed

McClain, Elizabeth K; Babbott, Stewart F; Tsue, Terance T; Girod, Douglas A; Clements, Debora; Gilmer, Lisa; Persons, Diane; Unruh, Greg

2012-06-01

The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to meet and demonstrate outcomes across 6 competencies. Measuring residents' competency in practice-based learning and improvement (PBLI) is particularly challenging. We developed an educational tool to meet ACGME requirements for PBLI. The PBLI template helped programs document quality improvement (QI) projects and supported increased scholarly activity surrounding PBLI learning. We reviewed program requirements for 43 residency and fellowship programs and identified specific PBLI requirements for QI activities. We also examined ACGME Program Information Form responses on PBLI core competency questions surrounding QI projects for program sites visited in 2008-2009. Data were integrated by a multidisciplinary committee to develop a peer-protected PBLI template guiding programs through process, documentation, and evaluation of QI projects. All steps were reviewed and approved through our GME Committee structure. An electronic template, companion checklist, and evaluation form were developed using identified project characteristics to guide programs through the PBLI process and facilitate documentation and evaluation of the process. During a 24 month period, 27 programs have completed PBLI projects, and 15 have reviewed the template with their education committees, but have not initiated projects using the template. The development of the tool generated program leaders' support because the tool enhanced the ability to meet program-specific objectives. The peer-protected status of this document for confidentiality and from discovery has been beneficial for program usage. The document aggregates data on PBLI and QI initiatives, offers opportunities to increase scholarship in QI, and meets the ACGME goal of linking measures to outcomes important to meeting accreditation requirements at the program and institutional level.

Use of a Structured Template to Facilitate Practice-Based Learning and Improvement Projects

PubMed Central

McClain, Elizabeth K.; Babbott, Stewart F.; Tsue, Terance T.; Girod, Douglas A.; Clements, Debora; Gilmer, Lisa; Persons, Diane; Unruh, Greg

2012-01-01

Background The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to meet and demonstrate outcomes across 6 competencies. Measuring residents' competency in practice-based learning and improvement (PBLI) is particularly challenging. Purpose We developed an educational tool to meet ACGME requirements for PBLI. The PBLI template helped programs document quality improvement (QI) projects and supported increased scholarly activity surrounding PBLI learning. Methods We reviewed program requirements for 43 residency and fellowship programs and identified specific PBLI requirements for QI activities. We also examined ACGME Program Information Form responses on PBLI core competency questions surrounding QI projects for program sites visited in 2008–2009. Data were integrated by a multidisciplinary committee to develop a peer-protected PBLI template guiding programs through process, documentation, and evaluation of QI projects. All steps were reviewed and approved through our GME Committee structure. Results An electronic template, companion checklist, and evaluation form were developed using identified project characteristics to guide programs through the PBLI process and facilitate documentation and evaluation of the process. During a 24 month period, 27 programs have completed PBLI projects, and 15 have reviewed the template with their education committees, but have not initiated projects using the template. Discussion The development of the tool generated program leaders' support because the tool enhanced the ability to meet program-specific objectives. The peer-protected status of this document for confidentiality and from discovery has been beneficial for program usage. The document aggregates data on PBLI and QI initiatives, offers opportunities to increase scholarship in QI, and meets the ACGME goal of linking measures to outcomes important to meeting accreditation requirements at the program and institutional level. PMID:23730444

ETV - HOMELAND SECURITY EVALUATION OF CYANIDE DETECTORS

EPA Science Inventory

EPA's Environmental Technology Verification (ETV) Program was established in 1995 to objectively verify the performance of technologies that measure / monitor the quality of our environment, both for background or at suspected contamination site. The ETV program has established...

Evaluating health inequity interventions: applying a contextual (external) validity framework to programs funded by the Canadian Health Services Research Foundation.

PubMed

Phillips, Kaye; Müller-Clemm, Werner; Ysselstein, Margaretha; Sachs, Jonathan

2013-02-01

Including context in the measurement and evaluation of health in equity interventions is critical to understanding how events that occur in an intervention's environment might contribute to or impede its success. This study adapted and piloted a contextual validity assessment framework on a selection of health inequity-related programs funded by the Canadian Health Services Research Foundation (CHSRF) between 1998 and 2006. The two overarching objectives of this study were (1) to determine the relative amount and quality of attention given to conceptualizing, measuring and validating context within CHSRF funded research final reports related to health-inequity; and (2) to contribute evaluative evidence towards the incorporation of context into the assessment and measurement of health inequity interventions. The study found that of the 42/146 CHSRF programs and projects, judged to be related to health inequity 20 adequately reported on the conceptualization, measurement and validation of context. Amongst these health-inequity related project reports, greatest emphasis was placed on describing the socio-political and economical context over actually measuring and validating contextual evidence. Applying a contextual validity assessment framework was useful for distinguishing between the descriptive (conceptual) versus empirical (measurement and validation) inclusion of documented contextual evidence. Although contextual validity measurement frameworks needs further development, this study contributes insight into identifying funded research related to health inequities and preliminary criteria for assessing interventions targeted at specific populations and jurisdictions. This study also feeds a larger critical dialogue (albeit beyond the scope of this study) regarding the relevance and utility of using evaluative techniques for understanding how specific external conditions support or impede the successful implementation of health inequity interventions. Copyright © 2012 Elsevier Ltd. All rights reserved.

Effectiveness of an occupational health intervention program to reduce whole body vibration exposure: an evaluation study with a controlled pretest-post-test design.

PubMed

Tiemessen, Ivo J H; Hulshof, Carel T J; Frings-Dresen, Monique H W

2009-12-01

An effective intervention program aiming to reduce whole body vibration (WBV) exposure at work will reduce the number of low back complaints in the near future. An evaluation study with a controlled pretest-post-test design. Nine companies and 126 drivers were included in the study. Cluster randomization on company level divided the drivers and their employers in an intervention group and a "care-as-usual" group. At baseline (T0) and intervention program was implemented and evaluated after 7 months (T1). The main outcome measure was WBV exposure. Process measures included knowledge, attitude, and (intended) behavior towards reduction of WBV exposure for the drivers and knowledge and WBV policy for the employers. At T1, no significant reduction was found in WBV exposure within both groups compared with T0. Probably due to poor to moderate compliance, the intervention program was not effective in reducing the WBV exposure on group level but small reductions in WBV exposure are possible when intervention compliance is high. Copyright 2009 Wiley-Liss, Inc.

Evaluation of preventive technologies in Germany: case studies of mammography, prostate cancer screening, and fetal ultrasound.

PubMed

Perleth, M; Busse, R; Gibis, B; Brand, A

2001-01-01

In this article, three preventive strategies-mammography screening for breast cancer, PSA screening for prostate cancer, and routine ultrasound in normal pregnancy-are discussed in the context of German health care. Epidemiologic data and German studies evaluating different aspects of these preventive measures were identified and analyzed. Only a few studies could be identified that investigate these preventive measures. Despite sufficient evidence, in part derived from a German study, there is not yet a mammography screening program. In contrast, ultrasound in pregnancy is offered routinely, although there are controversies regarding the benefit of this practice. PSA screening is not offered as part of the screening program for prostate cancer. However, PSA tests as well as mammographies are done in large numbers in German ambulatory care-a practice that could be considered wild or opportunistic screening. These case studies show that preventive programs and practices in Germany are not sufficiently based on sound evidence. The paucity of evaluation activities related to prevention in Germany is probably due to the low threshold to introduce new preventive programs into the German healthcare system in the past.

Competency-based goals, objectives, and linked evaluations for rheumatology training programs: a standardized template of learning activities from the Carolinas Fellows Collaborative.

PubMed

Criscione-Schreiber, Lisa G; Bolster, Marcy B; Jonas, Beth L; O'Rourke, Kenneth S

2013-06-01

American Council on Graduate Medical Education program requirements mandate that rheumatology training programs have written goals, objectives, and performance evaluations for each learning activity. Since learning activities are similar across rheumatology programs, we aimed to create competency-based goals and objectives (CBGO) and evaluations that would be generalizable nationally. Through an established collaboration of the 4 training programs' directors in North Carolina and South Carolina, we collaboratively composed CBGO and evaluations for each learning activity for rheumatology training programs. CBGO and linked evaluations were written using appropriate verbs based on Bloom's taxonomy. Draft documents were peer reviewed by faculty at the 4 institutions and by members of the American College of Rheumatology (ACR) Clinician Scholar Educator Group. We completed templates of CBGO for core and elective rotations and conferences. Templates detail progressive fellow performance improvement appropriate to educational level. Specific CBGO are mirrored in learning activity evaluations. Templates are easily modified to fit individual program attributes, have been successfully implemented by our 4 programs, and have proven their value in 4 residency review committee reviews. We propose adoption of these template CBGO by the ACR, with access available to all rheumatology training programs. Evaluation forms that exactly reflect stated objectives ensure that trainees are assessed using standardized measures and that trainees are aware of the learning expectations. The objectives mirrored in the evaluations closely align with the proposed milestones for internal medicine training, and will therefore be a useful starting point for creating these milestones in rheumatology. Copyright © 2013 by the American College of Rheumatology.

Implementation Measurement for Evidence-Based Violence Prevention Programs in Communities.

PubMed

Massetti, Greta M; Holland, Kristin M; Gorman-Smith, Deborah

2016-08-01

Increasing attention to the evaluation, dissemination, and implementation of evidence-based programs (EBPs) has led to significant advancements in the science of community-based violence prevention. One of the prevailing challenges in moving from science to community involves implementing EBPs and strategies with quality. The CDC-funded National Centers of Excellence in Youth Violence Prevention (YVPCs) partner with communities to implement a comprehensive community-based strategy to prevent violence and to evaluate that strategy for impact on community-wide rates of violence. As part of their implementation approach, YVPCs document implementation of and fidelity to the components of the comprehensive youth violence prevention strategy. We describe the strategies and methods used by the six YVPCs to assess implementation and to use implementation data to inform program improvement efforts. The information presented describes the approach and measurement strategies employed by each center and for each program implemented in the partner communities. YVPCs employ both established and innovative strategies for measurement and tracking of implementation across a broad range of programs, practices, and strategies. The work of the YVPCs highlights the need to use data to understand the relationship between implementation of EBPs and youth violence outcomes.

Hardware test program for evaluation of baseline range/range rate sensor concept

NASA Technical Reports Server (NTRS)

Pernic, E.

1985-01-01

The test program Phase II effort provides additional design information in terms of range and range rate (R/R) sensor performance when observing and tracking a typical spacecraft target. The target used in the test program was a one-third scale model of the Hubble Space Telescope (HST) available at the MSFC test site where the tests were performed. A modified Bendix millimeter wave radar served as the R/R sensor test bed for evaluation of range and range rate tracking performance, and generation of radar signature characteristics of the spacecraft target. A summary of program test results and conclusions are presented along with detailed description of the Bendix test bed radar with accompaning instrumentation. The MSFC test site and facilities are described. The test procedures used to establish background levels, and the calibration procedures used in the range accuracy tests and RCS (radar cross section) signature measurements, are presented and a condensed version of the daily log kept during the 5 September through 17 September test period is also presented. The test program results are given starting with the RCS signature measurements, then continuing with range measurement accuracy test results and finally the range and range rate tracking accuracy test results.

Implementing and evaluating a program to facilitate chronic disease prevention and screening in primary care: a mixed methods program evaluation.

PubMed

Manca, Donna Patricia; Aubrey-Bassler, Kris; Kandola, Kami; Aguilar, Carolina; Campbell-Scherer, Denise; Sopcak, Nicolette; O'Brien, Mary Ann; Meaney, Christopher; Faria, Vee; Baxter, Julia; Moineddin, Rahim; Salvalaggio, Ginetta; Green, Lee; Cave, Andrew; Grunfeld, Eva

2014-10-08

The objectives of this paper are to describe the planned implementation and evaluation of the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Primary Care (BETTER 2) program which originated from the BETTER trial. The pragmatic trial, informed by the Chronic Care Model, demonstrated the effectiveness of an approach to Chronic Disease Prevention and Screening (CDPS) involving the use of a new role, the prevention practitioner. The desired goals of the program are improved clinical outcomes, reduction in the burden of chronic disease, and improved sustainability of the health-care system through improved CDPS in primary care. The BETTER 2 program aims to expand the implementation of the intervention used in the original BETTER trial into communities across Canada (Alberta, Ontario, Newfoundland and Labrador, the Northwest Territories and Nova Scotia). This proactive approach provides at-risk patients with an intervention from the prevention practitioner, a health-care professional. Using the BETTER toolkit, the prevention practitioner determines which CDPS actions the patient is eligible to receive, and through shared decision-making and motivational interviewing, develops a unique and individualized 'prevention prescription' with the patient. This intervention is 1) personalized; 2) addressing multiple conditions; 3) integrated through linkages to local, regional, or national resources; and 4) longitudinal by assessing patients over time. The BETTER 2 program brings together primary care providers, policy/decision makers and researchers to work towards improving CDPS in primary care. The target patient population is adults aged 40-65. The reach, effectiveness, adoption, implementation, maintain (RE-AIM) framework will inform the evaluation of the program through qualitative and quantitative methods. A composite index will be used to quantitatively assess the effectiveness of the prevention practitioner intervention. The CDPS actions comprising the composite index include the following: process measures, referral/treatment measures, and target/change outcome measures related to cardiovascular disease, diabetes, cancer and associated lifestyle factors. The BETTER 2 program is a collaborative approach grounded in practice and built from existing work (i.e., integration not creation). The program evaluation is designed to provide an understanding of issues impacting the implementation of an effective approach for CDPS within primary care that may be adapted to become sustainable in the non-research setting.

Development and Evaluation of Science and Technology Education Program Using Interferometric SAR

NASA Astrophysics Data System (ADS)

Ito, Y.; Ikemitsu, H.; Nango, K.

2016-06-01

This paper proposes a science and technology education program to teach junior high school students to measure terrain changes by using interferometric synthetic aperture radar (SAR). The objectives of the proposed program are to evaluate and use information technology by performing SAR data processing in order to measure ground deformation, and to incorporate an understanding of Earth sciences by analyzing interferometric SAR processing results. To draft the teaching guidance plan for the developed education program, this study considers both science and technology education. The education program was used in a Japanese junior high school. An educational SAR processor developed by the authors and the customized Delft object-oriented radar interferometric software package were employed. Earthquakes as diastrophism events were chosen as practical teaching materials. The selected events indicate clear ground deformation in differential interferograms with high coherence levels. The learners were able to investigate the ground deformations and disasters caused by the events. They interactively used computers and became skilled at recognizing the knowledge and techniques of information technology, and then they evaluated the technology. Based on the results of pre- and post-questionnaire surveys and self-evaluation by the learners, it was clarified that the proposed program was applicable for junior high school education, and the learners recognized the usefulness of Earth observation technology by using interferometric SAR. The usefulness of the teaching materials in the learning activities was also shown through the practical teaching experience.

Evaluation of animal control measures on pet demographics in Santa Clara County, California, 1993-2006.

PubMed

Kass, Philip H; Johnson, Karen L; Weng, Hsin-Yi

2013-01-01

The measurable benefits of animal control programs are unknown and the aim of this study was to determine the impact of these programs on pet population changes. A prospective cross-sectional study of 1000 households was implemented in 2005 to evaluate characteristics of the owned and unowned population of dogs and cats in Santa Clara County, California. The same population was previously studied 12 years earlier. During this time period, the county instituted in 1994 and then subsequently disestablished a municipal spay/neuter voucher program for cats. Dog intakes declined from 1992-2005, as they similarly did for an adjacent county (San Mateo). However, cat intakes declined significantly more in Santa Clara County than San Mateo, with an average annual decline of approximately 700 cats for the 12 year period. Time series analysis showed a greater than expected decline in the number of cats surrendered to shelters in Santa Clara County during the years the voucher program was in effect (1994-2005). The net savings to the county by reducing the number of cat shelter intakes was estimated at approximately $1.5 million. The measurable benefits of animal control programs are unknown and the aim of this study was to determine the impact of these programs on pet population changes.

Changes in functional health status of older women with heart disease: evaluation of a program based on self-regulation.

PubMed

Clark, N M; Janz, N K; Dodge, J A; Schork, M A; Fingerlin, T E; Wheeler, J R; Liang, J; Keteyian, S J; Santinga, J T

2000-03-01

This study involving 570 women aged 60 years or older with heart disease, assessed the effects of a disease management program on physical functioning, symptom experience, and psychosocial status. Women were randomly assigned to control or program groups. Six to eight women met weekly with a health educator and peer leader over 4 weeks to learn self-regulation skills with physical activity as the focus. Evaluative data were collected through telephone interviews, physical assessments, and medical records at baseline and 4 and 12 months post baseline. At 12 months, compared with controls, program women were less symptomatic (p < .01), scored better on the physical dimension of the Sickness Impact Profile (SIP; p < 0.05), had improved ambulation as measured by the 6-minute walk (p < 0.01), and lost more body weight (p < .001). No differences related to psychosocial factors as measured by the SIP were noted. A self-regulation-based program that was provided to older women with heart disease and that focused on physical activity and disease management problems salient to them, improved their physical functioning and symptom experience. Psychosocial benefit was not evident and may be a result of measurement error or due to insufficient program time spent on psychosocial aspects of functioning.

Psychometric characteristics of process evaluation measures for a rural school-based childhood obesity prevention study: Louisiana Health.

PubMed

Newton, Robert L; Thomson, Jessica L; Rau, Kristi K; Ragusa, Shelly A; Sample, Alicia D; Singleton, Nakisha N; Anton, Stephen D; Webber, Larry S; Williamson, Donald A

2011-01-01

To evaluate the implementation of intervention components of the Louisiana Health study, which was a multicomponent childhood obesity prevention program conducted in rural schools. Content analysis. Process evaluation assessed implementation in classrooms, gym classes, and cafeterias. Classroom teachers (n  =  232), physical education teachers (n  =  53), food service managers (n  =  33), and trained observers (n  =  9). Five process evaluation measures were created: Physical Education Questionnaire (PEQ), Intervention Questionnaire (IQ), Food Service Manager Questionnaire (FSMQ), Classroom Observation (CO), and School Nutrition Environment Observation (SNEO). Interrater reliability and internal consistency were assessed on all measures. Analysis of variance and χ(2) were used to compare differences across study groups on questionnaires and observations. The PEQ and one subscale from the FSMQ were eliminated because their reliability coefficients fell below acceptable standards. The subscale internal consistencies for the IQ, FSMQ, CO, and SNEO (all Cronbach α > .60) were acceptable. After the initial 4 months of intervention, there was evidence that the Louisiana Health intervention was being implemented as it was designed. In summary, four process evaluation measures were found to be sufficiently reliable and valid for assessing the delivery of various aspects of a school-based obesity prevention program. These process measures could be modified to evaluate the delivery of other similar school-based interventions.

Assessing the Effectiveness of a School-Based Dental Clinic on the Oral Health of Children Who Lack Access to Dental Care: A Program Evaluation

ERIC Educational Resources Information Center

Carpino, Rachel; Walker, Mary P.; Liu, Ying; Simmer-Beck, Melanie

2017-01-01

This program evaluation examines the effectiveness of a school-based dental clinic. A repeated-measures design was used to longitudinally examine secondary data from participants (N = 293). Encounter intensity was developed to normalize data. Multivariate analysis of variance and Kruskal-Wallis test were used to investigate the effect of encounter…

Measurement and Modeling of Fugitive Dust from Off Road DoD Activities

DTIC Science & Technology

2017-12-08

each soil and vehicle type (see Table 2). Note, no tracked vehicles were run at YTC. CT is the curve track sampling location, CR is the curve ridge...Soil is SL = sandy loam. ...................... 116 Figure 35. Single-event Wind Erosion Evaluation Program (SWEEP) Run example results. ... 121...Figure 36. Single-event Wind Erosion Evaluation Program (SWEEP) Threshold Run example results screen

An Analysis of Test And Evaluation in Rapid Acquisition Programs

DTIC Science & Technology

2015-12-01

program manager is assigned the requirement and allocated resources to carry out with an acquisition plan. 2. Role of Test and Evaluation Test and...Manual, verify the Maintenance Allocation Chart, and ensure the completeness of the System Support Package. The Maintainability Demonstration measured...additional outside personnel that require additional logistics support, such as security, facilitates, and berthing placing additional strain on units. CLS

Development and Evaluation of a Success Index for Professionals in Postgraduate Training Programs

DTIC Science & Technology

1993-02-26

15 Predicting Success among Program Participants .... ......... .. 16 AEGD Success and Career Success .......... ................ .. 16...10), and general career success (8). Hough applied the principle of behavioral consistency and aspects of the biographical inventory to develop and...the opportunity to evaluate how measures of success in AEGD translate into career success . The 90 AERs were reviewed by two experienced senior dental

Los Angeles Community College District Campus Child Development Centers' Measures of Effectiveness Project. Second Evaluation Report, Fiscal Year 1983-1984.

ERIC Educational Resources Information Center

Carfagna-Hunt, Karen; And Others

The Los Angeles Community College District (LACCD) provides a Campus Child Development Center (CCDC) educational program for the preschool children of its college students at all nine of its campuses. In 1983-84, on-site, peer review team visits were conducted to evaluate the centers in terms of their achievement of six program objectives. The…

Evaluating the effectiveness of psychosocial resilience training for heart health, and the added value of promoting physical activity: a cluster randomized trial of the READY program.

PubMed

Burton, Nicola W; Pakenham, Kenneth I; Brown, Wendy J

2009-11-23

Depression and poor social support are significant risk factors for coronary heart disease (CHD), and stress and anxiety can trigger coronary events. People experiencing such psychosocial difficulties are more likely to be physically inactive, which is also an independent risk factor for CHD. Resilience training can target these risk factors, but there is little research evaluating the effectiveness of such programs. This paper describes the design and measures of a study to evaluate a resilience training program (READY) to promote psychosocial well-being for heart health, and the added value of integrating physical activity promotion. In a cluster randomized trial, 95 participants will be allocated to either a waitlist or one of two intervention conditions. Both intervention conditions will receive a 10 x 2.5 hour group resilience training program (READY) over 13 weeks. The program targets five protective factors identified from empirical evidence and analyzed as mediating variables: positive emotions, cognitive flexibility, social support, life meaning, and active coping. Resilience enhancement strategies reflect the six core Acceptance and Commitment Therapy processes (values, mindfulness, defusion, acceptance, self-as-context, committed action) and Cognitive Behavior Therapy strategies such as relaxation training and social support building skills. Sessions include psychoeducation, discussions, experiential exercises, and home assignments. One intervention condition will include an additional session and ongoing content promoting physical activity. Measurement will occur at baseline, two weeks post intervention, and at eight weeks follow-up, and will include questionnaires, pedometer step logs, and physical and hematological measures. Primary outcome measures will include self-reported indicators of psychosocial well-being and depression. Secondary outcome measures will include self-reported indicators of stress, anxiety and physical activity, and objective indicators of CHD risk (blood glucose, cholesterol [mmol.L-1], triglycerides, blood pressure). Process measures of attendance, engagement and fidelity will also be conducted. Linear analyses will be used to examine group differences in the outcome measures, and the product of coefficients method will be used to examine mediated effects. If successful, this program will provide an innovative means by which to promote psychosocial well-being for heart health in the general population. The program could also be adapted to promote well-being in other at risk population subgroups. ACTRN12608000017325.

Impact of a provider training program on the treatment of children with autism spectrum disorder at psychosocial care units in Brazil.

PubMed

Silva, Luciana C; Teixeira, Maria C T V; Ribeiro, Edith L; Paula, Cristiane S

2017-12-18

To develop, implement, and verify the impact of a training program for health care providers working with children with autism spectrum disorder (ASD) in psychosocial care centers for children and adolescents (Centro de Atenção Psicossocial à Infância e à Adolescência - CAPSi) in São Paulo, Brazil. This quasi-experimental study was conducted with 14 professionals from four CAPSi units. The training program consisted of six phases: 1) pre-intervention observation; 2) meeting with staff to assess the main needs of the training program; 3) developing materials for training and evaluation; 4) meetings to discuss program implementation; 5) a final meeting for case discussion and evaluation; and 6) distance supervision. Three measures were used to evaluate the training program: i) the Knowledge, Attitudes, and Practices (KAP) questionnaire; ii) videos containing questions designed to assess program comprehension; and iii) a satisfaction survey. Thirteen videos were produced to as visual aids for use during the training program, and a further 26 videos were developed to evaluate it. The program was well evaluated by the participants. The video responses and KAP questionnaire scores suggest that staff knowledge and attitudes improved after training. The positive findings of this study suggest that the tested training program is feasible for use with multidisciplinary teams working in the CAPSi environment.

Lanl Neutron-Induced Fission Cross Section Measurement Program

NASA Astrophysics Data System (ADS)

Laptev, A. B.; Tovesson, F.; Hill, T. S.

2014-09-01

A well established program of neutron-induced fission cross section measurement at Los Alamos Neutron Science Center (LANSCE) is supporting the Fuel Cycle Research program (FC R&D). Combining measurements at two LANSCE facilities, the Lujan Center and the Weapons Neutron Research facility (WNR), cover neutron energies over 10 orders of magnitude: from sub-thermal up to 200 MeV. A parallel-plate fission ionization chamber was used as a fission fragment detector. The 235U(n,f) standard was used as the reference. Fission cross sections have been measured for multiple actinides. The new data presented here completes the suite of long-lived Uranium isotopes that were investigated with this experimental approach. The cross section data are presented in comparison with existing evaluations and previous measurements.

Benchmark Evaluation of HTR-PROTEUS Pebble Bed Experimental Program

DOE PAGES

Bess, John D.; Montierth, Leland; Köberl, Oliver; ...

2014-10-09

Benchmark models were developed to evaluate 11 critical core configurations of the HTR-PROTEUS pebble bed experimental program. Various additional reactor physics measurements were performed as part of this program; currently only a total of 37 absorber rod worth measurements have been evaluated as acceptable benchmark experiments for Cores 4, 9, and 10. Dominant uncertainties in the experimental keff for all core configurations come from uncertainties in the ²³⁵U enrichment of the fuel, impurities in the moderator pebbles, and the density and impurity content of the radial reflector. Calculations of k eff with MCNP5 and ENDF/B-VII.0 neutron nuclear data aremore » greater than the benchmark values but within 1% and also within the 3σ uncertainty, except for Core 4, which is the only randomly packed pebble configuration. Repeated calculations of k eff with MCNP6.1 and ENDF/B-VII.1 are lower than the benchmark values and within 1% (~3σ) except for Cores 5 and 9, which calculate lower than the benchmark eigenvalues within 4σ. The primary difference between the two nuclear data libraries is the adjustment of the absorption cross section of graphite. Simulations of the absorber rod worth measurements are within 3σ of the benchmark experiment values. The complete benchmark evaluation details are available in the 2014 edition of the International Handbook of Evaluated Reactor Physics Benchmark Experiments.« less

Evaluating effects of developmental education for college students using a regression discontinuity design.

PubMed

Moss, Brian G; Yeaton, William H

2013-10-01

Annually, American colleges and universities provide developmental education (DE) to millions of underprepared students; however, evaluation estimates of DE benefits have been mixed. Using a prototypic exemplar of DE, our primary objective was to investigate the utility of a replicative evaluative framework for assessing program effectiveness. Within the context of the regression discontinuity (RD) design, this research examined the effectiveness of a DE program for five, sequential cohorts of first-time college students. Discontinuity estimates were generated for individual terms and cumulatively, across terms. Participants were 3,589 first-time community college students. DE program effects were measured by contrasting both college-level English grades and a dichotomous measure of pass/fail, for DE and non-DE students. Parametric and nonparametric estimates of overall effect were positive for continuous and dichotomous measures of achievement (grade and pass/fail). The variability of program effects over time was determined by tracking results within individual terms and cumulatively, across terms. Applying this replication strategy, DE's overall impact was modest (an effect size of approximately .20) but quite consistent, based on parametric and nonparametric estimation approaches. A meta-analysis of five RD results yielded virtually the same estimate as the overall, parametric findings. Subset analysis, though tentative, suggested that males benefited more than females, while academic gains were comparable for different ethnicities. The cumulative, within-study comparison, replication approach offers considerable potential for the evaluation of new and existing policies, particularly when effects are relatively small, as is often the case in applied settings.

Engineering evaluation of SSME dynamic data from engine tests and SSV flights

NASA Technical Reports Server (NTRS)

1986-01-01

An engineering evaluation of dynamic data from SSME hot firing tests and SSV flights is summarized. The basic objective of the study is to provide analyses of vibration, strain and dynamic pressure measurements in support of MSFC performance and reliability improvement programs. A brief description of the SSME test program is given and a typical test evaluation cycle reviewed. Data banks generated to characterize SSME component dynamic characteristics are described and statistical analyses performed on these data base measurements are discussed. Analytical models applied to define the dynamic behavior of SSME components (such as turbopump bearing elements and the flight accelerometer safety cut-off system) are also summarized. Appendices are included to illustrate some typical tasks performed under this study.

Recommendations From the International Colorectal Cancer Screening Network on the Evaluation of the Cost of Screening Programs.

PubMed

Subramanian, Sujha; Tangka, Florence K L; Hoover, Sonja; Nadel, Marion; Smith, Robert; Atkin, Wendy; Patnick, Julietta

2016-01-01

Worldwide, colorectal cancer is the fourth leading cause of death from cancer and the incidence is projected to increase. Many countries are exploring the introduction of organized screening programs, but there is limited information on the resources required and guidance for cost-effective implementation. To facilitate the generating of the economics evidence base for program implementation, we collected and analyzed detailed program cost data from 5 European members of the International Colorectal Cancer Screening Network. The cost per person screened estimates, often used to compare across programs as an overall measure, varied significantly across the programs. In addition, there were substantial differences in the programmatic and clinical cost incurred, even when the same type of screening test was used. Based on these findings, several recommendations are provided to enhance the underlying methodology and validity of the comparative economic assessments. The recommendations include the need for detailed activity-based cost information, the use of a comprehensive set of effectiveness measures to adequately capture differences between programs, and the incorporation of data from multiple programs in cost-effectiveness models to increase generalizability. Economic evaluation of real-world colorectal cancer-screening programs is essential to derive valuable insights to improve program operations and ensure optimal use of available resources.

Function library programming to support B89 evaluation of Sheffield Apollo RS50 DCC (Direct Computer Control) CMM (Coordinate Measuring Machine)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Frank, R.N.

1990-02-28

The Inspection Shop at Lawrence Livermore Lab recently purchased a Sheffield Apollo RS50 Direct Computer Control Coordinate Measuring Machine. The performance of the machine was specified to conform to B89 standard which relies heavily upon using the measuring machine in its intended manner to verify its accuracy (rather than parametric tests). Although it would be possible to use the interactive measurement system to perform these tasks, a more thorough and efficient job can be done by creating Function Library programs for certain tasks which integrate Hewlett-Packard Basic 5.0 language and calls to proprietary analysis and machine control routines. This combinationmore » provides efficient use of the measuring machine with a minimum of keyboard input plus an analysis of the data with respect to the B89 Standard rather than a CMM analysis which would require subsequent interpretation. This paper discusses some characteristics of the Sheffield machine control and analysis software and my use of H-P Basic language to create automated measurement programs to support the B89 performance evaluation of the CMM. 1 ref.« less

Destructive analysis capabilities for plutonium and uranium characterization at Los Alamos National Laboratory

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tandon, Lav; Kuhn, Kevin J; Drake, Lawrence R

Los Alamos National Laboratory's (LANL) Actinide Analytical Chemistry (AAC) group has been in existence since the Manhattan Project. It maintains a complete set of analytical capabilities for performing complete characterization (elemental assay, isotopic, metallic and non metallic trace impurities) of uranium and plutonium samples in different forms. For a majority of the customers there are strong quality assurance (QA) and quality control (QC) objectives including highest accuracy and precision with well defined uncertainties associated with the analytical results. Los Alamos participates in various international and national programs such as the Plutonium Metal Exchange Program, New Brunswick Laboratory's (NBL' s) Safeguardsmore » Measurement Evaluation Program (SME) and several other inter-laboratory round robin exercises to monitor and evaluate the data quality generated by AAC. These programs also provide independent verification of analytical measurement capabilities, and allow any technical problems with analytical measurements to be identified and corrected. This presentation will focus on key analytical capabilities for destructive analysis in AAC and also comparative data between LANL and peer groups for Pu assay and isotopic analysis.« less

Using archived data to measure operational benefits of ITS investments, volume 1 : ramp meters.

DOT National Transportation Integrated Search

2004-06-01

develop two case study evaluations for Oregon, including an evaluation of the COMET incident management program and the : Portland ramp metering system. The COMET evaluation will include a thorough literature review of other incident management : pro...

Demand generation activities and modern contraceptive use in urban areas of four countries: a longitudinal evaluation

PubMed Central

Speizer, Ilene S; Corroon, Meghan; Calhoun, Lisa; Lance, Peter; Montana, Livia; Nanda, Priya; Guilkey, David

2014-01-01

ABSTRACT Family planning is crucial for preventing unintended pregnancies and for improving maternal and child health and well-being. In urban areas where there are large inequities in family planning use, particularly among the urban poor, programs are needed to increase access to and use of contraception among those most in need. This paper presents the midterm evaluation findings of the Urban Reproductive Health Initiative (Urban RH Initiative) programs, funded by the Bill & Melinda Gates Foundation, that are being implemented in 4 countries: India (Uttar Pradesh), Kenya, Nigeria, and Senegal. Between 2010 and 2013, the Measurement, Learning & Evaluation (MLE) project collected baseline and 2-year longitudinal follow-up data from women in target study cities to examine the role of demand generation activities undertaken as part of the Urban RH Initiative programs. Evaluation results demonstrate that, in each country where it was measured, outreach by community health or family planning workers as well as local radio programs were significantly associated with increased use of modern contraceptive methods. In addition, in India and Nigeria, television programs had a significant effect on modern contraceptive use, and in Kenya and Nigeria, the program slogans and materials that were blanketed across the cities (eg, leaflets/brochures distributed at health clinics and the program logo placed on all forms of materials, from market umbrellas to health facility signs and television programs) were also significantly associated with modern method use. Our results show that targeted, multilevel demand generation activities can make an important contribution to increasing modern contraceptive use in urban areas and could impact Millennium Development Goals for improved maternal and child health and access to reproductive health for all. PMID:25611476

Changes in University Students after Joining a Service Leadership Program in China

ERIC Educational Resources Information Center

Shek, Daniel T. L.; Lin, Li

2016-01-01

This study examined the effectiveness of a 4.5-day service leadership program for students from Chinese universities using objective outcome evaluation. The participants were assessed before and after the program, with two post-test measurements (immediate assessment and assessment 12 days after the completion of class learning). At pretest and…

An Abstinence Program's Impact on Cognitive Mediators and Sexual Initiation

ERIC Educational Resources Information Center

Weed, Stan E.; Ericksen, Irene H.; Lewis, Allen; Grant, Gale E.; Wibberly, Kathy H.

2008-01-01

Objectives: To evaluate the impact of an abstinence education program on sexual intercourse initiation and on possible cognitive mediators of sexual initiation for virgin seventh graders in suburban Virginia. Methods: Measures of sexual behavior and 6 mediating variables were compared at 3 time periods for program participants and a matched…

A Conceptual Three-Dimensional Model for Evaluating Community-Based Substance Abuse Prevention Programs.

ERIC Educational Resources Information Center

Albers, Eric C.; Santangelo, Linda K.; McKinlay, George; Cavote, Steve; Rock, Stephen L.; Evans, William

2002-01-01

Presents a three-dimensional model for conceptualizing existing prevention programs, defining and measuring effects of prevention programs, and making a connection between those programmatic effects, and the interests of the funder. This paper describes the methodology and its use for promoting the efficiency and effectiveness of substance abuse…

77 FR 53257 - Medicare Program; Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-31

... Electronic Device (CIED) Procedures c. New Candidate HAC Condition: Iatrogenic Pneumothorax With Venous Catheterization 6. RTI Program Evaluation Summary a. RTI Analysis of FY 2011 POA Indicator Reporting Across.... Hospital Inpatient Quality Reporting (IQR) Program 1. Background a. History of Measures Adopted for the...

Outcomes of Parent Education Programs Based on Reevaluation Counseling

ERIC Educational Resources Information Center

Wolfe, Randi B.; Hirsch, Barton J.

2003-01-01

We report two studies in which a parent education program based on Reevaluation Counseling was field-tested on mothers randomly assigned to treatment groups or equivalent, no-treatment comparison groups. The goal was to evaluate the program's viability, whether there were measurable effects, whether those effects were sustained over time, and…

EVALUATING PROGRAMMED TEST INTERPRETATION USING EMOTIONAL AROUSAL AS A CRITERION.

ERIC Educational Resources Information Center

FORSTER, JERALD R.

THE QUESTION AS TO WHETHER OR NOT THE EMOTIONAL ASPECTS OF TEST INTERPRETATION MAKE IT INAPPROPRIATE FOR PROGRAMED METHODS WAS EXAMINED BY TWO METHODS OF TEST RESULT COMMUNICATION--PROGRAMED MATERIALS AND VERBAL COMMUNICATION BY A COUNSELOR. DEPENDENT VARIABLES WERE MEASURES OF EMOTIONAL AROUSAL RECORDED BY SKIN CONDUCTANCE UNITS AND THE GAIN IN…

Developing Mathematical Thinking: Changing Teachers' Knowledge and Instruction

ERIC Educational Resources Information Center

Brendefur, Jonathan L.; Thiede, Keith; Strother, Sam; Bunning, Kim; Peck, Duane

2013-01-01

In the present research, we evaluated the effectiveness of a multi-year professional development program in mathematics for elementary teachers. Each year the program focused on a different domain of mathematics. We found the program increased teachers' knowledge of (a) number and operations, (b) measurement and geometry, and (c) probability and…

4-H Night at the Movies: A Program for Adolescents and Their Families.

ERIC Educational Resources Information Center

Jurich, Anthony P.; Collins, Olivia P.

1996-01-01

Evaluates a program designed to improve self-esteem in adolescents. Used commercial movies about teenagers to enhance adolescents' interest and understanding of their world. Findings indicated significant increases in measures of self concept, self satisfaction, family self, and social self over the course of the program. (RJM)

Attitudes of Disabled Individuals Enrolled in a Workstation Center and a Community College Program

ERIC Educational Resources Information Center

Wada, Michael A.; Brodwin, Martin

1975-01-01

Measured attitudes of disabled individuals toward the staff who gave them rehabilitation services. Attitudes of two groups at a rehabilitation hospital were evaluated: (a) clients from a vocational rehabilitation workshop and (b) clients from a community college satellite program. Found clients in the community college program perceived staff as…

34 CFR 403.204 - What are the State's responsibilities for program evaluation and improvement?

Code of Federal Regulations, 2011 CFR

2011-07-01

... has not made sufficient progress in meeting the standards and measures developed as required by §§ 403... technical assistance and program activities the State will provide to enhance the performance of the...

National Weatherization Assistance Program Impact Evaluation - Client Satisfaction Survey: WAP Service Delivery from the Client's Perspective

DOE Office of Scientific and Technical Information (OSTI.GOV)

Miller, Carolyn; Carroll, David; Berger, Jacqueline

This report presents the results of a survey of recipients to measure satisfaction with services provided by local weatherization agencies being supported by funding from Department of Energy's Weatherization Assistance Program.

Instrumentation of the Red River Bridge at Boyce, Louisiana : final report.

DOT National Transportation Integrated Search

1991-01-01

The report describes the instrumentation program of Red River Bridge at Boyce, Louisiana. The objectives of the program were to measure and evaluate time-dependent deformations, deflections, and temperatures of the Red River Bridge superstructure. To...

Instrumentation of the Red River Bridge at Boyce, Louisiana : final report.

DOT National Transportation Integrated Search

1988-08-01

The report describes the instrumentation program of Red River Bridge at Boyce, Louisiana. The objectives of the program were to measure and evaluate time-dependent deformations, deflections, and temperatures of the Red River Bridge superstructure. To...

ENVIRONMENTAL HELATH INDICATORS-DIVERSE PROGRAMS, COMMON METHODOLOGICAL CHALLENGES

EPA Science Inventory

Abstract: Environmental Health Indicators are quantitative measures of important environmental health factors that can be used to identify specific areas and populations for intervention and prevention efforts and to evaluate the outcomes of implemented policies or programs for ...

Genetic programming approach to evaluate complexity of texture images

NASA Astrophysics Data System (ADS)

Ciocca, Gianluigi; Corchs, Silvia; Gasparini, Francesca

2016-11-01

We adopt genetic programming (GP) to define a measure that can predict complexity perception of texture images. We perform psychophysical experiments on three different datasets to collect data on the perceived complexity. The subjective data are used for training, validation, and test of the proposed measure. These data are also used to evaluate several possible candidate measures of texture complexity related to both low level and high level image features. We select four of them (namely roughness, number of regions, chroma variance, and memorability) to be combined in a GP framework. This approach allows a nonlinear combination of the measures and could give hints on how the related image features interact in complexity perception. The proposed complexity measure M exhibits Pearson correlation coefficients of 0.890 on the training set, 0.728 on the validation set, and 0.724 on the test set. M outperforms each of all the single measures considered. From the statistical analysis of different GP candidate solutions, we found that the roughness measure evaluated on the gray level image is the most dominant one, followed by the memorability, the number of regions, and finally the chroma variance.

The social construction of "evidence-based'' drug prevention programs: a reanalysis of data from the Drug Abuse Resistance Education (DARE) program.

PubMed

Gorman, Dennis M; Huber, J Charles

2009-08-01

This study explores the possibility that any drug prevention program might be considered ;;evidence-based'' given the use of data analysis procedures that optimize the chance of producing statistically significant results by reanalyzing data from a Drug Abuse Resistance Education (DARE) program evaluation. The analysis produced a number of statistically significant differences between the DARE and control conditions on alcohol and marijuana use measures. Many of these differences occurred at cutoff points on the assessment scales for which post hoc meaningful labels were created. Our results are compared to those from evaluations of programs that appear on evidence-based drug prevention lists.

A new version of the helicopter aural detection program, ICHIN

NASA Technical Reports Server (NTRS)

Mueller, A. W.; Smith, C. D.; Shepherd, K. P.; Sullivan, B. M.

1986-01-01

NASA Langley Research Center personnel have conducted an evaluation of the helicopter aural detection program I Can Hear It Now (ICHIN version-5). This was accomplished using flight noise data of five helicopters, obtained from a joint NASA and U.S. Army acoustics measurement program. The evaluation consisted of presenting the noise data to a jury of 20 subjects and to the ICHIN-5 program. A comparative study was then made of the detection distances determined by the jury and predicted by ICHIN-5. This report presents the changes made in the ICHIN-5 program as a result of this comparative study. The changes represent current psychoacoustics and propagation knowledge.

Survey of university programs in remote sensing funded under grants from the NASA University-Space Applications program

NASA Technical Reports Server (NTRS)

Madigan, J. A.; Earhart, R. W.

1978-01-01

NASA's Office of Space and Terrestrial Applications (OSTA) is currently assessing approaches to transferring NASA technology to both the public and private sectors. As part of this assessment, NASA is evaluating the effectiveness of an ongoing program in remote sensing technology transfer conducted by 20 university contractors/grantees, each supported totally or partially by NASA funds. The University-Space Applications program has as its objective the demonstration of practical benefits from the use of remote sensing technology to a broad spectrum of new users, principally in state and local governments. To evaluate the University-Space Applications program, NASA has a near-term requirement for data on each university effort including total funding, funding sources, length of program, program description, and effectiveness measures.

Measuring the implementation of early childhood development programs.

PubMed

Aboud, Frances E; Prado, Elizabeth L

2018-05-01

In this paper we describe ways to measure variables of interest when evaluating the implementation of a program to improve early childhood development (ECD). The variables apply to programs delivered to parents in group sessions and home or clinic visits, as well as in early group care for children. Measurements for four categories of variables are included: training and assessment of delivery agents and supervisors; program features such as quality of delivery, reach, and dosage; recipients' acceptance and enactment; and stakeholders' engagement. Quantitative and qualitative methods are described, along with when measures might be taken throughout the processes of planning, preparing, and implementing. A few standard measures are available, along with others that researchers can select and modify according to their goals. Descriptions of measures include who might collect the information, from whom, and when, along with how information might be analyzed and findings used. By converging on a set of common methods to measure implementation variables, investigators can work toward improving programs, identifying gaps that impede the scalability and sustainability of programs, and, over time, ascertain program features that lead to successful outcomes. © 2018 The Authors. Annals of the New York Academy of Sciences published by Wiley Periodicals, Inc. on behalf of New York Academy of Sciences.

Programs to improve adolescent sexual and reproductive health in the US: a review of the evidence.

PubMed

Manlove, Jennifer; Fish, Heather; Moore, Kristin Anderson

2015-01-01

US adolescents have high rates of teen pregnancy, childbearing, and sexually transmitted infections (STIs), highlighting the need to identify and implement effective programs that will help improve teen sexual and reproductive health. This review identified 103 random-assignment evaluations of 85 programs that incorporated intent-to-treat analyses and assessed impacts on pregnancy, childbearing, STIs, and their key determinants - sexual activity, number of sexual partners, condom use, and other contraceptive use - among teens. This review describes the evidence base for five broad program approaches, including abstinence education, comprehensive sex education, clinic-based programs, youth development programs, and parent-youth relationship programs. We also describe programs with impacts on key outcomes, including pregnancy/childbearing, STIs, and those that found impacts on both sexual activity and contraceptive use. Our review identified 52 effective programs: 38 with consistent impacts on reproductive health outcomes, and 14 with mixed findings (across subpopulations, follow-ups, or multiple measures of a single outcome). We found that a variety of program approaches produced impacts on sexual and reproductive health outcomes. Parent-youth relationship programs and clinic-based program evaluations more frequently showed impacts than other program approaches, although we also identified a number of abstinence-education, comprehensive sex education, and youth-development programs with impacts on sexual and reproductive health outcomes. Overall, we identified nine program evaluations with impacts on teen pregnancies or births, five with impacts on reducing STIs, and 15 with impacts on both delaying/reducing sexual activity and increasing contraceptive use (including condom use). Future efforts should conduct replications of existing program evaluations, identify implementation components linked to impacts, rigorously evaluate programs that appear promising, and expand the evidence base on programs that impact hormonal and long-acting contraceptive method use.

Development of a practice-based research program.

PubMed

Hawk, C; Long, C R; Boulanger, K

1998-01-01

To establish an infrastructure to collect accurate data from ambulatory settings. The program was developed through an iterative model governed by a process of formative evaluation. The three iterations were a needs assessment, feasibility study and pilot project. Necessary program components were identified as infrastructure, practitioner-researcher partnership, centralized data management and standardized quality assurance measures. Volunteer chiropractors and their staff collected data on patients in their practices in ambulatory settings in the U.S. and Canada. Evaluative measures were counts of participants, patients and completed forms. Standardized, validated and reliable measures collected by patient self-report were used to assess treatment outcomes. These included the SF-36 or SF-12 Health Survey, the Pain Disability Index, and the Global Well-Being Scale. For characteristics for which appropriate standardized instruments were not available, questionnaires were designed and and pilot-tested before use. Information was gathered on practice and patient characteristics and treatment outcomes, but for this report, only those data concerning process evaluation are reported. Through the three program iterations, 65 DCs collected data on 1360 patients, 663 of whom were new patients. Follow-up data recorded by doctors were obtained for more than 70% of patients; a maximum of 50% of patient-completed follow-up forms were collected in the three iterations. This program is capable of providing data for descriptive epidemiology of ambulatory patients, and, with continued effort to maximize follow-up, may have utility in providing insight into utilization patterns and patient outcomes.

Measuring sustainment of prevention programs and initiatives: a study protocol.

PubMed

Palinkas, Lawrence A; Spear, Suzanne E; Mendon, Sapna J; Villamar, Juan; Valente, Thomas; Chou, Chi-Ping; Landsverk, John; Kellam, Shepperd G; Brown, C Hendricks

2016-07-16

Sustaining prevention efforts directed at substance use and mental health problems is one of the greatest, yet least understood, challenges in the field of implementation science. A large knowledge gap exists regarding the meaning of the term "sustainment" and what factors predict or even measure sustainability of effective prevention programs and support systems. The U.S. Substance Abuse and Mental Health Services Administration (SAMHSA) supports a diverse portfolio of prevention and treatment grant programs that aim to improve population and individual level behavioral health. This study focuses on four SAMHSA prevention grant programs, two of which target substance abuse prevention at the state or single community level, one targets suicide prevention, and one targets prevention of aggressive/disruptive behavior in elementary schools. An examination of all four grant programs simultaneously provides an opportunity to determine what is meant by the term sustainment and identify and support both the unique requirements for improving sustainability for each program as well as for developing a generalizable framework comprised of core components of sustainment across diverse prevention approaches. Based on an analysis of qualitative and quantitative data of 10 grantees supported by these four programs, we will develop a flexible measurement system, with both general and specific components, that can bring precision to monitoring sustainment of infrastructure, activities, and outcomes for each prevention approach. We will then transform this system for use in evaluating and improving the likelihood of achieving prevention effort sustainment. To achieve these goals, we will (1) identify core components of sustainment of prevention programs and their support infrastructures; (2) design a measurement system for monitoring and providing feedback regarding sustainment within the four SAMHSA's prevention-related grant programs; and (3) pilot test the predictability of this multilevel measurement system across these programs and the feasibility and acceptability of a measurement system to evaluate and improve the likelihood of sustainment. This project is intended to improve sustainment of the supporting prevention infrastructure, activities, and outcomes that are funded by federal, state, community, and foundation sources.

Evaluation of a Brazilian Postgraduate Dental Program by the Dundee Ready Educational Environment Measure

ERIC Educational Resources Information Center

do Vale Placa, Rebeca; Ragghianti Zangrando, Mariana S.; Sant'Ana, Adriana C. P.; Greghi, Sebastião L. A.; de Rezende, Maria Lucia R.; Damante, Carla A.

2015-01-01

The evaluation of education environment is essential to provide to the professors a better understanding of the teaching process. One valuable tool for this assessment is the Dundee Ready Educational Environment Measure (DREEM). This questionnaire has 50 questions and is divided in five dimensions: D1--Perceptions of teaching, D2--Perceptions of…

Preservice Teacher and Interpreter American Sign Language Abilities: Self-Evaluations and Evaluations of Deaf Students' Narrative Renditions

ERIC Educational Resources Information Center

Beal-Alvarez, Jennifer S.; Scheetz, Nanci A.

2015-01-01

In deaf education, the sign language skills of teacher and interpreter candidates are infrequently assessed; when they are, formal measures are commonly used upon preparation program completion, as opposed to informal measures related to instructional tasks. Using an informal picture storybook task, the authors investigated the receptive and…

Developing Common Measures in Evaluation Capacity Building: An Iterative Science and Practice Process

ERIC Educational Resources Information Center

Labin, Susan N.

2014-01-01

A fundamental reason for doing evaluation capacity building (ECB) is to improve program outcomes. Developing common measures of outcomes and the activities, processes, and factors that lead to these outcomes is an important step in moving the science and the practice of ECB forward. This article identifies a number of existing ECB measurement…

The Impact of the Norton High School Early College Program on the Academic Performance of Students at Norton High School

ERIC Educational Resources Information Center

Barba, Eric Matthew

2012-01-01

The purpose of this study was to evaluate the effect of the Norton High School Early College Early College Program on academic measures for students at Norton High School. Measures of achievement include the results of the English Language Arts (ELA), Mathematics, Social Science, and Science portions of the California Standards Test (CST), Student…

Dimensions of Academic Growth and Development During College: Using Alumni Reports to Evaluate Education Programs. ASHE Annual Meeting Paper.

ERIC Educational Resources Information Center

Pike, Gary R.

This study attempted to validate the use of academic growth and development items from Tennessee alumni surveys as measures of program quality and effectiveness at the University of Tennessee (UTK), Knoxville. The argument is made that it is essential that the instruments used to assess students educational outcomes be valid measures of the goals…

Development and application of course-embedded assessment system for program outcome evaluation in the Korean nursing education: A pilot study.

PubMed

Park, Jee Won; Seo, Eun Ji; You, Mi-Ae; Song, Ju-Eun

2016-03-01

Program outcome evaluation is important because it is an indicator for good quality of education. Course-embedded assessment is one of the program outcome evaluation methods. However, it is rarely used in Korean nursing education. The study purpose was to develop and apply preliminarily a course-embedded assessment system to evaluate one program outcome and to share our experiences. This was a methodological study to develop and apply the course-embedded assessment system based on the theoretical framework in one nursing program in South Korea. Scores for 77 students generated from the three practicum courses were used. The course-embedded assessment system was developed following the six steps suggested by Han's model as follows. 1) One program outcome in the undergraduate program, "nursing process application ability", was selected and 2) the three clinical practicum courses related to the selected program outcome were identified. 3) Evaluation tools including rubric and items were selected for outcome measurement and 4) performance criterion, the educational goal level for the program, was established. 5) Program outcome was actually evaluated using the rubric and evaluation items in the three practicum courses and 6) the obtained scores were analyzed to identify the achievement rate, which was compared with the performance criterion. Achievement rates for the selected program outcome in adult, maternity, and pediatric nursing practicum were 98.7%, 100%, and 66.2% in the case report and 100% for all three in the clinical practice, and 100%, 100%, and 87% respectively for the conference. These are considered as satisfactory levels when compared with the performance criterion of "at least 60% or more". Course-embedded assessment can be used as an effective and economic method to evaluate the program outcome without running an integrative course additionally. Further studies to develop course-embedded assessment systems for other program outcomes in nursing education are needed. Copyright © 2016 Elsevier Ltd. All rights reserved.

Evaluation of a family-centered prevention intervention for military children and families facing wartime deployments.

PubMed

Lester, Patricia; Saltzman, William R; Woodward, Kirsten; Glover, Dorie; Leskin, Gregory A; Bursch, Brenda; Pynoos, Robert; Beardslee, William

2012-03-01

We evaluated the Families OverComing Under Stress program, which provides resiliency training designed to enhance family psychological health in US military families affected by combat- and deployment-related stress. We performed a secondary analysis of Families OverComing Under Stress program evaluation data that was collected between July 2008 and February 2010 at 11 military installations in the United States and Japan. We present data at baseline for 488 unique families (742 parents and 873 children) and pre-post outcomes for 331 families. Family members reported high levels of satisfaction with the program and positive impact on parent-child indicators. Psychological distress levels were elevated for service members, civilian parents, and children at program entry compared with community norms. Change scores showed significant improvements across all measures for service member and civilian parents and their children (P < .001). Evaluation data provided preliminary support for a strength-based, trauma-informed military family prevention program to promote resiliency and mitigate the impact of wartime deployment stress.

The feasibility of a telephone coaching program on heart failure home management for family caregivers

PubMed Central

Piamjariyakul, Ubolrat; Smith, Carol E.; Russell, Christy; Werkowitch, Marilyn; Elyachar, Andrea

2012-01-01

Objectives To test the feasibility of delivery and evaluate the helpfulness of a coaching heart failure (HF) home management program for family caregivers. Background The few available studies on providing instruction for family caregivers are limited in content for managing HF home care and guidance for program implementation. Method This pilot study employed a mixed methods design. The measures of caregiver burden, confidence, and preparedness were compared at baseline and 3 months post-intervention. Descriptive statistics were used to summarize program costs and demographic data. Content analysis research methods were used to evaluate program feasibility and helpfulness. Results Caregiver (n=10) burden scores were significantly reduced and raw scores of confidence and preparedness for HF home management improved 3 months after the intervention. Content analyses of nurse and caregiver post-intervention data found caregivers rated the program as helpful and described how they initiated HF management skills based on the program. Conclusion The program was feasible to implement. These results suggest the coaching program should be further tested with a larger sample size to evaluate its efficacy. PMID:23116654

Organizational Wellness Program Implementation and Evaluation: A Holistic Approach to Improve the Wellbeing of Middle Managers.

PubMed

Medina, Maria Del Consuelo; Calderon, Angelica; Blunk, Dan I; Mills, Brandy W; Leiner, Marie

2018-06-01

: Employee wellness programs can provide benefits to institutions as well as employees and their families. Despite the attempts of some organizations to implement programs that take a holistic approach to improve physical, mental, and social wellness, the most common programs are exclusively comprised of physical and nutritional components. In this study, we implemented a wellness program intervention, including training using a holistic approach to improve the wellbeing of middle managers in several multinational organizations. We included control and experimental groups to measure wellness and teamwork with two repeated measures. Our results indicated that employees receiving the intervention had improved measures of wellness and teamwork. A positive relationship was found between wellness and teamwork in the experimental group when compared with the control group. Taken together, the data suggest that implementation of these programs would provide valuable outcomes for both employees and organizations.

Curriculum evaluation and revision in a nascent field: the utility of the retrospective pretest--posttest model in a homeland security program of study.

PubMed

Pelfrey, William V; Pelfrey, William V

2009-02-01

Although most academic disciplines evolve at a measured pace, the emerging field of homeland security must, for reasons of safety and security, evolve rapidly. The Department of Homeland Security sponsored the establishment of a graduate educational program for key officials holding homeland security roles. Because homeland security is a nascent field, the establishment of a program curriculum was forced to draw from a variety of disciplines. Curriculum evaluation was complicated by the rapid changes occurring in the emerging discipline, producing response shift bias, and interfering with the pre-post assessments. To compensate for the validity threat associated with response shift bias, a retrospective pretest-posttest evaluative methodology was used. Data indicate the program has evolved in a significant and orderly fashion and these data support the use of this innovative evaluation approach in the development of any discipline.

Verification approach for the Shuttle/Payload Contamination Evaluation computer program - Spacelab induced environment

NASA Technical Reports Server (NTRS)

Bareiss, L. E.

1978-01-01

The paper presents a compilation of the results of a systems level Shuttle/payload contamination analysis and related computer modeling activities. The current technical assessment of the contamination problems anticipated during the Spacelab program are discussed and recommendations are presented on contamination abatement designs and operational procedures based on experience gained in the field of contamination analysis and assessment, dating back to the pre-Skylab era. The ultimate test of the Shuttle/Payload Contamination Evaluation program will be through comparison of predictions with measured levels of contamination during actual flight.

Introduction of blended learning in a master program: Developing an integrative mixed method evaluation framework.

PubMed

Chmiel, Aviva S; Shaha, Maya; Schneider, Daniel K

2017-01-01

The aim of this research is to develop a comprehensive evaluation framework involving all actors in a higher education blended learning (BL) program. BL evaluation usually either focuses on students, faculty, technological or institutional aspects. Currently, no validated comprehensive monitoring tool exists that can support introduction and further implementation of BL in a higher education context. Starting from established evaluation principles and standards, concepts that were to be evaluated were firstly identified and grouped. In a second step, related BL evaluation tools referring to students, faculty and institutional level were selected. This allowed setting up and implementing an evaluation framework to monitor the introduction of BL during two succeeding recurrences of the program. The results of the evaluation allowed documenting strengths and weaknesses of the BL format in a comprehensive way, involving all actors. It has led to improvements at program, faculty and course level. The evaluation process and the reporting of the results proved to be demanding in time and personal resources. The evaluation framework allows measuring the most significant dimensions influencing the success of a BL implementation at program level. However, this comprehensive evaluation is resource intensive. Further steps will be to refine the framework towards a sustainable and transferable BL monitoring tool that finds a balance between comprehensiveness and efficiency. Copyright © 2016 Elsevier Ltd. All rights reserved.

Moving forward: using the experience of the CDCs' Colorectal Cancer Screening Demonstration Program to guide future colorectal cancer programming efforts.

PubMed

Seeff, Laura C; DeGroff, Amy; Joseph, Djenaba A; Royalty, Janet; Tangka, Florence K L; Nadel, Marion R; Plescia, Marcus

2013-08-01

The Centers for Disease Control and Prevention (CDC) established and supported a 4-year Colorectal Cancer Screening Demonstration Program (CRCSDP) from 2005 to 2009 for low-income, under- or uninsured men and women aged 50-64 at 5 sites in the United States. A multiple methods evaluation was conducted including 1) a longitudinal, comparative case study of program implementation, 2) the collection and analysis of client-level screening and diagnostic services outcome data, and 3) the collection and analysis of program- and patient-level cost data. Several themes emerged from the results reported in the series of articles in this Supplement. These included the benefit of building on an existing infrastructure, strengths and weakness of both the 2 most frequently used screening tests (colonoscopy and fecal occult blood tests), variability in costs of maintaining this screening program, and the importance of measuring the quality of screening tests. Population-level evaluation questions could not be answered because of the small size of the participating population and the limited time frame of the evaluation. The comprehensive evaluation of the program determined overall feasibility of this effort. Critical lessons learned through the implementation and evaluation of the CDC's CRCSDP led to the development of a larger population-based program, the CDC's Colorectal Cancer Control Program (CRCCP). © 2013 American Cancer Society.

Disease management 360 degrees: a scorecard approach to evaluating TRICARE's programs for asthma, congestive heart failure, and diabetes.

PubMed

Yang, Wenya; Dall, Timothy M; Zhang, Yiduo; Hogan, Paul F; Arday, David R; Gantt, Cynthia J

2010-08-01

To assess the effect of TRICARE's asthma, congestive heart failure, and diabetes disease management programs using a scorecard approach. EVALUATION MEASURES: Patient healthcare utilization, financial, clinical, and humanistic outcomes. Absolute measures were translated into effect size and incorporated into a scorecard. Actual outcomes for program participants were compared with outcomes predicted in the absence of disease management. The predictive equations were established from regression models based on historical control groups (n = 39,217). Z scores were calculated for the humanistic measures obtained through a mailed survey. Administrative records containing medical claims, patient demographics and characteristics, and program participation status were linked using an encrypted patient identifier (n = 57,489). The study time frame is 1 year prior to program inception through 2 years afterward (October 2005-September 2008). A historical control group was identified with the baseline year starting October 2003 and a 1-year follow-up period starting October 2004. A survey was administered to a subset of participants 6 months after baseline assessment (39% response rate). Within the observation window--24 months for asthma and congestive heart failure, and 15 months for the diabetes program--we observed modest reductions in hospital days and healthcare cost for all 3 programs and reductions in emergency visits for 2 programs. Most clinical outcomes moved in the direction anticipated. The scorecard provided a useful tool to track performance of 3 regional contractors for each of 3 diseases and over time.

Design and implementation of an integrated, continuous evaluation, and quality improvement system for a state-based home-visiting program.

PubMed

McCabe, Bridget K; Potash, Dru; Omohundro, Ellen; Taylor, Cathy R

2012-10-01

To describe the design and implementation of an evaluation system to facilitate continuous quality improvement (CQI) and scientific evaluation in a statewide home visiting program, and to provide a summary of the system's progress in meeting intended outputs and short-term outcomes. Help Us Grow Successfully (HUGS) is a statewide home visiting program that provides services to at-risk pregnant/post-partum women, children (0-5 years), and their families. The program goals are to improve parenting skills and connect families to needed services and thus improve the health of the service population. The evaluation system is designed to: (1) integrate evaluation into daily workflow; (2) utilize standardized screening and evaluation tools; (3) facilitate a culture of CQI in program management; and, (4) facilitate scientifically rigorous evaluations. The review of the system's design and implementation occurred through a formative evaluation process (reach, dose, and fidelity). Data was collected through electronic and paper surveys, administrative data, and notes from management meetings, and medical chart review. In the design phase, four process and forty outcome measures were selected and are tracked using standardized screening and monitoring tools. During implementation, the reach and dose of training were adequate to successfully launch the evaluation/CQI system. All staff (n = 165) use the system for management of families; the supervisors (n = 18) use the system to track routine program activities. Data quality and availability is sufficient to support periodic program reviews at the region and state level. In the first 7 months, the HUGS evaluation system tracked 3,794 families (7,937 individuals). System use and acceptance is high. A successful implementation of a structured evaluation system with a strong CQI component is feasible in an existing, large statewide program. The evaluation/CQI system is an effective mechanism to drive modest change in management of the program.

Measures of Success: A Survey of Selected Vocational and Technical Education Programs in Florida which Train Students in Occupations Requiring Licenses.

ERIC Educational Resources Information Center

Florida State Advisory Council on Vocational and Technical Education, Tallahassee.

A study of 13 vocational and technical education programs in Florida was conducted which represented an attempt to identify valid and reliable sources of data whereby some vocational programs might be evaluated in specified terms of effectiveness. The programs selected for study were among those which require graduating students to pass licensing…

WWC Quick Review of the Manuscript "Paying for A's: An Early Exploration of Student Reward and Incentive Programs in Charter Schools"

ERIC Educational Resources Information Center

What Works Clearinghouse, 2008

2008-01-01

This review examined a study designed to evaluate whether offering student reward and incentive programs in charter schools affects academic achievement. The study measured effects by comparing changes in average grade-level test scores in schools that had incentive programs to changes in schools that did not have incentive programs. The study…

Design of the Dutch Obesity Intervention in Teenagers (NRG-DOiT): systematic development, implementation and evaluation of a school-based intervention aimed at the prevention of excessive weight gain in adolescents

PubMed Central

Singh, Amika S; Chin A Paw, Marijke JM; Kremers, Stef PJ; Visscher, Tommy LS; Brug, Johannes; van Mechelen, Willem

2006-01-01

Background Only limited data are available on the development, implementation, and evaluation processes of weight gain prevention programs in adolescents. To be able to learn from successes and failures of such interventions, integral written and published reports are needed. Methods Applying the Intervention Mapping (IM) protocol, this paper describes the development, implementation, and evaluation of the Dutch Obesity Intervention in Teenagers (DOiT), a school-based intervention program aimed at the prevention of excessive weight gain. The intervention focussed on the following health behaviours: (1) reduction of the consumption of sugar-sweetened beverages, (2) reduction of energy intake derived from snacks, (3) decrease of levels of sedentary behaviour, and (4) increase of levels of physical activity (i.e. active transport behaviour and sports participation). The intervention program consisted of an individual classroom-based component (i.e. an educational program, covering 11 lessons of both biology and physical education classes), and an environmental component (i.e. encouraging and supporting changes at the school canteens, as well as offering additional physical education classes). We evaluated the effectiveness of the intervention program using a randomised controlled trial design. We assessed the effects of the intervention on body composition (primary outcome measure), as well as on behaviour, behavioural determinants, and aerobic fitness (secondary outcome measures). Furthermore, we conducted a process evaluation. Discussion The development of the DOiT-intervention resulted in a comprehensive school-based weight gain prevention program, tailored to the needs of Dutch adolescents from low socio-economic background. PMID:17173701

Using qualitative and quantitative methods to evaluate small-scale disease management pilot programs.

PubMed

Esposito, Dominick; Taylor, Erin Fries; Gold, Marsha

2009-02-01

Interest in disease management programs continues to grow as managed care plans, the federal and state governments, and other organizations consider such efforts as a means to improve health care quality and reduce costs. These efforts vary in size, scope, and target population. While large-scale programs provide the means to measure impacts, evaluation of smaller interventions remains valuable as they often represent the early planning stages of larger initiatives. This paper describes a multi-method approach for evaluating small interventions that sought to improve the quality of care for Medicaid beneficiaries with multiple chronic conditions. Our approach relied on quantitative and qualitative methods to develop a complete understanding of each intervention. Quantitative data in the form of both process measures, such as case manager contacts, and outcome measures, such as hospital use, were reported and analyzed. Qualitative information was collected through interviews and the development of logic models to document the flow of intervention activities and how they were intended to affect outcomes. The logic models helped us to understand the underlying reasons for the success or lack thereof of each intervention. The analysis provides useful information on several fronts. First, qualitative data provided valuable information about implementation. Second, process measures helped determine whether implementation occurred as anticipated. Third, outcome measures indicated the potential for favorable results later, possibly suggesting further study. Finally, the evaluation of qualitative and quantitative data in combination helped us assess the potential promise of each intervention and identify common themes and challenges across all interventions.

Travtek Global Evaluation And Executive Summary

DOT National Transportation Integrated Search

2000-09-01

Several measures have been carried out in the Long Term Pavement Performance (LTPP) Program to ensure uniform distress data collection and interpretation. However, no systematic evaluation has been done to quantify the variability (bias and precision...

Evaluation of the TEAM Train-the-Trainer program

DOT National Transportation Integrated Search

1992-05-22

Author's abstract: The objective of this study was to evaluate the effectiveness of Techniques for Effective Alcohol Management (TEAM) Train-the-Trainer workshops. Effectiveness was measured in terms of the success facility representatives had, after...

Physiotherapists supporting self-management through health coaching: a mixed methods program evaluation.

PubMed

Dufour, Sinéad Patricia; Graham, Shane; Friesen, Josh; Rosenblat, Michael; Rous, Colin; Richardson, Julie

2015-01-01

To evaluate a program in support of chronic disease self-management (CDSM) that is founded on a health coaching (HC) approach, includes supervised exercise and mindfulness-based stress reduction components and is delivered within a private practice physiotherapy setting. An explanatory mixed method design, framed by theory-based program evaluation, was employed to evaluate an eight-week group-based program. Standardized self-rated and performance measures were evaluated pre- and post intervention. Additionally, participant focus groups were conducted following the intervention period. An inductive thematic approach was undertaken to analyze the qualitative data. Seventeen participants (N = 17) completed the study. Improvements were seen in both self-report and performance outcomes. Participants explained how and why they felt the program was beneficial. Six themes were generated: (1) group dynamic; (2) learning versus doing; (3) holism and comprehensive care; (4) self-efficacy and empowerment; (5) previous solutions versus new management strategies; and (6) healthcare provider support. This study established that a group program in support of CDSM founded on a HC approach demonstrated potential value from participants as well as favorable outcomes. A pragmatic randomized control trial is required to determine efficacy of this intervention.

Evaluation tools for undergraduate program planning in times of financial austerity.

PubMed

O'Palka, J; Harris, P R

1990-05-01

This article describes the administration and outcome of two evaluation tools developed by faculty of the dietetic program for ongoing assessment of a Plan IV dietetic education program over a 4-year period. Interns and internship directors were asked to evaluate the level of skills and knowledge base of interns compared with their internship classmates. Interns were also asked to rate the effectiveness of undergraduate course assignments and activities for internship preparation. As a result of the surveys, the home economics core course work was deleted, credits were shifted from food science to nutrient metabolism courses, and credits in clinical nutrition were increased. Projects in clinical nutrition and food systems management were modified. The surveys justified program requirements and utilization of resources, and provided an additional, effective measure of faculty competence.

Development and implementation of a competency-based clinical evaluation tool for midwifery education.

PubMed

Woeber, Kate

2018-07-01

The learning goals and evaluation strategies of competency-based midwifery programs must be explicit and well-defined. In the US, didactic learning is evaluated through a standardized certification examination, but standardized clinical competence evaluation is lacking. The Midwifery Competency Assessment Tool (MCAT) has been adapted from the International Confederation of Midwives' (ICM) "Essential Competencies" and from the American College of Nurse-Midwives' (ACNM) "Core Competencies", with student self-evaluation based on Benner's Novice-to-Expert theory. The MCAT allows for the measurement and monitoring of competence development in all domains of full-scope practice over the course of the midwifery program. Strengths of the MCAT are that it provides clear learning goals and performance evaluations for students, ensures and communicates content mapping across a curriculum, and highlights strengths and gaps in clinical opportunities at individual clinical sites and for entire programs. Challenges of the MCAT lie in balancing the number of competency items to be measured with the tedium of form completion, in ensuring the accuracy of student self-evaluation, and in determining "adequate" competence achievement when particular clinical opportunities are limited. Use of the MCAT with competency-based clinical education may facilitate a more standardized approach to clinical evaluation, as well as a more strategic approach to clinical site development and use. Copyright © 2018 Elsevier Ltd. All rights reserved.

Systematic Review of Empirically Evaluated School-Based Gambling Education Programs.

PubMed

Keen, Brittany; Blaszczynski, Alex; Anjoul, Fadi

2017-03-01

Adolescent problem gambling prevalence rates are reportedly five times higher than in the adult population. Several school-based gambling education programs have been developed in an attempt to reduce problem gambling among adolescents; however few have been empirically evaluated. The aim of this review was to report the outcome of studies empirically evaluating gambling education programs across international jurisdictions. A systematic review following guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement searching five academic databases: PubMed, Scopus, Medline, PsycINFO, and ERIC, was conducted. A total of 20 papers and 19 studies were included after screening and exclusion criteria were applied. All studies reported intervention effects on cognitive outcomes such as knowledge, perceptions, and beliefs. Only nine of the studies attempted to measure intervention effects on behavioural outcomes, and only five of those reported significant changes in gambling behaviour. Of these five, methodological inadequacies were commonly found including brief follow-up periods, lack of control comparison in post hoc analyses, and inconsistencies and misclassifications in the measurement of gambling behaviour, including problem gambling. Based on this review, recommendations are offered for the future development and evaluation of school-based gambling education programs relating to both methodological and content design and delivery considerations.

Utilization of telemedicine in the U.S. military in a deployed setting.

PubMed

Hwang, Jane S; Lappan, Charles M; Sperling, Leonard C; Meyerle, Jon H

2014-11-01

A retrospective evaluation of the Department of Defense teledermatology consultation program from 2004 to 2012 was performed, focusing on clinical application and outcome measures such as consult volume, response time, and medical evacuation status. A retrospective review of the teledermatology program between 2004 and 2012 was evaluated based on defined outcome measures. In addition, 658 teledermatology cases were reviewed to assess how the program was utilized by health care providers from 2011 to 2012. As high as 98% of the teledermatology consults were answered within 24 hours, and 23% of consults within 1 hour. The most common final diagnoses included eczematous dermatitis, contact dermatitis, and evaluation for nonmelanoma skin cancer. The most common medications recommended included topical corticosteroids, oral antibiotics, antihistamines, and emollients. Biopsy was most commonly recommended for further evaluation. Following teleconsultation, 46 dermatologic evacuations were "avoided" as the patient was not evacuated based on the consultants' recommendation. Consultants' recommendations to the referring provider "facilitated" 41 evacuations. Telemedicine in the U.S. military has provided valuable dermatology support to providers in remote locations by delivering appropriate and timely consultation for military service members and coalition partners. In addition to avoiding unnecessary medical evacuations, the program facilitated appropriate evacuations that may otherwise have been delayed. Reprint & Copyright © 2014 Association of Military Surgeons of the U.S.

Evaluation methodologies for an advanced information processing system

NASA Technical Reports Server (NTRS)

Schabowsky, R. S., Jr.; Gai, E.; Walker, B. K.; Lala, J. H.; Motyka, P.

1984-01-01

The system concept and requirements for an Advanced Information Processing System (AIPS) are briefly described, but the emphasis of this paper is on the evaluation methodologies being developed and utilized in the AIPS program. The evaluation tasks include hardware reliability, maintainability and availability, software reliability, performance, and performability. Hardware RMA and software reliability are addressed with Markov modeling techniques. The performance analysis for AIPS is based on queueing theory. Performability is a measure of merit which combines system reliability and performance measures. The probability laws of the performance measures are obtained from the Markov reliability models. Scalar functions of this law such as the mean and variance provide measures of merit in the AIPS performability evaluations.

Chapter 23: Combined Heat and Power Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kurnik, Charles W.; Simons, George; Barsun, Stephan

The main focus of most evaluations is to determine the energy-savings impacts of the installed measure. This protocol defines a combined heat and power (CHP) measure as a system that sequentially generates both electrical energy and useful thermal energy from one fuel source at a host customer's facility or residence. This protocol is aimed primarily at regulators and administrators of ratepayer-funded CHP programs; however, project developers may find the protocol useful to understand how CHP projects are evaluated.

The evolution of Rare Pride: using evaluation to drive adaptive management in a biodiversity conservation organization.

PubMed

Jenks, Brett; Vaughan, Peter W; Butler, Paul J

2010-05-01

Rare Pride is a social marketing program that stimulates human behavior change in order to promote biodiversity conservation in critically threatened regions in developing countries. A series of formal evaluation studies, networking strategies, and evaluative inquiries have driven a 20-year process of adaptive management that has resulted in extensive programmatic changes within Pride. This paper describes the types of evaluation that Rare used to drive adaptive management and the changes it caused in Pride's theory-of-change and programmatic structure. We argue that (a) qualitative data gathered from partners and staff through structured interviews is most effective at identifying problems with current programs and procedures, (b) networking with other organizations is the most effective strategy for learning of new management strategies, and (c) quantitative data gathered through surveys is effective at measuring program impact and quality. Adaptive management has allowed Rare to increase its Pride program from implementing about two campaigns per year in 2001 to more than 40 per year in 2009 while improving program quality and maintaining program impact. Copyright 2009 Elsevier Ltd. All rights reserved.

A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming.

PubMed

Hansen, William B; Derzon, James H; Reese, Eric L

2014-06-01

We propose a method for creating groups against which outcomes of local pretest-posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness. © The Author(s) 2012.

Critical thinking as an educational outcome: an evaluation of current tools of measurement.

PubMed

Adams, M H; Whitlow, J F; Stover, L M; Johnson, K W

1996-01-01

Critical thinking, an outcome criterion of the National League for Nursing and the Council of Baccalaureate and Higher Degree Programs, is an abstract skill difficult to measure. The authors provide a comprehensive review of four instruments designed to measure critical thinking and summarize research in which the tools were used. Analysis of this information will empower nursing faculty members to select a critical-thinking instrument that is individualized to the needs of their respective nursing programs.

Evaluation of a Statewide HIV-HCV-STD Online Clinical Education Program by Healthcare Providers - A Comparison of Nursing and Other Disciplines.

PubMed

Wang, Dongwen; Luque, Amneris E

2016-01-01

The New York State HIV-HCV-STD Clinical Education Initiative (CEI) has developed a large repository of online resources and disseminated them to a wide range of healthcare providers. To evaluate the CEI online education program and in particular to compare the self-reported measures by clinicians from different disciplines, we analyzed the data from 1,558 course completions in a study period of three months. The results have shown that the overall evaluations by the clinicians were very positive. Meanwhile, there were significant differences across the clinical disciplines. In particular, physicians and nurse practitioners were the most satisfied. In contrast, pharmacists and case/care managers recorded lower than average responses. Nurses and counselors had mixed results. Nurse practitioners' responses were very similar to physicians on most measures, but significantly different from nurses in many aspects. For more effective knowledge dissemination, online education programs should consider the unique needs by clinicians from specific disciplines.

The Effect of Reflective Garden Walking on Adults With Increased Levels of Psychological Stress.

PubMed

McCaffrey, Ruth; Liehr, Patricia

2016-06-01

The purpose of this evaluation was to determine the benefits of a reflective garden walking program on adults with increased levels of psychological stress. Outcomes measured included levels of hopefulness, personal growth, and quality of life. The evaluation used a one-group, pretest-posttest to determine the success of the Stroll for Well-Being to assist participants to overcome psychological stressors. Participants were recruited through local support groups. All participants signed informed consent to participate in the study program. A total of 195 participants completed the 6-week program, attended all meetings, and completed all measurement tools. All of the outcome measures statistically improved on the posttest compared to the pretest scores. The outcome measure that had the largest change in mean score was the Personal Growth Scale. Holistic nursing as a specialty should continue to explore the use of green spaces and nature on patients. More research is needed to increase the amount of evidence regarding spending time in nature and using reflection and journaling as a tool to reconnect with the natural environment. © The Author(s) 2015.

Quality measures and pediatric radiology: suggestions for the transition to value-based payment.

PubMed

Heller, Richard E; Coley, Brian D; Simoneaux, Stephen F; Podberesky, Daniel J; Hernanz-Schulman, Marta; Robertson, Richard L; Donnelly, Lane F

2017-06-01

Recent political and economic factors have contributed to a meaningful change in the way that quality in health care, and by extension value, are viewed. While quality is often evaluated on the basis of subjective criteria, pay-for-performance programs that link reimbursement to various measures of quality require use of objective and quantifiable measures. This evolution to value-based payment was accelerated by the 2015 passage of the Medicare Access and CHIP (Children's Health Insurance Program) Reauthorization Act (MACRA). While many of the drivers of these changes are rooted in federal policy and programs such as Medicare and aimed at adult patients, the practice of pediatrics and pediatric radiology will be increasingly impacted. This article addresses issues related to the use of quantitative measures to evaluate the quality of services provided by the pediatric radiology department or sub-specialty section, particularly as seen from the viewpoint of a payer that may be considering ways to link payment to performance. The paper concludes by suggesting a metric categorization strategy to frame future work on the subject.

Effectiveness of a Core-Competency-based Program on Residents' Learning and Experience.

PubMed

Charles, Lesley; Triscott, Jean; Dobbs, Bonnie; Tian, Peter George; Babenko, Oksana

2016-06-01

The Care of the Elderly (COE) Diploma Program is a six-to-twelve-month enhanced skills program taken after two years of core residency training in Family Medicine. In 2010, we developed and implemented a core-competency-based COE Diploma program (CC), in lieu of one based on learning objectives (LO). This study assessed the effectiveness of the core-competency-based program on residents' learning and their training experience as compared to residents trained using learning objectives. The data from the 2007-2013 COE residents were used in the study, with nine and eight residents trained in the LO and CC programs, respectively. Residents' learning was measured using preceptors' evaluations of residents' skills/abilities throughout the program (118 evaluations in total). Residents' rating of training experience was measured using the Graduate's Questionnaire which residents completed after graduation. For residents' learning, overall, there was no significant difference between the two programs. However, when examined as a function of the four CanMEDS roles, there were significant increases in the CC residents' scores for two of the CanMEDS roles: Communicator/Collaborator/Manager and Scholar compared to residents in the LO program. With respect to residents' training experience, seven out of ten program components were rated by the CC residents higher than by the LO residents. The implementation of a COE CC program appears to facilitate resident learning and training experience.

Shuttle atmospheric lidar research program

NASA Technical Reports Server (NTRS)

1979-01-01

The Shuttle atmospheric lidar program is discussed in relation to an understanding of the processes governing the Earth's atmosphere and in the capacity to evaluate the atmospheric susceptibility to manmade and natural perturbations. Applications of the lidar which are discussed are the determination of the global flow of water vapor and pollutants in the troposphere, improvement of chemical and transport models of the stratosphere and mesosphere, evaluation of radiative models of the atmosphere, investigation of chemistry and transport of thermospheric atomic species, and investigation of magnetospheric aspects of sun/weather relationships. The features of the lidar measurements discussed are the high spatial resolution, control of the source wavelength and intensity, and high measurement specificity.

Deuteron nuclear data for the design of accelerator-based neutron sources: Measurement, model analysis, evaluation, and application

NASA Astrophysics Data System (ADS)

Watanabe, Yukinobu; Kin, Tadahiro; Araki, Shouhei; Nakayama, Shinsuke; Iwamoto, Osamu

2017-09-01

A comprehensive research program on deuteron nuclear data motivated by development of accelerator-based neutron sources is being executed. It is composed of measurements of neutron and gamma-ray yields and production cross sections, modelling of deuteron-induced reactions and code development, nuclear data evaluation and benchmark test, and its application to medical radioisotopes production. The goal of this program is to develop a state-of-the-art deuteron nuclear data library up to 200 MeV which will be useful for the design of future (d,xn) neutron sources. The current status and future plan are reviewed.

Evaluation and characterization of the methane-carbon dioxide decomposition reaction

NASA Technical Reports Server (NTRS)

Davenport, R. J.; Schubert, F. H.; Shumar, J. W.; Steenson, T. S.

1975-01-01

A program was conducted to evaluate and characterize the carbon dioxide-methane (CO2-CH4) decomposition reaction, i.e., CO2 + CH4 = 2C + 2H2O. The primary objective was to determine the feasibility of applying this reaction at low temperatures as a technique for recovering the oxygen (O2) remaining in the CO2 which exits mixed with CH4 from a Sabatier CO2 reduction subsystem (as part of an air revitalization system of a manned spacecraft). A test unit was designed, fabricated, and assembled for characterizing the performance of various catalysts for the reaction and ultraviolet activation of the CH4 and CO2. The reactor included in the test unit was designed to have sufficient capacity to evaluate catalyst charges of up to 76 g (0.17 lb). The test stand contained the necessary instrumentation and controls to obtain the data required to characterize the performance of the catalysts and sensitizers tested: flow control and measurement, temperature control and measurement, product and inlet gas analysis, and pressure measurement. A product assurance program was performed implementing the concepts of quality control and safety into the program effort.

Recent Advances in Resonance Region Nuclear Data Measurements and Analyses for Supporting Nuclear Energy Applications

NASA Astrophysics Data System (ADS)

Dunn, Michael

2008-10-01

For over 30 years, the Oak Ridge National Laboratory (ORNL) has performed research and development to provide more accurate nuclear cross-section data in the resonance region. The ORNL Nuclear Data (ND) Program consists of four complementary areas of research: (1) cross-section measurements at the Oak Ridge Electron Linear Accelerator; (2) resonance analysis methods development with the SAMMY R-matrix analysis software; (3) cross-section evaluation development; and (4) cross-section processing methods development with the AMPX software system. The ND Program is tightly coupled with nuclear fuel cycle analyses and radiation transport methods development efforts at ORNL. Thus, nuclear data work is performed in concert with nuclear science and technology needs and requirements. Recent advances in each component of the ORNL ND Program have led to improvements in resonance region measurements, R-matrix analyses, cross-section evaluations, and processing capabilities that directly support radiation transport research and development. Of particular importance are the improvements in cross-section covariance data evaluation and processing capabilities. The benefit of these advances to nuclear science and technology research and development will be discussed during the symposium on Nuclear Physics Research Connections to Nuclear Energy.

Caregiver education in Parkinson's disease: formative evaluation of a standardized program in seven European countries.

PubMed

A'Campo, L E I; Spliethoff-Kamminga, N G A; Macht, M; Roos, R A C

2010-02-01

The formative evaluation of a standardized psychosocial education program for patients with Parkinson's disease (PD) and their caregivers. The results of the participation of the caregivers are presented next to the data of the patients. Caregivers (n = 137) and patients with PD (n = 151) participated in the 8-week program in separate groups. Measurements were performed on psychosocial problems (BELA-P/A-k), health state (EQ-5D VAS), quality of life (PDQ-39) and depression (SDS) 1 week before and 1 week after the program. Participants rated their mood on a visual analogue scale before and after each session, and they filled in an evaluation questionnaire after the last session. Scores on the BELA-P/A-k improved significantly on the 'bothered by scale' as well as the 'need for help scale'. No improvements were found on EQ-5D VAS, PDQ-39 and SDS. Mood ratings improved significantly after each session. Most participants evaluated the program as positive. Feedback led to improvements in the program, which are incorporated in a final manual. The program was feasible to run in the different countries. This exploratory study led to improvements in the program and recommendations for further research. A study on the effectiveness of the program is the next step.

The effects of a prelicensure extern program and nurse residency program on new graduate outcomes and retention.

PubMed

Friday, Laura; Zoller, James S; Hollerbach, Ann D; Jones, Katherine; Knofczynski, Greg

2015-01-01

Organizations are looking to new graduate nurses to fill expected staffing shortages over the next decade. Creative and effective onboarding programs will determine the success or failure of these graduates as they transition from student to professional nurse. This longitudinal quantitative study with repeated measures used the Casey-Fink Graduate Nurse Experience Survey to investigate the effects of offering a prelicensure extern program and postlicensure residency program on new graduate nurses and organizational outcomes versus a residency program alone. Compared with the nurse residency program alone, the combination of extern program and nurse residency program improved neither the transition factors most important to new nurse graduates during their first year of practice nor a measure important to organizations, retention rates. The additional cost of providing an extern program should be closely evaluated when making financially responsible decisions.

Evaluating Mission Drift in Microfinance: Lessons for Programs with Social Mission

ERIC Educational Resources Information Center

Hishigsuren, Gaamaa

2007-01-01

The article contributes to a better understanding of implications of scaling up on the social mission of microfinance programs. It proposes a methodology to measure the extent, if any, to which a microfinance program with a poverty alleviation mission drifts away from its mission during rapid scaling up and presents findings from a field research…

Youth-Initiated HIV Risk and Substance Use Prevention Program.

ERIC Educational Resources Information Center

Goggin, K.; Metcalf, K.; Wise, D.; Kennedy, S.; Murray, T.; Burgess, D.; Reese-Smith, J.; Terhune, N.; Broadus, K.; Downes, A.; Buckendahl, H.

This study evaluates the first year of a novel HIV and substance use prevention program for inner city youth (Offering New Youth eXperiences--ONYX). Baseline and follow-up measures of knowledge, attitudes, and risk behaviors were administered seven months apart to 441 youth participating in the ONYX program. Youth (n=71) who provided data at both…

Measuring the Impact of a Pilot Geospatial Technology Apprenticeship Program for the Department of Labor

ERIC Educational Resources Information Center

Gaudet, Cyndi; Annulis, Heather; Kmiec, John

2010-01-01

The Geospatial Technology Apprenticeship Program (GTAP) pilot was designed as a replicable and sustainable program to enhance workforce skills in geospatial technologies to best leverage a $30 billion market potential. The purpose of evaluating GTAP was to ensure that investment in this high-growth industry was adding value. Findings from this…

Development of a Criterion-Referenced, Performance-Based Assessment of Reading Comprehension in a Whole Literacy Program.

ERIC Educational Resources Information Center

Tibbetts, Katherine A.; And Others

This paper describes the development of a criterion-referenced, performance-based measure of third grade reading comprehension. The primary purpose of the assessment is to contribute unique and valid information for use in the formative evaluation of a whole literacy program. A secondary purpose is to supplement other program efforts to…

Developing a New System to Measure Outcomes in a Service Coordination Program for Youth with Severe Emotional Disturbance

ERIC Educational Resources Information Center

Shannon, Lisa M.; Walker, Robert; Blevins, Michele

2009-01-01

This paper presents information on re-developing an outcome evaluation for a state-funded program providing service coordination utilizing wraparound to youth with severe emotional disturbance (SED) and their families. Originally funded by the Robert Wood Johnson Foundation, the Kentucky IMPACT program has existed statewide since 1990. Changing…

Efficiency in Vocational Rehabilitation Program Service Delivery: The Impact of Socioeconomic Context

ERIC Educational Resources Information Center

Gere, Bryan O.; Burnett, Royce D.; Flowers, Carl R.; Akaaboune, Ouadie

2017-01-01

Background: State-federal (VR) program efficiency is the focus of empirical research because of increases in the magnitude and types of program requests, possibly funding cuts and class for models to more appropriately measure and evaluate performance. Objective: The purpose of this study was to examine the impact socioeconomic diversity has on…

Skylab experiments. Volume 6: Mechanics. [Skylab program accomplishments for high school level education

NASA Technical Reports Server (NTRS)

1973-01-01

Skylab program activities are presented in a form adapted to instruction of high school students. The overall goals of the program are discussed. The specific accomplishments of the mechanics investigations are described. The subjects involved are as follows: (1) evaluation of mobility aids, (2) mass measurement devices, and (3) space guidance crew/vehicle disturbances.

In situ correlative measurements for the ultraviolet differential absorption lidar and the high spectral resolution lidar air quality remote sensors: 1980 PEPE/NEROS program

NASA Technical Reports Server (NTRS)

Gregory, G. L.; Beck, S. M.; Mathis, J. J., Jr.

1981-01-01

In situ correlative measurements were obtained with a NASA aircraft in support of two NASA airborne remote sensors participating in the Environmental Protection Agency's 1980persistent elevated pollution episode (PEPE) and Northeast regional oxidant study (NEROS) field program in order to provide data for evaluating the capability of two remote sensors for measuring mixing layer height, and ozone and aerosol concentrations in the troposphere during the 1980 PEPE/NEROS program. The in situ aircraft was instrumented to measure temperature, dewpoint temperature, ozone concentrations, and light scattering coefficient. In situ measurements for ten correlative missions are given and discussed. Each data set is presented in graphical and tabular format aircraft flight plans are included.

The 'Healthy Dads, Healthy Kids' community effectiveness trial: study protocol of a community-based healthy lifestyle program for fathers and their children

PubMed Central

2011-01-01

Background The 'Healthy Dads, Healthy Kids' program was designed to help overweight fathers lose weight and positively influence the health behaviors of their children. The aim of the current study was to evaluate the previously established program in a community setting, in a large effectiveness trial. Methods/Design The Healthy Dads, Healthy Kids community trial consists of three stages: (i) Stage 1 - program refinement and resource development (ii) Stage 2 - community randomized controlled trial (iii) Stage 3 - community effectiveness trial. The program will be evaluated in five Local Government Areas in the Hunter Valley Region of NSW, Australia. For the community randomized controlled trial, 50 overweight/obese men (aged 18-65 years) from one Local Government Area with a child aged between 5-12 years of age will be recruited. Families will be randomized to either the program or a 6-month wait-list control group. Fathers and their children will be assessed at baseline, post-intervention (3-months) and 6-months. Inclusion criteria are: body mass index 25-40 kg/m2; no participation in other weight loss programs during the study; pass a health-screening questionnaire; and access to a computer with Internet facilities. In the community trial, the program will be evaluated using a non-randomized, prospective design in five Local Government Areas. The exclusion criteria is body mass index < 25 kg/m2 or lack of doctor's approval. Measures will be collected at baseline, 3-, 6- and 12-months. The program involves fathers attending seven face-to-face group sessions (three with children) over 3-months. Measures: The primary outcome is fathers' weight. Secondary outcomes for both fathers and children include: waist circumference, blood pressure, resting heart rate, physical activity, sedentary behaviors and dietary intake. Father-only measures include portion size, alcohol consumption, parenting for physical activity and nutrition and parental engagement. Process evaluation will determine the fidelity, dose (delivered and received), reach, recruitment and context of the program. Discussion As a unique approach to reducing obesity prevalence in men and improving lifestyle behaviours in children, our findings will provide important evidence relating to the translation of Healthy Dads, Healthy Kids, which will enable it to be delivered on a larger scale. Trial registration Australia and New Zealand Clinical Trials Register (ANZCTR): ACTRN12610000608066 PMID:22099889

State-of-the-science of patient navigation as a strategy for enhancing minority clinical trial accrual.

PubMed

Ghebre, Rahel G; Jones, Lovell A; Wenzel, Jennifer A; Martin, Michelle Y; Durant, Raegan W; Ford, Jean G

2014-04-01

Patient navigation programs are emerging that aim to address disparities in clinical trial participation among medically underserved populations, including racial/ethnic minorities. However, there is a lack of consensus on the role of patient navigators within the clinical trial process as well as outcome measures to evaluate program effectiveness. A review of the literature was conducted of PubMed, Medline, CINHAL, and other sources to identify qualitative and quantitative studies on patient navigation in clinical trials. The search yielded 212 studies, of which only 12 were eligible for this review. The eligible studies reported on the development of programs for patient navigation in cancer clinical trials, including training and implementation among African Americans, American Indians, and Native Hawaiians. A low rate of clinical trial refusal (range, 4%-6%) was reported among patients enrolled in patient navigation programs. However, few studies reported on the efficacy of patient navigation in increasing clinical treatment trial enrollment. Outcome measures are proposed to assist in developing and evaluating the efficacy and/or effectiveness of patient navigation programs that aim to increase participation in cancer clinical trials. Future research is needed to evaluate the efficacy of patient navigators in addressing barriers to clinical trial participation and increasing enrollment among medically underserved cancer patients. © 2014 American Cancer Society.

State-of-the-Science of Patient Navigation as a Strategy for Enhancing Minority Clinical Trial Accrual

PubMed Central

Ghebre, Rahel G.; Jones, Lovell A.; Wenzel, Jennifer; Martin, Michelle Y.; Durant, Raegan; Ford, Jean G.

2014-01-01

Background Patient navigation programs are emerging, that aim to address disparities in clinical trial participation among medically underserved populations, including racial/ethnic minorities. However, there is a lack of consensus on the role of patient navigators within the clinical trial process, as well as outcome measures to evaluate program effectiveness. Methods A review of the literature was conducted of PubMed, Medline, CINHAL, and other sources to identify qualitative and quantitative studies on patient navigation in clinical trials. The search yielded 212 studies, of which only 12 were eligible for this review. Results The eligible studies reported on development of programs for patient navigation in cancer clinical trials, including training and implementation among African American, American Indian and Native Hawaiians. Low clinical trial refusal, 4% to 6%, was reported among patients enrolled in patient navigation program. However, few studies reported on the efficacy of patient navigation on increasing clinical treatment trial enrollment. Conclusion Outcome measures are proposed to assist in developing and evaluating the efficacy and/or effectiveness of patient navigation programs that aim to increase participation in cancer clinical trials. Future research is needed to evaluate the efficacy of patient navigators in addressing barriers to clinical trial participation and increasing enrollment among medically underserved cancer patients. PMID:24643650

A Preliminary Evaluation of a School-Based Media Education and Reduction Intervention.

PubMed

Bickham, David S; Hswen, Yulin; Slaby, Ronald G; Rich, Michael

2018-06-01

While media education and reduction programs have been proposed to prevent adverse health and academic outcomes related to heavy electronic media use among school-aged children, few have been formally piloted and evaluated. We used a quasi-experimental design to evaluate the effectiveness of Take the Challenge (TtC), a school-based media education/reduction program for the primary prevention of sleep deprivation, dysfunctional social-emotional behaviors, and poor academic performance. Sixth- to eighth-grade students at a rural Midwestern U.S. middle school received the TtC program, while a similar school in the same district served as the comparison group. Health-related and academic measures were collected from students and teachers at both schools before and after the intervention. The primary outcome measure was student-reported electronic media use (television, video games, Internet). Secondary measures included student health behaviors (student-reported sleep, exercise, and outdoor play) and academic activities (teacher-reported homework and classroom performance). Compared to the comparison group, students receiving TtC slept more and reduced television viewing, background television time, after-school video gaming, and weekend Internet use. Teachers reported increases in the extent to which TtC students completed homework assignments and stayed on task in the classroom. Well-designed school-based programs such as TtC can reduce electronic media use among middle-school children and improve related health and academic outcomes.

Measurement of Quality of Nursing Practice in Congenital Cardiac Care.

PubMed

Connor, Jean Anne; Mott, Sandra; Green, Angela; Larson, Carol; Hickey, Patricia

2016-03-01

The impact of nursing care on patients' outcomes has been demonstrated in adult and pediatric settings. However, limited attention has been given to standardized measurement of pediatric nursing care. A collaborative group, the Consortium for Congenital Cardiac Care Measurement of Nursing Practice, was formed to address this gap. The purpose of this study was to assess the current state of measurement of the quality of pediatric cardiovascular nursing in freestanding children's hospitals across the United States. A qualitative descriptive design was used to assess the state of measurement of nursing care from the perspective of experts in pediatric cardiovascular nursing. Nurse leaders from 20 sites participated in audiotaped phone interviews. The data were analyzed by using conventional content analysis. Each level of data coding was increasingly comprehensive. Guided by Donabedian's quality framework of structure, process, and outcome, 2 encompassing patterns emerged: (1) structure and process of health care delivery and (2) structure and process of evaluation of care. Similarities in the structure of health care delivery included program expansion and subsequent hiring of nurses with a bachelor of science in nursing and experienced nurses to provide safety and optimal outcomes for patients. Programs varied in how they evaluated care in terms of structure, measurement, collection and dissemination of data. External factors and response to internal processes of health care delivery were similar in different programs; evaluation was more varied. Seven opportunities for measurement that address both structure and process of nursing care were identified to be developed as benchmarks. ©2016 American Association of Critical-Care Nurses.

A STUDY TO EVALUATE THE LEVELS OF DIOXIN-LIKE ...

EPA Pesticide Factsheets

The Environmental Protection Agency (EPA), in cooperation with USDA and the US Food and Drug Administration (FDA), has undertaken a program to study the presence of dioxin-like compounds in animal feeds. Two phases of this program have been completed, and this paper reports on the third phase. The first phase was a study on the mass balance of dioxins in lactating cows. The objective of that study was to quantify the role feeds play in total dairy cow exposures. The second phase of the program involved the collections and measurement of dioxins in minor feed components. Dioxins in specific targeted animal feed components of interest, including animal and plant byproducts were measured. The third phase involved sampling of total mixed dairy feeds and feed components from dairy facilities around the United States, and measuring for dioxin-like compounds. Lorber, M; Ferrario, J; Byrne, C; Greene, C; Cyrus, A. 2004. A Study to Evaluate the Levels of Dioxin-Like Compounds in Dairy Feeds in the United States. Organohalogen Compounds 66: 1958-1965. journal articles

Measuring Outcomes in a Community Resilience Program: A New Metric for Evaluating Results at the Household Level.

PubMed

Eisenman, David P; Adams, Rachel M; Rivard, Helene

2016-09-28

Community resilience programs require metrics for evaluation but none exist for measuring outcomes at the household and neighborhood level. We develop and describe a new index, the LACCDR index of community resilience, to examine how resilience varied across communities at baseline, prior to implementation of the Los Angeles County Community Disaster Resilience Project (LACCDR). We surveyed 4700 adult residents in the sixteen LACCDR communities in English, Spanish and Korean. Each of the survey domains were selected a priori as outcome indicators aligned to the theoretical levers of community resilience. Survey questions were drawn and adapted from published studies and national surveys. Factor analysis demonstrated five separate factors composed from 18 items and explaining 46.7% of the variance. The factors were characterized as community engagement, emergency supplies, communication with neighbors, civic engagement, and collective efficacy. Baseline results for the 16 communities are provided. We conclude that the LACCDR community resilience index can be used to measure resilience program outcomes at the neighborhood and household levels.

Impact of an Activity-Based Program on Health, Quality of Life, and Occupational Performance of Women Diagnosed With Cancer.

PubMed

Maher, Colleen; Mendonca, Rochelle J

We evaluated the impact of a 1-wk activity program on the health, quality of life (QOL), and occupational performance of community-living women diagnosed with cancer. A one-group pretest-posttest repeated-measures design was used. Participants completed a functional health measure (36-Item Short Form Health Survey [SF-36]), a QOL measure (World Health Organization Quality of Life-Brief version [WHOQOL-BREF]), and an occupational performance and satisfaction measure (Canadian Occupational Performance Measure [COPM]) before and 6 wk after program completion. The COPM was also administered on Day 5. Paired t tests for the SF-36 and WHOQOL-BREF showed no significant differences, except for the WHOQOL-BREF's Social Relationships subscale (p < .008). Repeated-measures analyses of variance showed a significant difference in COPM performance and satisfaction scores (p < .001). The activity program effectively improved occupational performance and satisfaction and social relationships of community-living women diagnosed with cancer. Copyright © 2018 by the American Occupational Therapy Association, Inc.

National Weatherization Assistance Program Characterization - Describing the Pre-ARRA Progam

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bensch, Ingo; Keene, Ashleigh; Cowan, Claire

2014-09-01

This report characterizes the Department of Energy s Weatherization Assistance Program (WAP) as it was administered in Program Year 2008. WAP has supported energy efficiency improvements to the homes of low-income households in the United States since 1976. The program provides grants, guidance, and other support to grantees: weatherization programs administered by each of the 50 states, the District of Columbia and some Native American tribes. Although there have been studies of some grantee-administered weatherization programs, the overall effectiveness of the national weatherization program has not been formally evaluated since Program Year 1989. Since that time, the program has evolvedmore » significantly, with an increased focus on baseload electric usage, continued evolution of diagnostic tools, new guidelines and best practices for heating-related measures, and adjustments in program rules. More recently, the program has also adjusted to large, temporary funding increases and changes in federal rules spurred by the American Recovery and Reinvestment Act (ARRA). Because the Weatherization Assistance Program of today is dramatically different from the one evaluated in 1989, DOE determined to undertake a new comprehensive evaluation of the national program. This new national evaluation is managed by Oak Ridge National Laboratory (ORNL). Under a competitive solicitation process, ORNL selected APPRISE, Inc., Blasnik & Associates, Dalhoff Associates and the Energy Center of Wisconsin to conduct the evaluation. The national evaluation comprises two independent evaluations. The first evaluation of which this report is a part focuses on Program Year 2008 (PY08). The second evaluation focuses on the ARRA-funded years of 2009 through 2011. This report, together with its companion the Eligible Population Study addresses specific program characterization goals established for the greater evaluation. The Energy Center led grantee and subgrantee data collection efforts, administering surveys to 51 grantees and 851 of the approximately 900 subgrantees that were slated to receive DOE weatherization funds in PY08. In all, seven different data collection instruments were used to gather the needed data two instruments for grantees and five for subgrantees. See Table 1 for a list of these survey instruments. These surveys were used to determine, among other things: Structure and funding of weatherization programs Training and staff development of service providers How weatherization services are delivered Clients served« less

Research performance measures and program evaluation.

DOT National Transportation Integrated Search

2003-10-01

The Iowa Department of Transportation hosted a Peer Exchange on October 8-9, 2003. The : purpose of this exchange was to give research managers from several state departments of : transportation the opportunity to discuss research performance measure...

Evaluating Internal Communication: The ICA Communication Audit.

ERIC Educational Resources Information Center

Goldhaber, Gerald M.

1978-01-01

The ICA Communication Audit is described in detail as an effective measurement procedure that can help an academic institution to evaluate its internal communication system. Tools, computer programs, analysis, and feedback procedures are described and illustrated. (JMF)

Teaching Assistant Professional Development in Biology: Designed for and Driven by Multidimensional Data

PubMed Central

Long, Tammy M.; Ebert-May, Diane

2014-01-01

Graduate teaching assistants (TAs) are increasingly responsible for instruction in undergraduate science, technology, engineering, and mathematics (STEM) courses. Various professional development (PD) programs have been developed and implemented to prepare TAs for this role, but data about effectiveness are lacking and are derived almost exclusively from self-reported surveys. In this study, we describe the design of a reformed PD (RPD) model and apply Kirkpatrick's Evaluation Framework to evaluate multiple outcomes of TA PD before, during, and after implementing RPD. This framework allows evaluation that includes both direct measures and self-reported data. In RPD, TAs created and aligned learning objectives and assessments and incorporated more learner-centered instructional practices in their teaching. However, these data are inconsistent with TAs’ self-reported perceptions about RPD and suggest that single measures are insufficient to evaluate TA PD programs. PMID:26086654

Opening the Conversation on REU Assessment and Evaluation

NASA Astrophysics Data System (ADS)

Pressley, S. N.; LeBeau, J. E.

2015-12-01

Project evaluation is a key component to ensuring success of any Research Experience for Undergraduates (REU) program. The Washington State University (WSU) Regional Atmospheric Chemistry: State-of-the-art Measurement and Modeling in the Pacific Northwest REU Site employs a mixed method approach to determine what is working well and what can use improvement (formative evaluation) and to determine impact and effectiveness of the project in reaching the stated goals (summative evaluation). Quantitative data is collected via a pre-/post-test measuring participants' research self-efficacy (RSE), motivation, background information, extent of socialization, and their interpretation of the value of the REU experience. Qualitative data is gathered through individual interviews with the REU students (at the beginning and end of the program) and faculty mentors (at the end). Beginning interviews focus on expectations for the REU program and student backgrounds. End interviews focus on student RSE development, interpretations of their experience, and the value of the experience. Faculty mentors are interviewed to gather insight on student performance in the program and perspectives on the overall success of the program in meeting the proposed goals. Students are provided an opportunity to comment on the strengths and weaknesses of workshops, providing critical feedback to the particular instructor and enabling the faculty to modify the workshop content and activities in future years. Finally, research results are evaluated during the final poster presentation, and faculty are interviewed to report on their perception of how each student learned and gained knowledge during the program. To evaluate the retention of students in engineering and science and identify chosen career paths, a longitudinal survey was created and it is administered via email each year. Many REU programs also employ the Undergraduate Research Students Self-Assessment (URSSA) online tool designed for evaluating REU experiences (University of Colorado at Boulder). Information presented in this presentation will include a discussion of the current tools used as described above as well as a discussion of the pros/cons of using URSSA. One of the concerns with using URSSA in conjunction with the other assessment tools is "survey fatigue".

Checkmate: Capturing Gifted Students' Logical Thinking Using Chess.

ERIC Educational Resources Information Center

Rifner, Philip J.; Feldhusen, John F.

1997-01-01

Describes the use of chess instruction to develop abstract thinking skills and problem solving among gifted students. Offers suggestions for starting school chess programs, teaching and evaluating chess skills, and measuring the success of both student-players and the program in general. (PB)

77 FR 22798 - 21st Century Conservation Service Corps Advisory Committee

Federal Register 2010, 2011, 2012, 2013, 2014

2012-04-17

... accountability and performance evaluation criteria to measure success; (2) the development of certification... existing barriers to successful 21CSC program implementation; (4) identifying partnership opportunities... state, local, and tribal governments, to expand support for conservation corps programs, career training...